Why are my Go executable files so large? - <AUTHOR> https://www.cockroachlabs.com/blog/go-file-size/ <comment><USER2> > prior to 1.2, the Go linker was emitting a compressed line table, and the > program would decompress it upon initialization at run-time. in Go 1.2, a > decision was made to pre-expand the line table in the executable file into > its final format suitable for direct use at run-time, without an additional > decompression step.

This is a good choice I think and the author of the article missed the most important point - it uses less memory to have an uncompressed table.

This sounds paradoxical but if a table has to be expanded at runtime then it has to be loaded into memory.

However if a table is part of the executable, the OS won't even load it into memory unless it is used and will only page the bits into memory that are used.

You see the same effect when you compress a binary with UPX (for example) - the size on disk gets smaller, but because the entire executable is decompressed into RAM rather than demand paged in, then it uses more memory.

<comment><USER3> No, there's no need for a tradeoff at all in a properly implemented compiler. You can certainly compress a table in memory and still have that table be indexable at runtime. Entropy coding is not the only type of compression that exists. DWARF solved this a decade ago.

<comment><USER4> I thought Go did use DWARF for debugging info. Your remark sound as if some easy optimization was ignored. Can you please elaborate?

<comment><USER3> This "pclntab" format duplicates what DWARF does, but is not DWARF: [https://golang.org/src/debug/gosym/pclntab.go](https://golang.org/src/debug/gosym/pclntab.go)

</comment></comment></comment></comment><comment><USER5> Is it just me or something like that runtime.pclntab shouldn't be included in production builds at all?

I mean, it makes all sense while you're developing and testing, but it should be reasonably possible to strip it out from production build binaries, and instead put it in a separate file so that if you _do_ get a crash with a stack trace, then some external script can transform the program counters to line numbers, not have it embedded in every deployed binary.

<comment><USER6> The Go language literally requires that pclntab be included in release builds. I'm with you—it seems kind of crazy that this was designed into the language—but there you have it.

The reason is that Go's standard library provides functions that allow retrieving a backtrace and symbolicating that backtrace at runtime:

* [https://golang.org/pkg/runtime/?m=all#Callers](https://golang.org/pkg/runtime/?m=all#Callers)

* [https://golang.org/pkg/runtime/?m=all#CallersFrames](https://golang.org/pkg/runtime/?m=all#CallersFrames)

Unlike in C or C++, where you can link in something like libbacktrace [0] to get a best-effort backtrace, those Go functions are guaranteed to be correct, even when functions have been inlined. This is no small feat, and indeed programs compiled with gccgo will often be incorrect because libbacktrace doesn't always get things right when functions have been inlined.

[0]: [https://github.com/ianlancetaylor/libbacktrace](https://github.com/ianlancetaylor/libbacktrace)

<comment><USER5> Is it _that_ common for these functions to be used? Perhaps transitively through some popular libraries? Just being in the standard library doesn't even necessarily mean that these functions (and the data they need) should get included by the linker.

<comment><USER6> Any program that uses a logging framework, including the stdlib log package, will wind up depending on runtime.Callers at least transitively. That's probably most Go programs; certainly most of the programs large enough to be worrying about binary size.

Unlike in C, there are no macros like __FILE__ and __LINE__, so there is no alternative to runtime.Callers (short of preprocessing your Go source code).

<comment><USER7> You can still get a backtrace without symbols though.

Why couldn't the Go team introduce a flag that strips symbols, while making clear to people that they should only use it if they are okay with backtraces looking like

      #1  0x00007ffff7ddb899 in __GI_abort () at abort.c:79
      #2  0x0000555555555156 in ?? ()
      #3  0x0000555555555168 in ?? ()
      #4  0x000055555555517d in ?? ()
      #5  0x0000555555555192 in ?? ()

or similar

<comment><USER8> Because one of the mantra of Go is _not_ telling users to have a "debug" build and a "release" build. The development build is the one that goes into production, with no difference in optimizations, symbols and whatnot. This has pros and cons, like all tradeoffs.

<comment><USER9> The "26M pclntab" issue sounds like an excellent argument against that design decision.

<comment><USER10> Counterpoint: our work infrastructure for symbolizing stripped binary stacks sucks. I'd love to get backtraces with symbols from the field.

But also this entire section could be compressed and seekable and only expanded on demand. No need to leave it as a bloated uncompressed blob.

<comment><USER7> > our work infrastructure for symbolizing stripped binary stacks sucks. I'd > love to get backtraces with symbols from the field.

If stripped binaries don't work for you, nobody is saying you should be forced to use them. You would presumably just not avail yourself of the option.

(As an aside -- what's wrong with your symbolification infrastructure at work? This is something I spent significant time working on in a past job, so I'm always curious how other people are doing it).

<comment><USER10> My employer's internal infrastructure for this just sucks; it's not endemic to stripping binaries.

Specifics? The automation to associate cores with specific builds is bad or non-existent; the automation to load up GDB with the right files and corresponding sources or source branch is bad/non-existent; the fileserver storing symbol data is slow and sometimes remote to the developer across a very thin pipe; etc, etc. All to some extent foot-shooting by my organization. But we have TBs and TBs of build artifacts and hundreds of unique daemons producing cores and also have to debug kernel cores and multiple product branches.

Part of the pain is perhaps that we're cross-building for an embedded FreeBSD- derived system, but the majority of our developers use Linux or Mac, so we can't necessarily use host-native (or host-native-only) tools.

I think basically the situation is begging for someone familiar with the problem-space to sit down and bang out a FUSE filesystem or two (e.g. fetching sources on-demand as GDB reads them, instead of checking out a full multi-GB repo copy) along with a shell or Python script to load up a core. But no one has done it yet and management doesn't really prioritize developer tools.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> This is where letting a large enterprise guide the development of a piece of widely-used software becomes questionable. At a FAANG the constraints are fundamentally different.

At work I routinely see CLIs programs clocking in at a gigabyte, because it was simpler to statically link the entire shared dependency tree than to figure out the actual set of dependencies, and once your binary grows that big, running LTO adds too much time to one's builds. And disk space is basically free at FAANG scale...

<comment><USER12> > This is where letting a large enterprise guide the development of a piece of > widely-used software becomes questionable. At a FAANG the constraints are > fundamentally different.

I see where you're going with this, but the conclusion of the article is that Go isn't (what you call) FAANG-y enough for them.

From the article: _" This design choice was intended to lower the start-up time of programs [...] This performance goal is not relevant to server software with long-running processes, like CockroachDB, and its incurred space cost is particularly inconvenient for large, feature-rich programs."_

What they want is a way to pass 1000 parameters to the compiler so that nobody ever gets a working binary unless they have a team of release engineers to make one for you. The Go team and their users at Google easily have access to said team of release engineers, but took the opposite approach. They optimized for programs that start fast even though most of those teams at Google are writing programs that run for a long time. They optimized for a compiler and runtime that don't have a billion knobs to turn, even though they can afford to pay an army of engineers to tune those knobs.

So I actually think this has very little to do with optimizing for FAANG. This is more about every program getting the same compiler options, which is merely a philosophy that some people on the Go team have, not really a philosophy that every FAANG has. (They use Java at Google too, and Java is more than happy to give you knobs to turn. It also doesn't start up very fast.)

<comment><USER13> > It also doesn't start up very fast

This has been changing rapidly with GraalVM and projects like Quarkus, which can build a static binary of a Java program that starts, as you'd expect, much faster (still not performance of a C/C++ program, but way faster than a regular Java program).

<comment><USER14> Did I miss something important about GraalVM? I thought it was another runtime on top of JVM, not a replacement for it. For a Java program, for example, GraalVM would presumably have no impact since it wouldn't be used for it.

<comment><USER15> GraalVM is several things.

In this context people are talking about the native-image tool which uses a small JVM written in Java (SubstrateVM) to compile to native code ahead of time, no HotSpot or other JVM dependencies.

There's also Truffle which is a way to put other languages onto the JVM (both HotSpot and SubstrateVM).

</comment></comment></comment></comment></comment><comment><USER16> I'm not sure why people are so worried about the size of the executable file here. If the runtime.pclntab table is never[1] used then it won't be paged into memory, and disk space is mostly free these days.

[1] Well, hardly _ever_! (Sorry not sorry for the obligatory Gilbert and Sullivan reference.)

If you're using the Go executable on a system without virtual memory support, yeah, that's going to suck, but it appears the Go runtime is horribly bloated and not really suited for super-tiny 16-bit processors in the micro-embedded space. But for something like Cockroachdb, why worry about the file size?

<comment> F-0X > disk space is mostly free these days.

This is the only "argument" ever presented, and I don't think it is any good. I care about file sizes. I want to get the most out of my hardware. Not needing to buy another drive is always going to be cheaper for me and every other user.

<comment><USER17> > Not needing to buy another drive is always going to be cheaper for me and > every other user.

128GB+ drives are standard on mid-range laptops. Even at 64GB are you really going to fill up disk space because of Go executables?

CockroachDB (a large software project) is only 123MB. I doubt most people even have 100 pieces of non-default software on their laptop or that executables are going to fill up storage and break anyone's bank these days.

If you're short on HD space, I'm typically targeting photos and videos, not software.

<comment><USER18> Well. Grow your entire system by 60%. Even with 128GB it will be non negligeable.

</comment></comment></comment></comment><comment><USER19> This is where go's insistence on reinventing the wheel feels terribly misplaced. Every major debug format has a way to associate code locations with line numbers. Every major debug format also has a way to separate the debug data from the main executable (.dSYM, .dbg, .pdb). In other words, the problem that the _massive_ pclntab table (over 25% of a stripped binary!) is trying to solve is already a well-trodden and solved problem. But go, being go, insists on doing things their own way. The same holds for their wacky calling convention ( _everything_ on the stack even when register calling convention is the platform default) and their zero-reliance on libc (to the point of rolling their own syscall code and inducing weird breakage).

Sure, the existing solutions might not be perfect, but reinventing the wheel gets tiresome after a while. Contrast this with Rust, which has made an overt effort to fit into existing tooling: symbols are mangled using the C++ mangler so that gdb and friends understand them, rust outputs nice normal DWARF stuff on Linux so gdb debugging just works, Rust uses platform calling convention as much as possible, etc. It means that a wealth of existing tooling just works.

<comment><USER20> I am not a fan of Go, and I also wish these things were true (and more[1], actually), but I find it hard to agree that its priorities are "terribly misplaced." Inside the context of Go's goals ( _e.g._ , "compile fast") and non-goals ( _e.g._ , "make it easy to attach debuggers to apps replicated a zillion times in Borg") these trade-offs make a lot of sense to me. Like: Go rewrote their linker, I think, 3 times, to increase the speed. If step 1 was to wade through the LLVM backend, I am not sure this would have happened. Am I missing something?

I love Rust, but Go is focused on a handful of very specific use cases. Rust is not. I don't know that I can fault Go for choosing implementation details that directly enable those use cases.

[1]: [http://dtrace.org/blogs/wesolows/2014/12/29/golang-is- trash/](http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/)

<comment><USER21> > non-goals (e.g., "make it easy to attach debuggers to apps replicated a > zillion times in Borg")

But wouldn't it still be nice to have a standardized way to analyze post- mortem dumps across languages?

<comment><USER22> Google's anointed production languages used to be five: C++, Java, JavaScript, Python, and Go. Not much to reasonably standardize across, especially if a standardized solution ends up with more compromises than a custom one.

<comment><USER3> But DWARF uses less space than Go's native format. So inventing a custom "linetab" format seems like the compromise, not using DWARF.

<comment><USER22> I suspect that the format is again copy-pasted from somewhere in Plan9, and existing Plan9 tools for it are ported, too.

</comment></comment></comment></comment></comment></comment><comment><USER23> There is an project called TinyGo [0] which brings Go to embedded systems where binary and memory size matter even more.

[0] [https://archive.fosdem.org/2019/schedule/event/go_on_microco...](https://archive.fosdem.org/2019/schedule/event/go_on_microcontrollers/)

</comment><comment><USER24> The next time you need to make an HTML treemap like this, try my tool: [https://github.com/evmar/webtreemap](https://github.com/evmar/webtreemap)

It provides a command line app that accepts simple space-delimited data and outputs an HTML file. See the doc: [https://github.com/evmar/webtreemap#command- line](https://github.com/evmar/webtreemap#command-line)

(It also is available as a JS library for linking in web apps, but the command line app is the one that I end up using the most. I actually built it to visualize binary size exactly like this post and then later generalized it.)

<comment><USER25> Another option is to generate a text file in the format expected by flamegraph. Especially useful when the data is hierarchical.

[https://github.com/brendangregg/FlameGraph](https://github.com/brendangregg/FlameGraph)

Exemple for Java:

[https://github.com/pcdv/deps- flamegraph/blob/master/README.m...](https://github.com/pcdv/deps- flamegraph/blob/master/README.md)

</comment></comment><comment> kris-s Great writeup, I believe there is a open issue from Rob Pike 2013 this would fall under: [https://github.com/golang/go/issues/6853](https://github.com/golang/go/issues/6853)

<comment><USER26> Except that this write up says this was a deliberate design decision to trade space for speed. That's not something to be fixed, unless you convince Go to make different trade offs or to provide more optimization options.

<comment> kris-s The linked issue is tagged "NeedsFix".

</comment></comment></comment><comment><USER27> The author guessed a few things wrong:

* fmt.Println pulling in 300KB isn't proof that Go's standard library isn't "well modularized". It's the wonders of Unicode and other code that is actually used.

* 900K for the runtime isn't surprising when you have complex garbage collection and goroutine scheduling among other things

<comment><USER28> > fmt.Println pulling in 300KB isn't proof that Go's standard library isn't > "well modularized". It's the wonders of Unicode and other code that is > actually used.

I would guess a large part is that it has to pull in the entire reflection library to support format verbs like %#v which renders the argument as a Go literal.

</comment></comment><comment><USER29> You can compress with upx (at the cost of increased startup time in the order of hundreds of ms, which is okay for servers) and/or not include all debug symbols. Doing both usually shaves >60% off a binary.

<comment><USER30> UPX transforms demand-paged, reclaimable page cache memory into a blob of unreclaimable anonymous memory.

It makes no sense for most use cases where I've seen it. It adds runtime costs both in terms of start-up and memory usage.

Maybe it helps in terms of binary sizes for downloads — but those are often compressed anyways! E.g. Your docker images are compressed and UPX'ed binaries in a layer aren't buying you anything (just adding runtime costs).

<comment><USER29> For a lot of applications the increase in startup size and memory usage are negligible. The startup time is increased by dozens or hundreds of ms, which is not a lot for a server application. I tried to measure the increase in memory, but wasn't really able to, so either it's a very subtle difference or it's very small.

Not everything uses Docker.

<comment><USER31> UPXed binaries have their code pages mapped as dirty - this means the OS can't page them back to disk if needs or wants to. In some cases, that's an acceptable cost to pay - for low-latency servers you might want to mlock all your executable pages so there's no risk of a page fault and disk read killing your tail latency. Of course if you're doing that then you have to either pay the full cost of your binary size in memory, or you have to have some warm up phase and then hope that everything you need is loaded then. In the first case you suddenly care a lot about binary size, because memory is quite a bit more expensive than disk.

But one valuable reason to use something like UPX is that you can attach a crappy and thus inexpensive disk to servers that you're not using for actual storage. Compression on disk lets you load from a slow disk faster, and of you weren't paging to disk anyway then UPX doesn't have much of a cost.

But if you're on a traditional desktop operating system, UPX will increase your effective memory footprint, and force writing to swap instead of merely dropping pages. On Android, which doesn't swap, you'll significantly increase your memory footprint.

</comment></comment></comment></comment><comment><USER32> Rob Pike submit issue: [https://github.com/golang/go/issues/36313](https://github.com/golang/go/issues/36313)

</comment><comment><USER33> Being an Electron and a Go developer, I'm not complaining too much about the size of Go executables. Electron backed softwareon the other hand...

<comment><USER34> Have you tried this? I've played around with it in the past.

[https://github.com/asticode/go-astilectron](https://github.com/asticode/go- astilectron)

</comment></comment><comment><USER35> Would it be possible to add a flag for the compiler to disable the line table pre-expansion?

<comment> w0rd-driven I can't exactly tell if we're saying the same thing but my thought was a flag to switch between the 1.2 way for faster startup and the earlier approach for longer running processes. The trade off is added complexity in identifying your binary usage patterns and keeping both methods in the tooling.

These kind of changes may not be breaking in a technical sense but it's very unexpected behavior if you're one to notice patterns like file sizes changing in such a significant way over time. An answer of "stick with v1.1x indefinitely if you want the old behavior" only feels like a very temporary answer.

</comment></comment><comment><USER36> Language flame wars aside, Bloaty McBloatface is a wonderful tool to analyze why the binary size is big: [https://github.com/google/bloaty](https://github.com/google/bloaty)

I frequently use this tool to answer questions for C++ binaries, another language that has a penchant for producing large executables.

</comment><comment><USER37> 70MB source seems somewhat bloat for such project, no?

<comment><USER38> For an ACID compliant, resilient, consistent, distributed, auto-sharding and auto-tuning for low latency, highly scalable SQL database? No.

<comment><USER39> 70MB of source is such an extreme amount I don't know how it could be reasonably justified, there must be an enormous amount of waste. All of sqlite is 6MB.

<comment><USER40> When you have the features of sqlite, you can have the size of sqlite.

PostreSQL is 36MB. [1] Granted Go is much terser than C and has a larger standard library, but we're not at absurd levels.

[1]

         find src -name '*.c' -o -name '*.h' | xargs cat | wc -c

</comment></comment></comment></comment><comment><USER41> So what is the solution then? Will they just have to fork Go and compress the table again like before? It's completely insane that it would eventually surpass the size of the program itself.

<comment><USER42> Is the line table in go executables absolutely essential? Shouldn't they be strippable, the way you can strip debug symbols from a C binary?

<comment><USER29> They are, and many people don't include them in production builds. This article is incomplete for not mentioning it.

<comment><AUTHOR> You're right, but that wouldn't make the main problem go away though. I just built a simple plugin for Kubernetes's kubectl and it's about 32MB with go build's -ldflags -s -w where 16MB of that is still the pclntab mentioned in the article.

<comment><USER29> The problem with removing that completely is that you won't get _any_ information on panics. I don't think this is what you really want, and the current behaviour strikes me as a reasonable middle ground.

32M is really large for a simple plugin, and to be honest I think that says just as much about Kubernetes as it says about Go.

</comment></comment></comment></comment></comment><comment><USER43> > prior to 1.2, the Go linker was emitting a compressed line table, and the > program would decompress it upon initialization at run-time. in Go 1.2, a > decision was made to pre-expand the line table in the executable file into > its final format suitable for direct use at run-time, without an additional > decompression step.

Sounds like a good case for a flag.

</comment><comment><USER44> Why are they not considering keeping the pre 1.2 compressed runtime.pclntab and have the data be decompressed to a separate file on first run. This way, memory footprint is kept low whilst keeping executable size down?

</comment><comment><USER45> sidenote : currently trying to get up to date with the best way to get distributed acid key value storage those days. Is coackroach the new standard ? I tried to find benchmarks comparing it to things like postgres for various use case but only found articles that read like ads.

<comment><USER46> > the best way to get distributed acid key value storage

You will need to define "best way", "distributed" and "acid" based on your requirements.

For most people, multi-master MySQL and Redis with vector clocks is a great combination.

> Is coackroach the new standard

No database less than 5 - 10 years old works in production, so no, it's not a standard.

> I tried to find benchmarks comparing it to things like postgres for various > use case

Well, without more detailed requirements, good luck. Also comparing pg to distributed databases is ... like comparing apples and oranges.

Source: experienced DBA.

<comment><USER45> distributed : able to scale horizontally, ideally over multiple availability zone, to ensure both performance and data safety.

acid : able to perform bank account money transfert style transaction.

last time i checked, pg was able to shard and work in multiple replication configuration, ensuring a good scalability story up to terabytes of active data..

Does multi master mysql enable row locking ?

</comment></comment></comment><comment><USER47> > _runtime.pclntab_

Ah yes, an example of Google's long and descriptive identifier names, a lament of which surfaced here recently. ([https://news.ycombinator.com/item?id=21843180](https://news.ycombinator.com/item?id=21843180))

> _I'm glad we now live in a futuristic utopia where keyboard farts like p, > idxcrpm, and x3 are rare._

<comment><USER48> What's interesting is that Go encourages letter soup while the rest of Google styles encourage long descriptive names.

Sometimes I think it was intentionally done to troll the rest of Google that used long_descriptive_names_ with an 80 col limit (Java argued up to 100 but no one else did, not even JS which uses Java's long names).

I always thought it was funny walking around the office seeing everyone with their squished and wrapped code on a huge 32in monitor. Lots of the code in google3 looks like haiku squished to the right margin.

</comment></comment><comment><USER34> Go is a _hybrid_ language.

Its not a jvm, but its runtime has jvm-like features such as garbage collection and reflection but also a thread scheduling system called goroutines.

I love the fact it is monolithic in nature.. One exe is all you need no matter which platform you use. Everything is statically compiled into the binary.

No bundling the jvm and a load of jar files, or lib*.so dependencies.

<comment><USER49> Toolkits to precompile a Java app into a single native binary with the runtime and stdlib inside have been around for 20 years or so.

</comment></comment><comment><USER50> > _there is about 70MB of source code currently in CockroachDB 19.1_

 _That 's_ what is insane here; way more so than Go executable size issues.

</comment><comment><USER51> go really should be the same size as c with static link, somehow it's 4x larger in size, why is that?

upx can help, with upx, c static binary is still much smaller.

both removed debugging info.

</comment><comment><USER52> TLDR: They included debugging information when they did not intend to.

<comment><USER53> What's the fix?

<comment><USER54> Don't use Go.

</comment></comment></comment><comment><USER55> Because Go isn't C?

</comment><comment><USER56> Disclosure: I work on Google Cloud (but this applies generally).

There seem to be a lot of arguments about disk space (arguably free in 2020), memory (free-ish because as tytso points out, they won't get backed) and then bandwidth.

I _think_ most people saying "bandwidth" might mean "time to fetch", because GCS, GCR, S3, etc. all have free egress on the transfer from them to GCE/EC2. If you have a self-hosted Docker Hub or something on an EC2 instance, that's not the case (you may pay Zone to Zone egress of $.01/GB).

If you were paying a penny per gigabyte egress, a 100MB-ish binary is only .1 GB and therefore at best 1/1000th of a dollar. On the 16 vCPU hosts that CockroachDB prefers (see their recent benchmarks), that's equivalent to about 5 _seconds_ of runtime.

A fair retort is that in container land, this becomes death by 1000 cuts as each binary includes all the same stuff over and over again (for minikube, the .iso is nearly 1 GB [1] now, but not because of the 50MB binary itself).

Even so, a _1GB_ image takes almost as long to pull from a object store like GCS as a 100 MB image (at many Gbps, the constant factors dominate). If you're trying to run something this large as a function on Cloud Run or Lambda or Knative, you'll probably be sad (you've burned about 800 vcpu seconds, economically, of compute time) but that's why there are layers.

tl;dr: your 50 or 100 MB binary doesn't "cost" much, but a 1 GB container image without shared layers does.

[1] [https://github.com/kubernetes/minikube/issues/5013](https://github.com/kubernetes/minikube/issues/5013)

</comment><comment><USER57> Use upx

[https://upx.github.io/](https://upx.github.io/)

</comment><comment><USER58> I think the GO runtime libraries are linked into the binary so there are no external dependencies for GO itself.

<comment><USER59> Exactly. The whole point of Go is to fix the issue with Python where you have to have all the right libraries installed wherever you intend to run.</comment></comment>


Designing a 'Single Server' MMORPG - <AUTHOR> http://www.gamesetwatch.com/2009/05/opinion_designing_a_single_ser.php

<comment><USER2> One problem with Eve Online a few years back, was that they chose to organize servers by star system. In other words, loads were distributed by named location. (Which was why certain populous systems like Jita often lagged.) Also, this distribution was static and didn't adapt to changing situations. (Like roving fleets of 200 ships.)

Processor load should be distributed by population and user activity. The unit of processing should not be named locations. Aggregations of players should be the basis for the units of processing. With a space game like Eve, this is easy. When people form a fleet or a squadron, the squadron's frame of reference becomes a "locale." When two fleets merge, two locales merge. This would be an inherently dynamic load distribution system. The merging operation could be disguised with "warp-in" effects. You could even have a locale moved from server to server, and this could be covered for as a "warp storm" where both sides are effected.</comment>


Backbone patterns - <AUTHOR> http://ricostacruz.com/backbone-patterns

<comment><USER2> One of the patterns looks like it requires more processing power than necessary. The rendering of a template should be fast, since you will likely be doing it many times on a page.

        ContactView = Backbone.View.extend({
          template: function() {
            var template = _.template($("#template-contact").html());
            return template.apply(this, arguments);
          },

          render: function() {
            // This is a dictionary object of the attributes of the models.
            // => { name: "Jason", email: "email@example.com" }
            var dict = this.model.toJSON();

            // Pass this object onto the template function.
            // This returns an HTML string.
            var html = this.template(dict);

            // Append the result to the view's element.
            $(this.el).append(html);

            // ...
          }
        })

Note that the template is already on the page and has to be serialized by jQuery with the `_.template($("#template-contact").html());` line. It would be faster to already have the template in a JavaScript string, ready to be interpolated with your values.

Otherwise, for every view you will be fetching the template from the DOM. The difference in implementation would be from this:

Mail me at <%= email %>

to this:

It's a bit more work, but it reduces the amount of work that has to happen just to render one of many templates.

<comment><USER3> I think a simpler way would be to cache the value of `_.template($().html());`.

</comment></comment><comment><USER4> It looks great.

If anyone want to see a rails-backbone application example Railscasts has published today "Backbone on rails part 1"[1].

[1] <http://railscasts.com/episodes/323-backbone-on-rails-part-1>

PS It's a paid screencast, but railscasts pro is worth the money.

</comment><comment><USER5> It's great to see knowledge like this compiled into one place. Unfortunately, it looks like the resource is only superficially open source.

I was about to make a pull request to fix a typo, when I found someone had already done so: <https://github.com/rstacruz/backbone-patterns/pull/3>

If a objective correction like this takes 6 weeks (and counting!) to be merged, does the open nature of this page really matter?

Not that this detracts from the message, but take the claim "This is a massive work-in-progress." with a grain of salt.

<comment><AUTHOR> The last commit is the January 29th, I guess he just didn't notice the pull request

</comment></comment><comment><USER6> Nice... I am writing my first site using backbone and as a Java Dev sometimes I have a hard time to do things in backbone, I want to make everything a into reusable objects and views as far as I am seeing right now are not always to be used like that.

Sometimes They are used to listen to collections and create smaller things( like a factory ) sometimes they can represent a whole small popup, sometimes a single HTML entity. I am sure my code is not 100% yet, but patterns like this help a lot.

<comment><USER7> Not sure why you're being downvoted, but I tend to agree. I've found that my backbone apps are a lot nicer to deal with when I think of my views as potentially re-usable components.

It takes a bit of forethought but you end up with much nicer, composable interfaces that allow you to respond quickly to changing requirements.

</comment></comment><comment><USER8> I know in the javascript world there's a preference by large to use camelCasing but I'm not a fan of it. I don't think its easier to read, and there can be confusion with what to capitalize.

Also whats the purpose of putting your templates inline? I compile all my templates server side into a js file called template.js and when I need a template I simply call:

Template.documents.index

</comment><comment><USER9> Notice how this carefully doesn't use silent: true, seeing as in the latest version of Backbone, they have repurposed the silent option for completely different functionality.

<comment><USER10> Not quite. (Although you and I have certainly discussed this plenty on GitHub.) For folks who haven't been following along:

Performing "silent" changes to a model are a power-user feature of Backbone, where you can twiddle attributes on a model temporarily, or many times, without receiving change events. Then, when you're ready, you get a single change event with the diff in model state.

Prior to 0.9, you'd get the single diff, but not all of the individual "change:attribute" events for the diff. 0.9 and above have been improved to give you both. Yes, it's something to watch out for if you want to upgrade an existing app.

<comment><USER9> The problem is your complete unwillingness to compromise and to make it easier for people to upgrade an existing app.

The way that silent works now should be it's own option, say "defer". Because you're deferring the changes until a later date. It could even be called delay or later. Whatever it is called, it is in no way silent anymore.

My proposal is to add a new option, mute, which bypasses the change events and causes the attribute to change but does not trigger change:attribute.

Edit: <https://github.com/documentcloud/backbone/pull/981>

<comment><USER10> While I appreciate the effort and persistence, having {silent: true, mute: true} be part of the model API is definitely _not_ the direction that the Backbone API should be going.

That said, what you do with your copy of backbone.js in your own project is your business. I'd highly encourage you to extend your version with your preferred behavior for "silent" -- in fact, the source is annotated to make this sort of thing easier for folks that find the need.

</comment></comment></comment></comment><comment><USER11> This is nice to have it in one place, but I still not sure I like bootstrapping model data that way. Does anyone else find it a bit messy? Also, I'm using AMD/Require.js throughout my Backbone apps, so how could I bootstrap the model data in that manner, and still have access to it inside my modules?

<comment><USER10> So, the reason why you want to bootstrap the models you need for an initial render, directly into the page ... instead of doing this:

        myApp.init = function() {
          Books.fetch();
          Authors.fetch();
          Notes.fetch();
          ...
        };

... is because, all else being equal, it will noticeably improve the speed of your initial page load. Instead of N separate HTTP requests, all of your model data is available in the same HTTP request that fetches the HTML. Put the bootstrap at the bottom of your page, and you're in pretty good shape.

<comment><USER12> Depending on the other content of the page and the nature of your traffic/user data, it might make more sense to take advantage of caching by making the initial loaded resource (the HTML file) completely static and loading the bootstrapped data in a single separate request.

<comment><USER10> Yes, absolutely. If the initial HTML page is heavy, and can be cached for all users, then separate makes sense. If the initial HTML page is light, or user- specific (the usual case for a backbone app) then bootstrapping in a single HTTP request makes sense.

</comment></comment></comment></comment><comment><USER13> Im still walking the backbone path so this is pretty useful.

Im curious how one would setup a instant search using it? I have done it previously by attaching a jQuery keyup event listener but I know I should be using a view. I have had a poke on google for this but haven't turned up anything.

</comment><comment><USER14> I'd like to nominate private subviews as a pattern.

        (function(){
            var myPrivateView = Backbone.View.extend({...});
            app.views.Foo = Backbone.View.extend({
                /* use myPrivateView somewhere in here */
            });
        })();

<comment><USER3> Interesting. Any concrete examples of how it can be useful in a real-world project?

<comment><USER14> It's a good way to break a view into logical sub-components, while still only exposing one overall view to the world. Really just a variation on data hiding and encapsulation.

</comment></comment></comment><comment><USER15> I have a hard time seeing the advantages of backbone.js over something simpler like knockout.js. Is it that good? Is it a question of having a complicated enough webapp?

<comment><USER2> Just look at the examples of what was built using Backbone.js vs. Knockout.js. <http://backbonejs.org/#examples> vs. <http://knockoutjs.com/examples/>. Backbone.js has many more real-world examples that demonstrate it's flexibility and ease.

<comment><USER16> Those aren't really comparable links.

The Backbone link goes to a "who is using Backbone" page. Which is nice, and quite convincing that Backbone is a serious library used in some huge projects.

The Knockout link, however, goes to a "how do you use Knockout" page. Which is also nice, and a very useful documentation source that Backbone is sadly lacking.

But as a new developer, if anything, the Knockout link is a better reason to pick it. Good documentation goes a LOT further than knowing that Groupon has selected the library, in my book.

</comment></comment></comment><comment><USER17> This is nice! Been using backbone for about a year, and often I wonder if there are better ways. I'm learning a lot of new solutions by reading this. Thanks.

<comment><USER18> Yes, thanks!, this is fabulous, between the backbone recipes book (<http://recipeswithbackbone.com/>) and this, and the absolutely fabulous book by Alex McGaw (author of Spine, <http://shop.oreilly.com/product/0636920018421.do>), my JS app mojo has been steadily improving!

</comment></comment><comment><USER19> I think I disagree with the buffer pattern. I'd probably run the function with an argument of next() so you could keep the function more portable.

<comment><USER3> That's how jQuery implements it for jQuery.fn.queue, and I think I'll revise it that way too.

</comment></comment><comment><USER20> not to forget the Memoiz Module pattern from [http://weblog.bocoup.com/organizing-your-backbone-js- applica...](http://weblog.bocoup.com/organizing-your-backbone-js-application- with-modules/)</comment>


"You are 6 times more likely to be killed by a pig than by a shark" - <AUTHOR> http://www.shark.ch/Information/Accidents/index.html <comment><USER2> Maybe for the population in general. But for example, I surf and see sharks all the time. But I don't think I've ever seen a pig in real life that wasn't in an enclosed pen.

For a cool list of local shark sightings, read here:

[http://www.sharkresearchworldwide.org/pacific_coast_shark_ne...](http://www.sharkresearchworldwide.org/pacific_coast_shark_news.htm)

<comment><USER3> Yeah; it'd be great if they'd post likelihood ratios: For example, if I live in Florida, a shark attack hotspot, and like to go swimming at night--shark feeding time--what are my chances now?

<comment><USER4> My takeaway from the article is that I should spend no time thinking about shark attacks since it's only a distraction from the much larger danger of drowning,

The pig statistic is mere distraction too.

<comment><USER3> Well, .00000003% or so of the US population gets eaten by sharks every year, so there must be _some_ circumstances under which it becomes a significant risk.

The question is whether those circumstances are identifiable enough to be worth the cognitive load of avoiding: If 90% of shark attacks happened to people swimming at night in Florida, and only .00000006% of the US population actually swims at night in Florida, it might be an activity worth avoiding.

</comment></comment></comment></comment><comment><USER5> <http://xkcd.com/795/>

<comment><USER6> ah conditional probability. I wonder how many thousands of years of development we as a society have lost from a lack of understanding of this concept.

If there is a piece of maths that should be drilled into every student more than any other, its at least a vague understanding of the difference between joint and conditional probability. and the underlying knowledge the ability to make this distinction implies. Much more likely to be useful to a randomly selected daily life than trig, calculus or geometry.

</comment></comment><comment><USER7> Maybe the fear of sharks prevents people from getting themselves into situations where they would be killed by one. My takeaway is that I need to start cultivating a healthy fear of pigs.

<comment><USER8> Don't worry, if you see pigs in the wild the fear will come naturally.

<comment><USER9> I was out in the Arizona desert with a friend, and he was walking a few dozen feet ahead and he scared some javelinas out of the underbrush. They took off running past me, and it was a little unnerving. Panicked, not-so-bright little creatures, who knows what they might do?

Another friend of mine told me of a night he was camping in Georgia, and a wild boar came sniffing around his tent. Said it was the worst smelling thing he'd ever been around, and was a bit scared not knowing what it might do if he startled it.

</comment></comment></comment><comment><USER10> I think these should take into account the number of people on land and the time they spend on land vs that of water when comparing such statistics.

<comment><USER11> I was going to say the exact same thing. I think not only what you mentioned but also how much time they spend with the animals should be considered. I'm sure those killed by pigs are farmers that spend a lot of time with pigs. The attacks over the time around humans ratio should be looked at.

</comment></comment><comment><USER12> Of all animals, I've been attacked by swans the most. And you almost never see them. They're just really assholes.

</comment><comment><USER13> The submission title is misleading and inexact. It's not a quote from the article. The article actually says

"Alone in the USA and Canada approximately 40 people are killed each year by pigs – six times more than by sharks worldwide."

</comment><comment><USER14> I'm pretty confident this is false. It's been repeated a lot in articles (including the usually solid Bruce Schneier), but in an admittedly brief search, I can find no evidence to back it up.

Iowa is one of the largest pork producing states, and I can't find any mention of the deaths by pigs in the years I paged through: <http://www.public- health.uiowa.edu/face/Summaries/index.html>

New Mexico, a mostly rural state with a significant number of animal caused fatalities, does not seem to attribute any deaths to pigs out of the 63 deaths (including 2 by sheep) reported here: <http://www.nmwild.org/wp- content/uploads/animalrelated.pdf>

I can't find recent data from the CDC, but this report has fairly complete coverage from 1991-2001. Snakes are at 5 per year for the US, spiders 6, bees and wasps at 50, but pigs are not mentioned specifically anywhere in the report: <http://www.scark.org/docs/Animal%20Related%20Fatalities.pdf>

My best guess is that this number was either intended as sample copy and left in a final report, falsely conflates "other specified animal" with pigs, is a worldwide rather than US/Canada specific number, or attempts to included deaths from "swine flu" as death due to a pig. Please don't repeat this trope unless you have better evidence that it is indeed true.

<comment><USER15> In all likelihood, the stats are referring to wild boars and feral pigs, which can be extremely aggressive and have been know to attack and gore humans from time to time.

Here's a normal pig that attacked a number of people: [http://www.dailymail.co.uk/news/article-410319/Nurse- savaged...](http://www.dailymail.co.uk/news/article-410319/Nurse-savaged- enraged-giant-pig.html)

<comment><USER14> Yes, from time to time. But I'd bet you any sum of money that there are less than 40 such fatal attacks per year in the US and Canada. And I say this as someone who hunts wild boars, and currently has 2 wild boar hams soaking in brine in preparation for a Christmas dinner.

And what do you mean "in all likelihood"? Do you mean that you have some evidence that there are more wild boar attacks per year in the US and Canada than there are shark attacks worldwide? Or do you mean that the concept is kind of cool despite being entirely without evidence?

</comment></comment></comment><comment><USER16> I used to work with the Cape Eleuthera Institute (<http://ceibahamas.org>) -- they are doing world class shark research to raise awareness about how important and endangered sharks are.

</comment><comment><USER17> "You are much more likely to be killed by bees, dogs, bats, and certainly in Hong Kong, automobiles." [http://www.csmonitor.com/World/Global- News/2010/0204/The-glo...](http://www.csmonitor.com/World/Global- News/2010/0204/The-global-odds-of-a-shark-attack-You-re-more-likely-to-eat- one-than-be-eaten)

</comment><comment><USER18> Not nearly as silly as it sounds. Feral hogs in the south can top out at 400-800 pounds (though this is by no means the average) and are quite clever, and sometimes aggressive. I'm not sure if there have been confirmed fatal attacks on humans, but they do carry brucellosis.

</comment><comment><USER19> If you never go swimming, or to the sea (or above), you have a 0% chance of being killed by a shark.

If you don't work with pigs (a pigfarm ?) or visit any farms, you probably have a 0% chance of getting killed by pigs as well (unless you count over- eating ;-) ..

<comment><USER20> Unless you live somewhere - like europe - where there are vicious wild boar in the forests.

<comment><USER21> i live in vienna, i see boars regularly in the viennese forest. you just walk past them, stay calm, no problem. if there are piglets you stop and observe, never get between the piglets and their mother.

i can say with certainty that i would react quite differently if i saw a shark while swimming in the sea. but of course, pissing myself in the sea is no big embarrassment.

<comment><USER22> In the forest of dean the boars have become a pest. They rip up tents and steal food.

<comment><USER23> In Oregon, the local department of fish and wildlife has declared "feral pigs" a nuisance, meaning they can be hunted as much as wanted. They rip of farms, destroy fences etc. Every once in a while, they eviscerate someone, which keeps it interesting :)

<comment><USER20> Doesn't the Constitution give you the right to arm pigs - or is it only bears?

<comment><USER23> Razor tusks and all, not sure if the pigs need to be armed! <http://en.wikipedia.org/wiki/Hogzilla>

</comment></comment></comment></comment></comment></comment></comment><comment><USER24> Two words: Shark flu</comment>


Dear the internet: yes, digital magazines do cost money to create - <AUTHOR> http://reverttosaved.com/2012/07/18/dear-the-internet-yes-digital-magazines-do-cost-money-to-create/ <comment><USER2> > Just possibly, there's the teeny tiny issue of interactive content

And it goes on. You have a communications problem. Don't talk down to your customers, talk to your customers.</comment>


Silicon Valley's reality: The party is over - <AUTHOR> http://www.cnbc.com/2016/02/09/silicon-valleys-reality-the-party-is-over.html <comment><USER2> "Life is normal at the early stage," said Ganesan. "But late stage people are very cautious."

This is what I've been predicting. Seed and early stage has always been astronomically high risk, and so that's more likely to be priced in. I think we'll see a small pull-back there but nothing too serious, and it might be totally offset by the opening of equity crowdfunding (which I have mixed feelings about but it's a thing).

I think this is the end of the "unicorn." The thing that's going to dry up is peoples' appetites for companies that are post-A and still don't have a good revenue picture or have their feet under them at all. By the end of an A runway there should be some sign of liftoff before people are going to be willing to shell out more. There's also going to be a lot less tolerance for drunken money spending unless revenue growth is really good.

Investors are going to start looking at bang/buck ratio instead of just bang. It's easy to make a lot of noise and look big when you have $100m+ in the bank. I think the worst of the unicorns are companies that are mostly built to market themselves to _investors_ more than customers and to chase cheap investor money. Those are dead.

I personally think the unicorn bust might actually be _good_ for bootstrapped or less lavishly funded companies. It's hard and very demoralizing to have to compete with SV insider companies running on seemingly endless fountains of VC cash. Those companies can just run you over with sheer brute force and lose money doing it and not care.</comment>


Investors Don't Like Acqui-hires - <AUTHOR> http://uncrunched.com/2012/08/26/investors-dont-like-acqui-hires/

<comment><USER2> This is pretty obvious right?

All of the comments on the pandodaily article revolved around the fact that VC doesn't get their homerun payday during an acqui-hire.

It seems like the discussion around acqui-hires revolves around what the commenter's personal interests are for the most part.

However, it makes sense that the acqui-hire would become more prevalent for a couple reasons. There are many more startups being created, both increasing the supply of startups to acqui-hire and decreasing the %age of startups that exit in a home run. Also, the market price for developers is going up, and seeing someone execute and deliver a product in a product vertical you want to execute in makes you worth a large multiple of an unknown and/or average developer.

</comment><comment><USER3> Investors who don't want their startups to accept an acqui-hire already have all the protection they need: make a better offer.

<comment><USER4> No. The logic employed here is the same logic that underpins liquidation preferences. If investors don't want to fund acqui-hire outcomes, they do in fact have more tools than a counteroffer.

The best operators are always going to get the best terms, but an industry- wide concern over acqui-hires could easily manifest itself in poorer terms for everyone else.

<comment><USER3> Presumably these acquisitions happen most of the time because the startup failed, this is the situation I was alluding to where they can just double down if they think they / the startup can do better but otherwise nothing is going to make this look like they picked Instagram.

If they're trying to protect against people who quit too easily or aim too low then they made the wrong bet, they're not going to avoid this if they predefine a price or terms where it's a very mediocre OK.</comment></comment></comment>


We are only two to three bad elections from the end of the West - <AUTHOR> http://www.slate.com/articles/news_and_politics/foreigners/2016/03/can_nato_and_the_eu_survive_donald_trump_french_nationalists_and_a_brexit.html <comment><USER2> anythkng that resembled the freedom afforded by all those who fought to their death was lost on and following 09/11/2001.</comment>


The 773M Record "Collection #1" Data Breach - <AUTHOR> https://www.troyhunt.com/the-773-million-record-collection-1-data-reach/ <comment><USER3> This is frankly terrifying and very ironic.

Websites put so much effort into tracking every little thing about their users, from where they come from to what they do. Hotjar ([https://hotjar.com](https://hotjar.com)) goes ahead and tracks mouse movements and now we even have crazy f-ed up startups like Peekmap ([https://peekmap.com](https://peekmap.com)) that claim to predict eye gaze without the webcam.

And yet they get pwned so easily.

 __So much effort into violating user privacy, so little effort into enforcing user security. __

<comment><USER4> and receive no meaningful legal consequences. These people should be on the hook for all damage done with this dump, but they won't be, so it doesn't really matter. It's not ironic, it's just business as usual.

Collecting data on users should be _extremely_ risky, even if they consent to it's collection.

<comment><USER5> I think it's time for an external, trustworthy entity to spawn that would vet and endorse companies that respect their users. Something like the "USDA Organic" label but for user privacies. Maybe it'd be an EFF-like entity that audits companies in exchange for a fee and endorse that "Company X, and the product/services it uses, are respecting user privacy". We could then derive a chain of trust between companies, maybe have a browser extension that tells when we're using a website that is endorsed by such entity?

<comment><USER6> I've gone back and forth on this. The very likely outcome of such a thing in practice is another PCI-like process. We both know an "EFF-like" organisation selected by a Government will one of the big accounting firms or similar in practice.

Particularly once there's a certification fee, it quickly becomes a racket, where people with strong ethics and skills get pushed aside by someone who paid a fortune to sit a course. Language lawyers will find ways to sign off on major issues, and some largely irrelevant thing ends up becoming the majority of the process.

<comment><USER7> You're catastrophizing by jumping to a negative outcome, similar to a cognitive distortion. It doesn't have to become a racket; that is a leadership choice. Individual identity issuing, public key certifying authority, banking, news, healthcare, truth-worthiness and many more areas would all be served best by _non-profits_ that are funded by a combination of grants, _modest_ fees and/or donations. There are some human activities that are too important to be privatized, like the fire department and the NTSB... whether the government should or shouldn't be responsible for running X is a topic for another time.

</comment></comment></comment></comment></comment><comment><USER8> Troy won't store the passwords associated with the username, which is a choice I can absolutely respect.

But as he discusses in the post, that leaves users knowing that their email address was in the data dump, but with no way of knowing which site it came from, or what password was breached.

So while this increases the number of records in HIBP, and perhaps makes the password popularity tracker a bit more comprehensive, it still leaves users exposed.

I know which password of yours was breached, and that information is now effectively public, but you probably don't know where to find it yourself, and I won't tell you which one it was. So I guess just assume all your passwords are cracked and use a password manager.

I don't really hold it against Troy, because again, I respect his decision not to store plains directly associated with usernames. He did as much as he was willing to with the data, and it's better than nothing, but not great all the same.

<comment><USER9> The slightly annoying thing here is that I _already_ use a password manager, so while the impact to me is minimal, I wish I knew which password specifically I have to rotate, instead of assuming that I need to rotate, like, all of them...

<comment><USER10> What we may need is the next step: a standardized way of changing passwords that would allow us to rotate them in bulk directly from the password manager.

<comment><USER11> You might be interested in this, from just over a month ago:

[https://news.ycombinator.com/item?id=18618193](https://news.ycombinator.com/item?id=18618193)

</comment></comment></comment></comment><comment><USER12> Reading this tweet ( [https://twitter.com/troyhunt/status/1085095504197779456](https://twitter.com/troyhunt/status/1085095504197779456) ), I've just donated the price of a coffee to Troy ( [https://haveibeenpwned.com/Donate](https://haveibeenpwned.com/Donate) ), and you should too.

HIBP is quickly becoming a critical piece of the Internet security infrastructure, and Troy should be lauded for undertaking it basically by himself.

<comment><USER13> I like the service. I just donated, too.

</comment></comment><comment><USER14> > Collection #1 is a set of email addresses and passwords totalling > 2,692,818,238 rows. It's made up of many different individual data breaches > from literally thousands of different sources. (And yes, fellow techies, > that's a sizeable amount more than a 32-bit integer can hold.)

I hate to be that guy [1], but no, that does fit in a 32-bit integer - as long as it's unsigned.

From the tweet, it seems like SQL Server puts the result of a COUNT into a signed 32-bit integer, which really surprises me.

[1] I lied, i love being that guy.

</comment><comment><USER15> I got a notification today that my domain has been included in this collection.

But as far as I can see it is gibberish spam-mails. I see 500+ entries such as:

       email@example.com
       email@example.com
       firstname.lastname@example.com

i.e. None of these emails at my domain are real, nor have they ever been real.

That said if you allow password-based authentication on a server which is shared you might consider using my PAM module:

[https://github.com/skx/pam_pwnd](https://github.com/skx/pam_pwnd)

It does lookups of previously-leaked passwords. Best practice these days is SSH-keys for authentication, but this would cover weak sudo passwords too, etc.

</comment><comment><USER16> Someone from a well-known leak forum is claiming that the "Collection #1" discovered by Troy Hunt is only part #1 of all available collections (there are at least 5, and additional other dumps). He also posted a screenshot of the original sales thread of the owner. The dumps together seem to have a total size of almost 1TB.

Not sure whether it's cool to post any links here.

</comment><comment><USER17> If you're using keepass, there are some plugins to check against HIBP: [https://keepass.info/plugins.html](https://keepass.info/plugins.html)

I'm gonna download the passwords offline and try this plugin: [https://github.com/mihaifm/HIBPOfflineCheck](https://github.com/mihaifm/HIBPOfflineCheck)

(you can grab the offline passwords from here: [https://haveibeenpwned.com/Passwords](https://haveibeenpwned.com/Passwords) )

</comment><comment><USER18> What's the latest consensus on the best password manager these days. I see he is recommending 1Password, but I recently found Bitwarden which looks quite good.

<comment><USER19> Bitwarden ([https://bitwarden.com/](https://bitwarden.com/)) is great and scores well in feature comparisons -- there was one on here recently. It's open source and has recently been audited too. It's free for the basic service, and really cheap for additional features. Great mobile apps and a web vault. And you can self-host. No bad points really.

<comment><USER20> The things that held me back from Bitwarden is the relatively short age of the company at 2~ years and the fact that there is only one dev. I'm reaching here. But even though the code is open source, he still owns the distribution. He can potentially be compromised (whether maliciously or not) and release an update that uploads the entire vault to him unencrypted. It could take a while before the internet caught on that the source code doesn't match the release build.

This of course could happen in a company like 1Password and there is at some point that I need to make the call and trust the person(s) coding the password manager. I feel that with 1Password there's at least the large size of the company which would mean more eyeballs and accountability. There is also the history of the company at 12~ years. This includes vetting and buy in from larger companies, which inspires a vote of confidence.

FWIW Bitwarden checks off nearly all the other boxes for me and I think the single dev has done a seriously bang up job.

<comment><USER19> All valid points. I guess nothing is perfect and you just need to decide where you're happy to compromise.

</comment></comment></comment></comment><comment><USER21> There is the rumor that it is called _Collection #1_ because it was part of a larger dump consisting of _Collection #1_ , _Collection #2_ , etc. There is also the rumor that the whole set was sold for - now hold on tight - <USER2> ginormous sum of $45.

</comment><comment><USER22> My email/pw is in there but there is easy way to know from which website so I don't know which password I have to change.

All my passwords are randomly generated so they are different for all websites.

</comment><comment><USER23> so strange ...

i've checked again if i was pwned and on the top there is a service i've never signed up - Apollo, a sales acceleration platform

i'm a simple dev and never subscribed to a sales service ....

<comment><USER24> I got the same. Anyone here know what is Apollo?

<comment><USER25> Same here. I had never heard of them. Turns out they're a YC'15 startup ([https://www.apollo.io/company/](https://www.apollo.io/company/)). There are no passwords in the data they lost according to HIBP. They seem to collect personal data from various sources and help other companies increase sales.

<comment><USER26> So not only do we have to worry about websites we actually use being compromised, but we also have to worry about these sketchy third-party companies that have purchased our data being hacked.

</comment></comment></comment></comment><comment><USER27> Let say my email appeared on Pwned list. And given most ( at least I think most ) people have zillions of web forums, services, sites, services using the email address.

What should you do now? I mean editing and changing password in everyone of them seems like a daunting task. And many of those services I no longer use anyway.

I am thinking of completely giving up the identity and start over, which seems easier. Or any other thoughts and comments?

Edit: I will definitely pay Apple a monthly fee if there is some simple and easy way to have online identity using email along with FaceID or Touch ID as 2FA. Getting rid of password while increasing security is something that should have happened but has yet to happened.

<comment><USER28> Just use a different e-mail and password for each web site.

</comment></comment><comment><USER29> Got a few 'hacker' emails on one of my throwaway addresses on this list the last few days. That account was leaked before in another list so this was not worrisome as I get those all the time for this address.

What did strike me as odd this time is that they did not end op in my spam folder but in my inbox. I'm using Gmail which normally for me has a very good spam/phishing detection. Somehow these mails came through though? Maybe its just an instance and Google was late to catch up with the cat/mouse game on this attack. Or these phishers are getting more sophisticated?

<comment><USER30> Gmail, and other large providers, use filters that adapt based on user input. If you report the messages as spam then the filter will learn, and hopefully catch them the next time.

</comment></comment><comment><USER4> Anyone got a link to the actual data?

<comment><USER31> There's a .torrent of Collection #1 available here:

[http://www.mediafire.com/file/mluhkk4dpqi8vfm/Collection_1.t...](http://www.mediafire.com/file/mluhkk4dpqi8vfm/Collection_1.torrent/file)

I found the link via a comment on /r/pwned [1]. I think it originally came from RaidForums [2].

[1] [https://www.reddit.com/r/pwned/comments/agsjie/troy_hunt_the...](https://www.reddit.com/r/pwned/comments/agsjie/troy_hunt_the_773_million_record_collection_1/eeb0xix)

[2] [https://raidforums.com/Thread-Collection-1-5-Zabagur- AntiPub...](https://raidforums.com/Thread-Collection-1-5-Zabagur-AntiPublic- Latest-120GB-1TB-TOTAL-Leaked-Download)

</comment></comment><comment><USER32> _Oh, it must be Tuesday._ I've just updated my blog post[0] with some password best practices and it's amazing how little has changed in the last 4 years.

[0] [https://darekkay.com/blog/another-password-leak-oh-must- tues...](https://darekkay.com/blog/another-password-leak-oh-must-tuesday/)

</comment><comment><USER33> Funny, I downloaded about 700GB of password dumps last week trying to figure out how someone got one of my passwords (no big deal, they never managed to access anything)

Maybe it was this one.

</comment><comment><USER34> HIBP doesn't protect the privacy of searched passwords!

Showing 20 bits of the password hash narrows down the possible passwords to one millionth. You should check it locally by downloading the password hash list.

</comment><comment><USER35> Here's one more record to add: my HN password is my username. Feel free to use this account for anonymous well-intentioned posting.

<comment><USER36> Heh, plausible deniability

... but with a non-trivial risk of someone else locking you out from your own account.</comment></comment>


CIA's communications suffered a catastrophic compromise - <AUTHOR> https://www.yahoo.com/news/cias-communications-suffered-catastrophic-compromise-started-iran-090018710.html?.tsrc=fauxdal <comment><USER2> You can't make this stuff up.

 _> Google's search functions allow users to employ advanced operators — like "AND," "OR," and other, much more sophisticated ones — that weed out and isolate websites and online data with extreme specificity.

> once the Iranian double agent showed Iranian intelligence the website used > to communicate with his or her CIA handlers, they began to scour the > internet for websites with similar digital signifiers or components — > eventually hitting on the right string of advanced search terms to locate > other secret CIA websites._

So it was complete security by obscurity, all indexed and on Google.

These are the people managing the greatest mass surveillance apparatus the world has ever known.</comment>


Ask HN: Does my IP assignment contract kill all side projects? - <AUTHOR>

I'm currently in the process of co-founding a company that will be in the field of language learning software. I often have lots of ideas for projects and tinker with different things. I'm worried that some project that i believe is totally unrelated to my company will actually be owned by the company due to the ambiguous language in the IP Assignment contract supplied to us by our lawyers.
The part that worries me reads like this: (Inventions are assigned to the company) "regardless of whether they are conceived or made during regular working hours or at my place of work, (that) are directly or indirectly related to the Company's business or potential business, result from tasks assigned to me by the Company, or are conceived or made with the use of the Company's resources, facilities or materials."
Would this apply to a something like a birdwatching app i made on my own time on the weekend? Our lawyer says that is a possibility. Aren't there tons of people who work on multiple projects at the same time, how do they manage? Our lawyer says that investors would not look favorably on any narrowing of this contract. Why can't i just say that anything i work on that is directly related to the company is owned by the company? Why does the language need to be so broad?
Any and all opinions are greatly appreciated. Thanks!!!! <comment><USER2> Probably yes.

I've heard that some people have had success at negotiating down the scope of their IP assignments to "Anything directly related to the company is yours, anything not directly related is mine", but it comes down to negotiating leverage. Basically, you need to be a bigshot in your industry and they need to be a relatively small, desperate company for this to work. They have to want you more than you want them, because otherwise they know that everyone else will require equally restrictive IP agreements.

As for how people manage to work on side projects - usually, if they become companies, the employee has to negotiate an explicit agreement with their employer to retain ownership. For the Apple I, Woz initially believed that HP owned the rights to it, and then some middle-manager at HP signed it away. For Viaweb, I believe there was a panic at some point where some Yahoo pointed out that it was quite possible that Altavista owned the rights to their code (since RTM worked there one summer) and they needed to get some manager there to sign off on it. PG may be able to tell you more if he's reading this.

Another common strategy founders do is write the prototypes and perform market research while still employed, and then throw everything away and start fresh when they form the company. Usually the scope of IP only extends to code (& design docs & other written descriptions), because if it's not written down, you can't prove when they dreamed it up. This was the Netscape strategy: write Mosaic, quit and rewrite it as Netscape. They got sued for it anyway, but I think it was for trademark infringement and not copyright infringement.

<comment><AUTHOR> This is my own company. I feel like i should be able to have my own side projects later, but will investors get squeamish if they see I have a non- standard agreement?

<comment><USER2> I dunno; I'd imagine that it depends on the investor. One of my previous employers had the California clause ("except for inventions developed entirely on their own time on their own machines" blah blah) even though they weren't a California company, perhaps because their law firm was more familiar with that.

One thing they probably _will_ get squeamish with is if the founders have different IP agreements than the employees. They'll probably also wonder how committed you are to the company if you're already thinking about side projects...

</comment></comment></comment><comment><USER3> For correct legal advice you need to go to a lawyer or good law source (like Nolo press). But in general unless you are in a state like California which I think makes this kind of assignment of rights impossible you are correct in reading this contract as killing all of your rights. Even in California most of the employment contracts I have seen lay out some limits (like on company time, machines or directly related to the business). Overall you may or may not have rights (depending on what state or country you are in) and on top of that you may or may not have lost your rights depending on what sort of contract you have signed.

<comment><AUTHOR> so are basically all startup founders' IP locked into their own companies for as long as they are with them?

<comment><USER3> Only if you sign a contract like that in founding or funding the company. Most entrepreneurs I know actually start many things as once- so I would guess they don't let set a clause get in and/or they set up another shell entity to own their other work.</comment></comment></comment>


I am a student - <AUTHOR>

i am a student. i live in Bangladesh. <comment><USER2> ...And so we are...we are students in this school called Life :-)</comment>


Show HN: Introduction to Simple Computational Geometry Concepts Using C++ - <AUTHOR> http://wykobi.com/tutorial0.html <comment><USER2> I often wonder what would happen if we learned high school concepts like geometry by recreating concepts like this - maybe alongside learning the proofs, or even before.

Being able to derive proofs on tests is important, but I think equally important is to have created knowledge you can reference in the future.

</comment><comment><USER3> This is neat. I'm actually in a Computational Geometry course this semester. Computational Geometry is pretty neat stuff and has some cool applications. Like for example mixing 3 paints to get certain ratios can be reduced to a convex hull problem where all the possible paint color ratios are contained in the triangle with the 3 points (representing the ratio of the colors of the paints) as vertices. My course doesn't involve any coding and is more about algorithm design through pseudo code and proofs.

<comment><AUTHOR> You're absolutely right, formal computational geometry is more biased towards exact algorithms and their associated complexities - specifically solving problems that more often than not have a geometric structure associated with them.

It is however fun from time to time to have some visual tangible results too otherwise things do get a bit dry.

<comment><USER3> Yeah I definitely agree it is fun to actually utilize pre built algorithms on a problem.

</comment></comment></comment><comment><USER4> Very interesting. I like the examples.

<comment><AUTHOR> Thanks! - btw if you have any ideas for other examples please do let me know, I'm very eager to add more interesting yet also accessible examples.

</comment></comment><comment><USER5> i _remember_ submitting this earlier, and sure enough : [https://news.ycombinator.com/item?id=13800862](https://news.ycombinator.com/item?id=13800862)

but this was a while back :), so not marking it dupe.

<comment><AUTHOR> @signa11 thanks for the note.

btw I've added a whole lot more examples since the last time, and I've also updated the library itself.</comment></comment>


Ask HN: Which HN articles have reached or surpassed 1000pts? - <AUTHOR>

In the wake of Edward Snowden's outing in The Guardian's article "Edward Snowden: the whistleblower behind the NSA revelations", which has just surpassed 1000pts on HN, I am curious to know what other HN posts have reached this 1000pt mark? <comment><USER2> I counted 69, so this article is probably #70 : [https://www.hnsearch.com/search#request/submissions&q=+&sort...](https://www.hnsearch.com/search#request/submissions&q=+&sortby=points+desc)

An important detail is that the size of HN had increased, so to compare the articles the points should be adjusted. I remember a story a few yeas ago that got a lot of points, but when I searched it I realized it has only ~89.</comment>