I'll Give MongoDB Another Try In Ten Years - <AUTHOR> http://diegobasch.com/ill-give-mongodb-another-try-in-ten-years <comment><USER2> It's almost like the commenters who are bashing the author of the post did not read the bit in bold, which is his main point:

 _If you tell a database to store something, and it doesn't complain, you should safely assume that it was stored._

This has nothing to do with the 2Gb limitation. Nowhere in the documentation does it mention that it will silently discard your data. What will happen with the 64-bit version if you run out of disk space, more silently discarded data?

I know a lot of you may have cut your teeth on MySQL which, in its default configuration, will happily truncate your strings if they are bigger than a column. Guess what? Anyone serious about databases does not consider MySQL to be a proper database with those defaults. And with this, neither is MongoDB, though it may have its uses if you don't need to be absolutely certain that your data is stored.

EDIT: Thanks for pointing out getLastError. My point still stands, since guaranteed persistence is optional rather than the default. In fact, reading more of the docs points out that some drivers can call getLastError by default to ensure persistence. That means that MongoDB + Driver X can be considered a database, but not MongoDB on its own.

I'm just struggling to imagine being willing to lose some amount of data purely for the sake of performance, so philosophically it's not a database unless you force it to be. Much like MySQL.

EDIT2: Not trying to be snarky here, but I would love to hear about datasets people have where missing random data would not be an issue. I'm serious, just want to know what the use case is that MongoDB's default behaviour was designed for.

EDIT3: (Seriously) I'm sure MongoDB works splendidly when you setup your driver to ensure that a certain numbers of servers will confirm receipt of the data (if your driver supports such an option), nowhere am I disputing that. But that number really should have a lower bound of 1, enforced by MongoDB itself. And to the guy who called me stupid: you are what's wrong with HN.

<comment><USER3> > Nowhere in the documentation does it mention that it will silently discard > your data.

Demonstrably false. <http://www.mongodb.org/display/DOCS/getLastError+Command>

"MongoDB does not wait for a response by default when writing to the database. Use the getLastError command to ensure that operations have succeeded."

<comment><USER4> It blows my mind that you'd have to post this at all. Who's writing to mongo without making sure the write succeeded?

<comment><USER5> The people over at MongoDB who wrote the tutorial? [http://api.mongodb.org/wiki/current/Ruby%20Tutorial.html#Rub...](http://api.mongodb.org/wiki/current/Ruby%20Tutorial.html#RubyTutorial- InsertingaDocument)

<comment><USER4> Then they fucked up.

<comment><USER6> "It's like, literally, right there in the brief manual. Takes an hour to read and understand."

Huh.

<comment><USER4> Yeah, manual. Like I said in the other comment.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> There's a bigger question here. I get why diego was flabbergasted by the default, and I also hear legitimate claims that the documentation should have been read. But what I want to know is: Why are MongoDB advocates in such a bad mood?

It's legit to criticize a language or a database. However, it seems to me that when MongoDB gets involved, the tone is far more aggressive and defensive. What's up with that? It's just software, bits and config files. It's not like someone called your mom a harlot.

Here's what I think. New developers, for a long time, have come into the industry and become overwhelmed with everything they need to learn. Let's take typical database servers. Writing a SELECT is easy enough, but to truly be an expert you have to learn about data writing operations, indexing, execution plans, triggers, replication, sharding, GRANTs, etc. As it's a mature technology, you start out barely an apprentice, with all these experienced professionals around you.

In recent years, software development has really been turned on its head. We're not building apps using the same stack we've used for a long time: OO + RDBMS + view layer + physical hardware. The younger the technology, the better, it seems. In theory, a 3 year developer and a 20 year developer are now pretty equal when we're talking about a technology that's been around 2-3 years. That wouldn't be true if we were dealing with say, OO design patterns. (Even when new languages come along, you still get to keep your experience in the core competencies.)

Attacks on these new technologies are perceived as an assault on this new world order, and those who have walked into being one of the "newb elite" respond emotionally to what they see as a battle for the return to the old guard. Am I totally off base here?

<comment><USER8> I think you're way off base. Even assuming your claims about experience with traditional databases (which I disagree with), we don't see the same kind of emotional tone when talking about equally new datastores like redis or couchdb.

Mongodb was very aggressively marketed; its advocates produced benchmarks comparing it directly to traditional relational databases as though the use cases were the same. I think that set the tone for future discussion in a way that's still being felt.

If you're as old as your opinions suggest you'll remember the early days of Java were very similar - Sun marketing pushed it no end, and so tempers ran high and discussions were emotionally charged in a way that never happened when talking about perl or python or TCL.

<comment><USER7> I'm not terribly old: 35. Been doing web development as a career since 1999.

More relevant, is my experience. I didn't come in when Java came out. I started (1997-1998) with some high-level dynamic web languages: ASP classic, ColdFusion (To this day, I still do CF - I'm a CF user group manager and I speak at CF conferences). Building HTML and JavaScript since 1996 (GeoCities, HotDog, and HomeSite). Nerded around with programming 1995-1997 in high school (TI Basic, Pascal, and Qbasic) In the days when I started web development, a lot of folks were still monkeying around with Perl and flatfiles. I can't really speak to early days of Java: until 2000, didn't really use it. ColdFusion 6 went from C++ to Java, at which point CF devs ran on the JVM and could target it.

From the beginning I was a consumer of RDBMSes. Started with Access and moved on to SQL Server. There wasn't a need to know the full DB, only the pieces you needed for CRUD. Perhaps for newbs that has changed, and they have to learn the full SQL administrative experience. Personally I doubt that. Do some db migrations in Rails: you don't even need to know what SQL engine you're running on. (A good thing, IMO, but still means a lesser body of knowledge)

Good point that a lot of products try so hard to be the "new sexy" that they suggest an inaccurate comparison, or at best, implement a subset of what they're trying to replace.

</comment></comment></comment><comment><USER9> It's pretty incredible that the author of a post called "I'll Give MongoDB Another Try. In Ten Years." criticises a comment on this same post telling him to read the tutorial all the way through as "unnecessarily aggressive".

Aside from that, though, the 32 bit limitation is clear in the documentation and present on the download page. It's fine not to read the documentation before you use something but you can't then complain that it did something you did not expect. Mongodb is a little different from other databases. So is Redis. You can't blow everything off that is conceptually different.

<comment><USER10> See my comment below. If you use a package manager you never see the warning.

<comment><USER11> One would presume, if you're going to use a database you would check it's limitations first, this one is well documented.

There are plenty of valid arguments for not using MongoDB, but this is the weakest I have seen so far.

<comment><USER12> I don't think his problem was the size limit, it was the lack of exceptions being raised or log errors being recorded.

</comment></comment></comment></comment><comment><USER13> We've been interviewing candidates to join our team for a few months, and I've also lent some interviewing support to other startups in our area.

I've noticed a trend across about 20+ candidates, all of whom are smart people: people are using Mongo without actually understanding what the hell it's trying to solve by getting away from the RDBMS paradigm.

I'm not sure if this is because 10gen markets it as a general purpose tool, but I have yet to talk with a candidate who can actually describe why they were using the DB vs. a SQL database. I'm all for learning new things, but I can't help but wonder if the string of negative MongoDB posts is coming from people who pick it b/c it's new, then realise pretty far in that this is nothing like a normal DB, and "having no schema" isn't really a reason to go with a tool as foundational as a data store.

I think Mongo is great for really specific problems that its designed to solve. It's probably pretty bad for a general purpose tool, but I'd be surprised if anyone serious actually considers it one.

<comment><USER14> What about scalability? Trying to cluster and shard MySQL is a very difficult task, but with MongoDB it is trivial. No schema can be good, but scaling out easily is the big plus I see.

<comment><USER6> >but with MongoDB it is trivial

We did not have this experience when I worked at a large datamining company. It was a nightmare.

<comment><USER15> It would be interesting to hear what the difficulties were.

Mongo markets ease of sharding as an advantage, and if that is not the case at times, it limits the attractiveness of losing RDBMS features.

</comment></comment></comment></comment><comment><USER16> I think blaming the user here is partially valid (he didn't read the docs), but that's not the whole story.

There is a discontinuity between the ease-of-use story and the blame-the-user story, regardless of how well documented the async insert behavior is.

And it doesn't have to be this way. There are ways of designing interfaces, APIs, and even naming that go a long way to prevent your users from shooting themselves in the foot.

Take postgres. It also supports at least a couple kinds of async insert, one of which is a part of libpq (postgres C client library). It's called "sendQuery" and it's documented under the "Asynchronous Command Processing" section. It's hard to imagine a user trying to use that and expecting it to return an error code or exception. Even if the user doesn't read the docs, or reads some fragment from a blog post, they will still see that the name suggests async and that it returns an int rather than a PGResult (which means it obviously doesn't fit into the normal sync pattern).

There is no reason mongo couldn't be clear about this distinction -- say, rename "insert" to "async_insert" and have "insert" be a wrapper around async_insert and getLastError. But instead, it's the user's fault because they didn't read the docs.

Careful API design is important to reduce the frequency of these kinds of errors. In postgres, it's relatively hard to shoot yourself in the foot this badly in such a simple case. I'm sure there are gotchas, but there is a conscious effort to prevent surprises of this sort.

<comment><USER17> > There is no reason mongo couldn't be clear about this distinction -- say, > rename "insert" to "async_insert" and have "insert" be a wrapper around > async_insert and getLastError. But instead, it's the user's fault because > they didn't read the docs.

Because if you don't read enough of the docs to understand that 'insert' is asynchronous insert, you don't understand MongoDB and haven't done your research.

Why should 'insert' default to synchronous? Why shouldn't we instead have a sync_insert function instead? The only reason is that you're assuming familiarity for people coming from SQL/synchronous-oriented DBMS, but why should they be forced into an awkward design just because it's what people are familiar with from _other_ DBMS?

<comment><USER18> A good system is forgiving; it encourages exploration; if there's a choice between safety and performance it defaults to safety. If/when profiling shows the safe behaviour to be a bottleneck, then users can Google the issue and discover "Oh, I just need to set flag X; I can live with the consequences here".

Expecting the user to be an expert in your product from the start is simply not realistic; a well-designed system facilitates use by people of varying levels of expertise.

<comment><USER17> > A good system is forgiving; it encourages exploration; if there's a choice > between safety and performance it defaults to safety.

Not if you're choosing a system that's explicitly marked for performance over safety.

> Expecting the user to be an expert in your product from the start

The 'product' in this case is a non-relational database, not an iGadget. The user can and should be expected to be familiar with the main strengths and weaknesses of the database as a whole.

There is no way you can convince me that someone who has done a reasonable level of due-diligence in investigating MongoDB can be surprised when it behaves asynchronously.

<comment><USER18> Kudos to you for doing your research. If you're saying "don't use MongoDB without doing at least N days of research first", then you're very much at odds with (my perception of) the 10gen marketing message.

I think you're right though: MongoDB should not be used without _lots_ of research into its limitations.

<comment><USER17> > I think you're right though: MongoDB should not be used without _lots_ of > research into its limitations.

That's true about any database, not just MongoDB; nothing new here.

> then you're very much at odds with (my perception of) the 10gen marketing > message.

10Gen is fairly straightforward about the original issue, having blogged openly _several_ times about their decisions - but at the end of the day, any engineer should do research beyond the simple marketer's pitch.

I won't doubt that there are people who make snap judgements about fundamental architecture based on marketing pitches[1], but that's very unfortunate, and the marketers really can't be blamed, _especially_ when they make no effort to conceal the truth or deceive you!

[1]<http://www.pinaldave.com/bimg/dilbert5.jpg>

<comment><USER18> > That's true about any database, not just MongoDB; nothing new here.

That's exactly the point where we started. A well-designed system fails "safe"; it should obey the principle of least surprise. Specifically: MongoDB should default to synchronous writes to disk on every commit; official drivers should default to acknowledging every network call; MongoDB shouldn't allow remote access from the network by default. Once you want higher performance or remote access, you can read about the configuration options to change and learn on-the-fly, evaluating the trade-offs as needed.

Other systems are safe by default (e.g. PostgreSQL), and their out-of-the box performance and setup complexity suffers because of it. MongoDB could ship "safe" (with the same trade-offs), but chooses not to. That sort of marketing- led decision-making has no place in my technology stack.

<comment><USER17> The Principle of Least Surprise has local scope. You may be surprised to find asynchronous writes on an arbitrary database, but not for a database that is _documented, advertised, and marketed_ as asynchronous-by-default.

'Surprise' is relative to the current environment and paradigm (in this case, asynchronicity)- if you find _that_ surprising, then that means that you should have read the basic documentation properly.

> MongoDB could ship "safe" (with the same trade-offs), but chooses not to.

Because that's one of the main points of choosing MongoDB...

<comment><USER19> This "main point" is never mentioned in their philosophy page. And the introduction mentions "Optional streaming writes (no acknowledgements)" which sounds like the default is synchronous writes.

<http://www.mongodb.org/display/DOCS/Philosophy>

I admit that the default unsafe tuning of MongoDB becomes quite obvious when you read more of the manual, but I can hardly say 10gen is without blame for causing this confusion.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> This is the latest in a long line of negative posts on MongoDB based solely on first impressions because either:

1) it does not behave exactly like SQL

2) the user didn't read any more than a Quickstart Guide

3) the user fundamentally misunderstands the aim of the new technology or the application it is intended for

Ember.js suffers from the same ignorance.

What makes it worse is all the morons who upvote without even reading the detail purely because the title reinforces some misconceived bias they already have.

'NoSQL' is part of the problem. This technology has absolutely no comparison with SQL other than it persists data.

<comment><USER21> I'm sorry. Which part of "It silently ignored my data" do you not understand?

You call people "morons", yet it appears that you did not read the article yourself.

Whether SQL or not, scalable or not, old or new, or whatever... Is completely immaterial here.

When a database silently stops accepting data, and apparently has done so for 3 years, you have to at least admit that there are strange design goals at play.

Now, the entire claim of the article might be incorrect. Did you verify that yourself?

Edit: Spelling

<comment><USER22> It's stated plainly and prominently in <http://www.mongodb.org/downloads> that 32-bit version is limited to 2GB. It's mentioned elsewhere in the documentation, but the OP didn't bother to read them. "A gem and two lines" and it worked, so he expected it to work forever. That's not how engineers usually work. Most of the time, they over-engineer, not the other way around! They research the hell out of any new technology they want to use. I'm definitely less talented than OP and others on HN, but even I know a hell of a lot about Redis, MongoDB and CouchB, _and I haven't even started to write a line of code_.

And anyone who has read more than an introduction to mongo knows that you SHOULD use getLastError to be safe. If you do that, no data will be dropped.

<comment><USER23> I think they need to change the word "limit". This is what "limit" means:

        [root@li321-238 tmp]# dd if=/dev/zero of=./filesystem bs=1M count=128
        [root@li321-238 tmp]# mkfs.ext2 filesystem
        [root@li321-238 tmp]# mount -o loop filesystem myfilesystem
        [root@li321-238 tmp]# dd if=/dev/zero of=myfilesystem/im_too_large bs=1M count=129
        dd: writing `myfilesystem/im_too_large': No space left on device

that is, a "limit" means, the program stops, complains. It's "limited".

A program that continues along without issue, only changing its behavior in some unannounced (documented or not) way, is not "limited". It's free as a bird.

</comment></comment></comment></comment><comment><USER24> While this article is a bit flippant, I think ten years is a pretty good number when you consider the vast amount of engineering effort that has already been poured into projects like Postgres.

This brings me back to the recent discussion about reading other people's code: it is almost certainly smarter to extend an existing database until it's capable of meeting your needs, rather than write one from scratch.

The fact that many programmers don't see it that way is a testament to their irrational fear of diving into other people's code.

<comment><USER25> 10 years and PostgreSQL still has no easy, manageable solution for replication or sharding. And it's JSON support is still nothing more than a bolted on hack on top of a BLOB.

People need to stop acting like PostgreSQL is some holy grail database. It isn't.

<comment><USER26> Correctly and efficiently querying sharded tables is not only a very complicated dark art but also heavily patented. I thought they had a replication story, though.

</comment></comment></comment><comment><USER3> Not reading the documentation (or hell, the red "note" text under each 32 bit download link at <http://www.mongodb.org/downloads>) for basic limitations will bite you in the ass in 2022 just as easily as it will in 2012.

<comment><USER10> Except that it was hidden deeply in the documentation. It's like making a car that has the gas and brake pedals switched, and then blaming accidents on people not reading section 5 of the owner's manual.

I'm hardly an inexperienced programmer. I've used Cassandra, SimpleDB, Voldemort, etc. I wrote part of the Inktomi Search Engine in the 90s, and plenty of (what today would be called) NoSQL stores over the years.

A default that's so counterintuitive for a database should be featured prominently with a huge neon sign. It wasn't in the Ruby tutorial, or in any of the many documents I read. It's buried deep in the Mongo website, and the first Google match about the 32-bit limitation is a blog post from 2009.

<comment><USER27> It's on the download page.

<http://www.mongodb.org/downloads>

<comment><USER10> I installed it through Ubuntu (apt-get). Most people must (or should) be installing MongoDB through a package manager. Once again, assuming that people will see the warning because it's on the download page is shortsighted.

<comment><USER28> See dmaz's comment about the log files. I'm a complete mongodb noob as well, but found out pretty quickly that there's a 2GB limit on 32bit systems. It really is hard to miss.

</comment></comment></comment></comment></comment><comment><USER29> I'm a pretty big detractor of mongo, but I don't agree with this post. One of mongo's main design decisions is to defer writes, making this sort of thing possible. I think it's a crappy tradeoff but it is one of the things that makes mongodb mongodb. If you use it without knowing this you haven't researched it well.

<comment><USER30> When is this behaviour useful? (Benchmarks don't count)

<comment><USER31> Well, any time you'd rather have speed over completeness. Maybe you're aggregating tweets from the Twitter API and if the occasional one goes missing, it's not a big deal, or perhaps you can grab it on the next update. Maybe you're generating a real-time stats dashboard for your site and if one pageview gets lost every million, it's not a big deal.

Look, I agree that in most cases you probably want to do everything you can to make your data 100% complete. But failed writes should be really rare, and there are plenty of times I'd trade the rare missing write for cheaper/faster database servers.

<comment><USER30> This wasn't the occasional write though, this was every write after 2GB. That should set off alarm bells somewhere.

<comment><USER32> The way MongoDB is designed, this would be outside the scope of a driver. A storage library that's based on the ruby MongoDB driver could certainly do it, though. That's what the OP ideally should have been using. In fact MongoDB would be a good choice for his use case, if he would switch to a 64-bit VM and handle error conditions (heh).

</comment></comment></comment></comment></comment><comment><USER33> I'm using MongoDB for experimental work for about 6 months, it has a few amazing advantages, it's the MVP / POC king, you just do it, it's the agile iteration master, it's definitely not the best choice for doing any statistics, or any financial like transaction handling.

However, it starts to feel like Anti MongoDB is just considered cool today, when I see someone that worked with MongoDB for a year, upgraded to 2.2, knows it inside out and still hates it, I would listen, and start to worry. but until then, I'm going to keep using it, and saving time.

<comment><USER34> Agreed. I've been using it almost daily for 2 years. It was not an instant learning curve (what DB is?), but it's an absolute joy to develop for.

People who would rather not bother, can stick with their tools, work slower, and be happy.

</comment></comment><comment><USER35> I used MongoDB one afternoon, and guess what! It doesn't have table-locking writes?! :)

In all seriousness, I built a 10 machine Mongo cluster, talked with a 10gen consultant a full day, went to Mongo meetup, and ran all sorts of benchmarks before ever using it in production. I still don't feel like I have the expertise to write a snarky blog post about it.

<comment><USER36> > It doesn't have table-locking writes?! :)

Not really following the snark there. Are you trying to compare MongoDB to MySQL's MyISAM storage engine? Like there aren't numerous other extremely valid RDBMS solutions out there, which don't do table locks during a write? (MySQL InnoDB, Percona, Maria, Aria, Postgresql, Firebird, etc...)

<comment><USER37> No. He is saying out of all faults Mongo has - blog author picks the one which is rather well known.

<comment><USER17> Worse, it's the necessary tradeoff of one of MongoDB's self-proclaimed benefits (asynchronicity) - it's not an edge case; it's part of the core reasons you'd switch to MongoDB (or at least take into account when making that choice). If that surprises you, it's because you haven't switched for the right reasons _or_ haven't researched enough to know that every benefit has a tradeoff (and to know _what_ those tradeoffs are).

To give an analogy, it'd be as if someone read this post and decided to use a SQL database _solely_ because they care about write-durability... and then complaining when they "suddenly" encounter an error when trying to include an extra field with an INSERT on-the-fly. ('You mean SQL has fixed schemas??')

<comment><USER38> Where on this page is the word asynchronous mentioned:

<http://www.mongodb.org/>

If they just wrote asynch on that page somewhere most experienced programmers would immediately understand the implications. And also understand how the amazing performance was being achieved.

</comment></comment></comment></comment></comment><comment><USER39> The author doesn't mention if he had called getLastError after inserting data:

<http://www.mongodb.org/display/DOCS/getLastError+Command>

The MongoDB "way" is that clients know the importance of their data and can choose write strategies which make the proper trade-off between insertion throughput/latency and durability/consistency.

</comment><comment><USER40> First, your friends with MongoDB are fsync and safe. These are both documented and discussed in more than a few places: [http://nosql.mypopescu.com/post/1052759609/mongodb-safe- and-...](http://nosql.mypopescu.com/post/1052759609/mongodb-safe-and-fsync- explained)

So, assuming you are writing an ecommerce application, here's where I think these flags come in.

\- Session data: fsync = true. Wait for a response, and ensure it's written to disk

\- Internal web analytics: safe = false. Who cares if it's written, I've got an application to serve!

\- Orders: fsync = true. I know, RDBMS, transactions, blah blah blah.

People tend to look at NoSQL and wonder why it doesn't function like MySQL, then they loudly complain how bad the software is. Nobody is writing articles about how Memcached doesn't function like MySQL.

</comment><comment><USER41> Maybe in another 10 years, he'll have managed to lay hands on the elusive 64-bit computer.

</comment><comment><USER42> I had a very similar experience about a year ago. Except instead of running out of RAM on a 32-bit instance, I was running out of disk on a 64-bit instance. That's right, my database ran out of disk, and the driver didn't throw an exception.

Yes, I realize that there's a "safe=True" option to my python driver. But I'm _writing to a database._ As others have said here and elsewhere, the default behavior of a database and its drivers should be to complain loudly when a write fails. It is ridiculous that safe!=True by default. If I want to turn off this feature to improve performance, I will.

</comment><comment><USER43> I stopped reading at "WTF zomg LOL zombie sandwiches!". Just another script kiddie who can't read documentation and blames his tools instead.

</comment><comment><USER44> Does he have a point that it should have very vocally complained to him that his size limit have been reached, and records were not being stored?

Yes. Without question.

Is this his own fault for not reading the documentation and understanding that he should have opted for the 64bit version outright?

Yes. Without question.

</comment><comment><USER45> When I wrote some hobby code for Postgres using the PHP driver I had to manually check error codes after each and every operation. This came as no surprise to me.

Exception throwing database drivers are a relatively new thing not an old thing. The only thing MongoDB does differently is that the writes are fire and forget in that the database hasn't returned a response of any kind when the function returns.

In native code you can forget about using exceptions in a database driver because exception handling can be exceptionally broken on some platforms. SmartOS I am looking in your direction.

<comment><USER46> Exception throwing is not that new. It's just a question of what is the style in the language you're using. In Java, for example, throwing exceptions has been the norm since JDBC was invented in 1997. PHP is definitely different in that exceptions are rare. Same story with C++. I'm not super experienced with Ruby, but they seem pretty common there, so I would've expected to get one.

</comment></comment><comment><USER47> The world is moving towards 64 bit. 9 out of 10 machines I lay hands on run 64 bit. Just move on and stop complaining.

<comment><USER3> In fairness, this would eventually happen on a 64 bit machine too, just not as quickly.

No excuse for not reading the docs, though.

<comment><USER48> The 64-bit limit is 8.6 exabytes. There won't be anyone using Mongo that runs into that limit this century.

<comment><USER49> But when you run out of disk space (as one of the posters wrote) it doesn't complain either. And running of of disk space happens.

</comment></comment></comment></comment><comment><USER30> Interesting that you're quoting the zen of python, but using ruby. I wonder if the python mongo client would have the same behaviour.

There seems to be a number of people commenting, telling you to read the documentation, but I'm with you, that is completely counter-intuitive behaviour and should be viewed as a bug.

<comment><USER50> This has nothing to do with the client library, so it would not matter which language you use to interface with MongoDB.

<comment><USER30> As I understand it, it has everything to do with the client library, some clients may call getLastError on every operation and raise errors when they occur, for example.

<comment><USER51> PyMongo, for instance, can do this for most operations by passing safe=True.

</comment></comment></comment></comment><comment><USER52> A lot of bashing of MongoDB lately is a sign to give a technology at least a try.

<comment><USER34> Great point. That's usually how you to tell when a technology is starting to disrupt things. Really smart people / experts in their field (which Diego definitely is) start to bash it. In this case, he has a point but has way overblown things. But that's ok, that means Mongo is on the right track.

<comment><USER53> I am trying to reconcile the ideas of "Mongo is on the right track" with burning a user by losing data.

</comment></comment></comment><comment><USER54> Just read the documentation and learn the APIs, this is what happens when you just copy and paste code from a tutorial.

<comment><USER10> This type of attitude is not constructive. Of course I read the documentation. Much more than "copy and paste from the tutorial." I looked at tons of code samples as needed, read blog posts, etc. The limitation wasn't obvious at all.

This reminds me of the attitude that I had to correct in developers that worked for me:

\- There is a huge difference between "it works" and "it does what the user expects in a friendly way."

Steve Jobs said that if you need to read a user manual (particularly to do the most vanilla usage of a product), the problem is the product. Not you.

<comment><USER55> > Steve Jobs said that if you need to read a user manual (particularly to do > the most vanilla usage of a product), the problem is the product. Not you.

He's talking about consumer products, not databases that were intended for use by technology experts. There's a big difference there.

The onus is on you to understand the limitations of software before you start using it. You complain that the 32-bit warning doesn't show up in the package manager, but you still should have read the documentation before committing to a new technology. It's that simple.

Is it a flaw that mongo doesn't work well on 32 bit systems? Maybe. Probably.

Is it a flaw that you didn't do the requisite research before committing to a database and subsequently complaining about it? Definitely.

</comment></comment></comment><comment><USER56> All I'm getting from this conversation is that there is a heavy prevalence of Stockholm Syndrome among MongoDB users.

</comment><comment><USER57> I ran into this same nasty surprise building a prototype to store requests in Mongo instead of Postgres. It was enough to scare me away, too. Glad I noticed it while it was still just a script+Makefile simulation.

Another problem with Mongo I never heard anyone else raise is that there are no namespaces. If I install Mongo, all the tables/collections live in the same namespace. What if I want to use it for multiple projects? How do other people solve this problem?

<comment><USER14> You have multiple databases, just like a sql solution. One database per project.

<comment><USER57> Can you elaborate please? With a Postgres/MySQL/Oracle installation I can say `CREATE DATABASE` and get a new namespace. I couldn't find anything like that with Mongo. Am I just missing something?

<comment><USER48> "use foobar" creates a new database called foobar.

<comment><USER57> Ah, somewhere along the line I got the impression that in Mongo a "database" and a "collection" were the same thing, but that's not true. Glad to know this was just my mistake!

<comment><USER48> Nope. Collections are analogous to tables, and documents to rows.

</comment></comment></comment></comment></comment></comment><comment><USER58> I agree with posts stating that you should read the docs before using a tool you don't know. But I also think that these two really important points should be mentioned in the Getting Started guide, in bold: \- The 32bit 2GB limitation (seriously when I started with MongoDB I wasn't expecting this!) \- The fire-and-forget policy

These are really not points to be discovered in chapter whatever of the docs.

<comment><USER25> It's on the download page:

<http://www.mongodb.org/downloads>

<comment><USER58> Indeed. It is definitely not big enough though.

</comment></comment></comment><comment><USER59> Two paths for working with new technologies:

\- Download, Brief 3rd party tutorial, Production, Break, Complain, RTFM / Complain

\- RTFM, Smile, Download | Move On, Staging, Production

Seems most of the issues from this article came from a lack of reading and investigating.

</comment><comment><USER60> I can think of several cases where throwing an exception is counterintuitive to Mongo's design and applications. Let's say, if your app returns control to the user while storing data asynchronously, throwing an exception might not be the best way of handling errors. In fact, if throwing exceptions were Mongo's default, I wonder how long it would take for a blog post entitled "Mongo blew up my app" to appear.

</comment><comment><USER61> Interesting you switched to Couch. I was hesitant to recommend it reading the post because I feared you were turned off JSON stores entirely, glad to hear that's not the case.

In general it feels like Couch actually takes storing data seriously. Append- only and whatnot. It's slower and a little bulkier than Mongo, but it does the important things right (1.0 bugs notwithstanding.)

I'd love a follow-up blog post on your experience with Couch.

</comment><comment><USER33> Isn't the write result containing the info? e.g. if the write failed, it will contain the error if you just check for it? if so, and I haven't checked (but I assume it's so) then this post is equivalent to ranting on Go's lack of exception handling. like it, don't like it, it is what it is, you can either use it, or fork it and make your own database / language.

</comment><comment><USER62> The fact that this is on the front page shows that HN is no longer a real news trading site for real hackers. :(

</comment><comment><USER63> i welcome the post. even though most of my stuff runs on 64bit, i actually do have a few 32 bit systems here and there. I never knew. Because as the op mentions it's not written anywhere _obvious_.

another thing I didn't realize was that because of the memory mapped systems which i guess is fine performancewise it's hard to estimate memory usage on a machine. from what I understand there is no possibility to limit the memory usage. Which means that the only way you can limit the amount of memory used is by keeping the size of the database below your memory. quite important things to know imho.

here's an interesting post mentioned in the comments: <http://www.zopyx.com/blog/goodbye-mongodb>

<comment><USER22> > _it's not written anywhere _obvious_._

Isn't <http://www.mongodb.org/downloads> an obvious place?

<comment><USER49> Yeas, there is a small "note" there. But for me the problem is not that the author didn't know about 2GB data limit.

The problem is that Mongodb didn't complain when he was inserting data above the limit. A data store doesn't complain when it runs out of space? It should be mentioned as the biggest problem with 32bit version.

</comment></comment></comment><comment><USER64> One of the main reasons I hear people advocating MongoDB is it's ease of horizontal scaling, via replication and auto sharding. I wonder how many projects have such large data sets that they really require sharding of their data?

I understand having another node or two for fail over but I reckon with the spec of the largest offerings from AWS or Linode most people will never need to worry about this and can manage everything on one Postgres or MySQL db. Why complicate things before you have to.</comment>


Go code refactoring: the 23x performance hunt - <AUTHOR> https://medium.com/@val_deleplace/go-code-refactoring-the-23x-performance-hunt-156746b522f7 <comment><USER2> One of the things I like about go is that the path of least resistance is straightforward code. Getting a program working and working correctly while being straightforward is usually the starting point to getting a correct program that runs fast enough. Go's tools for testing and benchmarking and it's compilation speed make it relatively easy it iterate on a program while tuning it.

This example was really nicely done; it's a classic problem, parsing input data. Some problems, on the other hand, require a different approach. See for example Knuth's wonderful introduction to boolean satisfiability in _The Art of Computer Programing, Vol 4B_ , see [1]. Here the algorithms are really important to performance.

[1] [https://www.youtube.com/watch?v=g4lhrVPDUG0](https://www.youtube.com/watch?v=g4lhrVPDUG0)

</comment><comment><USER3> "The execution time is now dominated by the allocation and the garbage collection of small objects (e.g. the Message struct), which make sense because memory management operations are known to be relatively slow."

A perfect example of why Go should switch to a generational garbage collector with bump allocation in the nursery, like Java HotSpot, .NET, V8, etc. etc. use. There are consequences to Go's decision to make extreme throughput sacrifices in order to optimize for latency (low pause times) above all else.

<comment><USER4> Ian Lance Taylor replied to this question once: ([https://groups.google.com/d/msg/golang- nuts/KJiyv2mV2pU/wdBU...](https://groups.google.com/d/msg/golang- nuts/KJiyv2mV2pU/wdBUH1mHCAAJ))

> Now let's consider a generational GC. The point of a generational GC relies > on the generational hypothesis: that most values allocated in a program are > quickly unused, so there is an advantage for the GC to spend more time > looking at recently allocated objects. Here Go differs from many garbage > collected languages in that many objects are allocated directly on the > program stack. The Go compiler uses escape analysis to find objects whose > lifetime is known at compile time, and allocates them on the stack rather > than in garbage collected memory. So in general, in Go, compared to other > languages, a larger percentage of the quickly-unused values that a > generational GC looks for are never allocated in GC memory in the first > place. So a generational GC would likely bring less advantage to Go than it > does for other languages.

> More subtly, the implicit point of most generational GC implementations is > to reduce the amount of time that a program pauses for garbage collection. > By looking at only the youngest generation during a pause, the pause is kept > short. However, Go uses a concurrent garbage collector, and in Go the pause > time is independent of the size of the youngest generation, or of any > generation. Go is basically assuming that in a multi-threaded program it is > better overall to spend slightly more total CPU time on GC, by running GC in > parallel on a different core, rather than to minimize GC time but to pause > overall program execution for longer.

> All that said, generational GC could perhaps still bring significant value > to Go, by reducing the amount of work the GC has to do even in parallel. > It's a hypothesis that needs to be tested. Current GC work in Go is actually > looking closely at a related but different hypothesis: that Go programs may > tend to allocate memory on a per-request basis. This is described at > [https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft...](https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk- > tcRRJcDFANj2VwCB0/view) . This is work in progress and it remains to be seen > whether it will be advantageous in reality.

<comment><USER3> I'm aware of Ian's arguments. He's (a) too dismissive of the huge throughput benefits of generational GC; (b) ignoring the fact that both HotSpot and C# allocate objects on the stack too, with effectively superior escape analysis to that of Go because of speculative devirtualization; (c) neglecting the fact that Go's situation is almost the same as that of .NET, where the generational hypothesis certainly holds; (d) treating "request-oriented" GC as something other than generational GC, when it's really just a somewhat limited form of generational GC.

It's just a pointless thing for the Go team to dig in their heels over. They should implement generational GC, like most other languages with optimizing compilers do.

<comment><USER5> I've seen you rebut Ian's remarks a number of times. I'm not sufficiently knowledgeable about GCs, but I would really like to hear some of his responses to your criticisms. I think I (and probably lots of others) would learn a lot from the debate.

<comment><USER6> There is most likely none.

Go team at Google would have enough real usage feedback from running Go systems in their massive data centers. They most likely find it reasonable with all things considered.

This debate about Generational GC is theoretical in nature, claiming Java GC design is universally applicable irrespective of language and its use cases.

There may indirect approval to Go approach by Java when developing value types:

[http://openjdk.java.net/jeps/169](http://openjdk.java.net/jeps/169)

key Quote:

"In modern JVMs, object allocation is inexpensive, with a cost comparable to out-of-line procedure calling. But even this cost is often a painful overhead when compared to individual operations on primitive values. Thus, Java programmers face a binary choice between existing primitive types (which avoid allocation) and other types (which allow data abstraction and other benefits of classes). When they need to define small composite values such as complex numbers, pixels, or pairs of return values, neither approach serves. This dilemma often has no good solution, and the workarounds distort Java programs and APIs. "

If we go by claims that Java and its GC supporters always made:

1) RAM is Cheap 2) GC is incredibly fast so as not to worry about allocation.

There should be no need to waste tremendous multi year engineering effort to add value types to Java.

<comment><USER7> That's a straw man. Just because GC is fast doesn't mean it can't be made faster.

> Go team at Google would have enough real usage feedback from running Go > systems in their massive data centers. They most likely find it reasonable > with all things considered.

That's not just an appeal to authority, that's a big assumption. As far as I know, no Go maintainer has claimed that they have gathered large amounts of performance data and concluded that a generational GC wouldn't improve performance.

Also, calling generational GC "Java GC design" is disingenuous. Generational GC long precedes the creation of Java, and is used almost universally. Common Lisp (various implementations), .Net, the Lua VM (and the LuaJIT VM), BEAM, SELF, and Haskell (GHC at least) all use generational garbage collection. In fact, the current Go implementation is the only GC implementation I'm aware of that isn't generational.

<comment><USER8> Neither Lua nor LuaJIT use a generational design, they're both tri-color incremental mark and sweep collectors without a separate nursery.

There was a plan for a generational GC for LuaJIT [0] (not to mention a lot of other cool features and a very clever design) but it was never implemented (and at this point it probably never will be).

[0] [http://wiki.luajit.org/New-Garbage-Collector](http://wiki.luajit.org/New- Garbage-Collector)

<comment><USER5> Also, most of the listed languages don't have value types or added them expressly because their GC scheme was insufficient.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Great article. I was recently optimizing a go program. I was focusing on trying to improve the underlying algorithms, but then I realized after profiling that most of the time was spent creating temporary slices (many, many times). When I changed it to reuse existing slices, I saw a bigger performance gain than all the other effort combined.

<comment><USER5> Yeah, this is usually what I run into. I once had a recursive algorithm to collect a flat list of nodes in a tree. I found that building slices was the bottleneck, so I changed the signature to `func (t Tree) CountNodes() int` and `func (t Tree) Nodes(outSlice []Node)`, and then I'd call it like:

        nodes := make([]Node, t.CountNodes())
        t.Nodes(nodes)

Even though this required me to traverse the tree twice (once to count and the other to populate the list of nodes) the savings were considerable.

</comment></comment><comment><USER10> Kind of defeats the purpose of using golang for a task like this. The whole point of golang is using the little greenlet threads, but actually using them in this case is terrible on performance.

The remaining performance left behind is all in memory allocation and garbage collection - something you could optimize relatively easily if it were written in C. Such as by using a memory pool, so that you wouldn't need allocations or garbage collection at all.

Of course if performance isn't a big issue for your task, then none of this is really important.

<comment><USER11> It's surprising that the per-file Goroutines were so expensive, though. (The original per-line Goroutine, sure, that's excessive if you care about performance.) Just using long-lived workers seems non-idiomatic for Go, but it certainly pays big dividends in this example.

<comment><USER12> Per-file may have had other problems not related to the Go runtime, such as IO contention. I'm not going to check it, but it would be easy to verify that just by using a limited number of them at a time. Spawning a new goroutine in that case is not strictly necessary, but would still be good software engineering.

One of the problems I see repeatedly when people try to benchmark things with concurrency is when they don't specify a problem that is CPU-intensive enough, so it ends up blocked on other elements of the machine. For a task like this, I'd expect optimized Go to easily keep up with a conventional hard drive, and with just a bit of work, come within perhaps a factor of 2 or 3 of keeping up with the memory bandwidth on a consumer machine (including the fact that since you're going to read a bit, then write some stuff, you're not going to get the full sequential read performance out of your RAM), not because Go is teh awesomez but because the problem isn't that hard. To get big concurrency wins, you need a problem where the CPU is chewing away at something but isn't constantly hitting RAM or disk or network for it, such that those systems become the bottleneck.

<comment><USER13> Hi jerf, please note that

\- the benchmark was designed to repeatedly parse an in-memory byte slice (not the hard drive), thus IO contention is unlikely here ;

\- concurrency is a big win when IO is a bottleneck : keep processing dozens of things while some of them are waiting for data from network or hdd.

<comment><USER12> "the benchmark was designed to repeatedly parse an in-memory byte slice (not the hard drive), thus IO contention is unlikely here"

You could still be getting IO contention from the RAM system. RAM is not uniformly fast; certain access patterns are much faster than others.

"concurrency is a big win when IO is a bottleneck : keep processing dozens of things while some of them are waiting for data from network or hdd."

Concurrency is a win when IO is a bottleneck on a _single task_. Once you've got enough tasks running that all your IO is used up, adding more may not only fail to speed things up, but may slow things down. I'm speaking of situations where you've _used up_ your IO. The tasks you're benchmarking are so easy per- byte that I think there's a good chance you used up your IO, which at this level of optimization, must include a concept of memory as IO.

I think you'd be helped by stepping down another layer from the Go VM and thinking more about how the hardware itself works regardless of what code you are running on it. Go can't make the hardware do anything it couldn't physically do, and I'm getting a sense you deeply understand those limits.

</comment></comment></comment></comment></comment><comment><USER14> One trick I use is passing data files through the command line "sort", "uniq", and "wc" utilities. Those are great benchmarks to see how off you are from tuned code on your machine.

</comment><comment><USER15> Does anyone now profiling tools like that for node.js?</comment>


Beyond the doomsday economics of "proof-of-work" in cryptocurrencies [pdf] - <AUTHOR> https://www.bis.org/publ/work765.pdf <comment><USER2> I wonder why the paper didn't consider a constant block reward (i.e. pure linear emission) as a way to mitigate the issues caused by relying on transaction fees for long term security.</comment>


Maori legend of man-eating bird is true  - <AUTHOR> http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10597177

<comment><USER2> Why does the article present this as some kind of news? We've known of Haast's eagle for years and it's connection to the Maori legend is common knowledge among anyone who's heard of the bird.

(Still cool though.)

<comment><USER3> If I read it correctly, the article is implying that recent scientific study of the strength of the bird's pelvis has shown that the bird was likely to have been strong enough to make killer blows and crush prey (backing up the Maori legend) rather than just scavenge like a vulture.

<comment><USER4> That isn't a new discovery either. I've spent a great deal of time studying New Zealand and Māori culture and history specifically (so much that I actually changed my keyboard layout to Māori so I could write in the language)... this is old news. Like, generations old.

Edit:

Here's my fairly old page about this very topic: [http://kotiro.petermichaud.com/research/mythical- creatures/p...](http://kotiro.petermichaud.com/research/mythical- creatures/pouakai/)

They have a legend about the "Pouakai" that people have thought for a long time was the eagle. That old link proves this is old news (or maybe I broke the story first without realizing 8)

</comment></comment></comment><comment><USER5> It's amazing that this eagle most likely descended from the Australian little eagle. Imagine if Steller's Sea Eagles had made it to NZ. [http://upload.wikimedia.org/wikipedia/commons/8/82/Stellers_...](http://upload.wikimedia.org/wikipedia/commons/8/82/Stellers_Sea_Eagle_with_Man.jpg)

<comment><USER6> The problem is that ecological niche is a better determiner of size than ancestry...

<comment><USER5> Sure. But evolution prefers to reuse rather then innovate, this can often lead to interesting limits. For example, we still have a blind spot where the optic nerve connects to the retina, in all this time we have not evolved a blind spot free eye. Thus I was wondering if there are some odd limitations in how big, how quickly a little eagle can evolve, vs what the giant raptors could do. It was basically evolutionary fan.fic.</comment></comment></comment>


How Big Can a WorkFlowy Account Get? - <AUTHOR> http://blog.workflowy.com/post/31419974930/how-big-can-a-workflowy-account-gethnsearch

<comment><USER2> Didn't know these guys were still running. It's an interesting take on a interface that we're used to on the desktop. This is an open source project to me though and not a business. Build out your interface to host on other platforms (think DISQUS) and you guys would be solid.</comment>


Ask HN: What a manager did that helped you the most in your career? - <AUTHOR> <comment><USER2> Told me that he will fire me if I don't start to be responsible. I was 23 , it was my first job and I was procrastinating a lot. He saw the potential in me and wanted to kick off my responsible part. Since then I am a more responsible person.</comment>


In Carlos Ghosn's Escape, Plotters Exploited an Airport Security Hole - <AUTHOR> https://www.wsj.com/articles/plotters-exploited-airport-security-hole-in-ghosns-escape-11578270953 <comment><USER2> Wow, imagine if he had been caught attempting this getaway. Can't wait to read the book when it comes out.

</comment><comment><AUTHOR> [http://archive.is/qxAxK](http://archive.is/qxAxK)

<comment><USER2> thanks</comment></comment>


Ask HN: Moving away from the JAP tag - <AUTHOR>

I'm one of the two programmers working on an news application to be launched soon. The founders hired a marketing guy and some designers who are taking care of the interface and identity of the application.
I perceive that I'm being tagged as JAP(just a programmer) and I'd like to move away from that tag. If the startup is succesful it'd be nice to be considered for a higher level position.
I'd like to know if someone has/had in a similar situation. Any advice is very welcomed. <comment><USER2> First place to start is with a self-assessment, I'd think. What skills do you have outside of programming that can be helpful? What training have you done on other subjects? I also think that, before you say "I want to be something else", you need to defined what the "something else" is. It might be that "what you want" and "what you are currently" are not that close. It then falls on you to identify how to add those skills to your skill set, all while making sure your team knows your ambitions.</comment>


Show HN: Clean and simple declarative dotfile management - <AUTHOR> https://github.com/markrawls/punctual <comment><USER2> this is neat. i use a bare git dir for mine ([https://github.com/tmm/dotfiles](https://github.com/tmm/dotfiles)), but am looking to switch to something more opinionated and structured.

<comment><AUTHOR> I'm glad you like it! I initially used a bare git repo, but I didn't like the complex setup. I typically only work on my desktop machine but as I travel more for work I have to keep my canonical emacs and window manager configurations synchronized between devices, and Python is almost always available.

</comment></comment><comment><USER3> I'm interested because sometimes a line or two in a dotfile can make a big difference.

But I'm not sure I understand what a "dotfile manager" is. I generally just edit my dotfiles by hand. Could you explain more the problems that this solves?

<comment><AUTHOR> A dotfile manager in this context is a way to transfer them between machines, not to edit them. As long as you're able to sync the folder containing the dotfiles (through Dropbox or Git or a thumb drive), you can use this program to install/uninstall your dotfiles on whatever machine you're using.

It allows you to keep non-centralized files in a centralized location with very little setup, basically. My window manager config lives in several different directories around my home folder, but with this I just set a mapping from repo locations to the home directory and everything is handled automatically.

</comment></comment><comment><USER4> What does it do? The Readme and documentation fail to mention any actual functionality or purpose.

I don't even know if this is for your own dotfiles, or things like .*rc configuration in a project.</comment>


The Weirdness of 50s Bomb Shelters - <AUTHOR> http://allday.com/post/1192-the-weirdness-of-50s-bomb-shelters <comment><USER3> I really don't care for the snide tone of the article. Bomb shelters weren't weird; they weren't irrational. Recall that WWII had involved mass area bombing of civilian targets; many Europeans and Brits were alive in the 50s due to their nations' bomb shelter programs (I don't know about Japan or the Soviets, so I can't say there).

A nuke isn't some magical death-and-destruction device: it's just a really powerful bomb, which releases some really nasty pollutants. It's a difference of degree, not of kind: other bombs are weaker and still releases less nasty pollutants.

Sure, no home bomb shelter could survive a direct hit; but home shelters weren't made to resist direct hits: they were made to resist blast waves from hits on nearby cities; they were made to resist nearby conventional hits.

What made them less useful investments was a factor of several things. Precision targeting made area bombing less likely. Improved air defenses made large-scale WWII-style bombing raids less likely. It might also be the case that larger bombs made the risk-reward ratio different.

While cleaning out my grandparents' home a few months ago I came across a government civil defense pamphlet my grandfathers' firm had republished. It was full of good, scientifically-accurate, well-founded advice. It wasn't alarmist; it was up-to-date for the time when it was published.

It certainly wasn't weird to be prepared for something which at the time was likely to happen.

<comment><USER4> In related news, even the much-maligned "Duck and Cover" is full of good advice (which is coincidentally also useful against large meteors).

[http://en.wikipedia.org/wiki/Duck_and_Cover_(film)#Accuracy_...](http://en.wikipedia.org/wiki/Duck_and_Cover_\(film\)#Accuracy_and_usefulness)

</comment></comment><comment><USER5> The rational reason we don't build nuclear shelters anymore is because they're no longer effective. As weapon yields increased, it's become apparent that a concrete, underground hideout is not going to save you.

The irrational reason is that it's been 60+ years since atomic weapons were deployed, and we are confident the danger has passed. But really, there's plenty of weapons out there: [http://www.motherjones.com/politics/2011/11/map- nuclear-bomb...](http://www.motherjones.com/politics/2011/11/map-nuclear- bombs-power-weapons)

<comment><USER6> An underground hideout was never going to save you if you were near enough to the blast -- I always thought it was more of a way to protect yourself from nuclear fallout -- wouldn't that still be a valid reason to build a shelter?

<comment><USER7> You don't need reinforced shelters against fallout; above-ground buildings will do fine, as long as they can be made reasonably airtight (as additional protection, one could provide for material to replace broken windows)

Also, I think most shelters would be far enough from blast areas to provide some protection.

Eizo Nomura (who, amazingly, doesn't have a Wikipedia page) was 170 meter from ground zero in Hiroshima, and survived ([http://en.m.wikipedia.org/wiki/Atomic_bombings_of_Hiroshima_...](http://en.m.wikipedia.org/wiki/Atomic_bombings_of_Hiroshima_and_Nagasaki#The_bombing))

Akiko Takakura was above ground at less than 300m and lived for 60 years.

Even though later bombs were much more powerful that makes it likely that shelters at the edge of the blast radius can save lives.

Not all of them, but any program to protect civilians is a statistics game.

All the construction in SF isn't 100% effective against earthquakes, either.

<comment><USER8> Per Cresson Kerney, who's nuclear war survival stuff was actually tested against reality (vs. the deathtraps like at least one of those early shelter plans in the OP which was dreamed up by D.C. bureaucrats) says air tightness concerns are entirely overblown, so to speak ^_^. As I recall, the heavy stuff that falls out early is too big to be a concern of that sort, the light stuff decays to safe (enough) levels before enough of it drops and becomes an issue. We have a lot of data on this thanks to those eeeevil above ground tests.

What you need is plenty of mass between sources of radiation, i.e. fallout particles where they settle, and from "skyshine". In a sufficiently bad situation, which would be pretty common, a simple ditch where you made sure the fallout kept out and didn't accumulate on, say, the tarp you were using for that, you'd still get zapped by gamma rays that bounce off atoms in the air and come down into the ditch.

So the most basic expedient fallout shelter is a ditch with hollow core interior doors over it or the like, and foot of dirt on top, which through arching holds most of its weight.

An above-ground building is iffy, because you want, say, 3 feet of dirt all around you, and either a foot on top or if the roof is high enough that's not as much of an issue. Middle stories of a tall building could suffice.

</comment></comment></comment></comment><comment><USER9> I was born in the 1980s in Ukraine, so I have both an experience to share and questions to ask. Questions first: how common was it to build your own shelter in the US? Was it something that most people did? Most people talked about? Were the people who built them considered as irrational as people who build "zombie apocalypse" shelters today? Were they stocked with guns the same way that today's shelters are?

Experience: I grew up in Kharkiv, a city with roughly a 1m population, and the second largest city in Ukraine. Most people lived in large apartment complexes that were organized into planned neighborhoods. Imagine four buildings surrounding a common area that was laid out as a small park. Inside this common area that my building was surrounding were three bomb shelters. We called them "old", "weird", and "new" based on their outside appearance. The entrances were fairly small: just a triangle structure with a door and a staircase going down. We never went too deep into them when they were open (the authorities would periodically close them up), but at one point I remember using an iron bar to break the chain on one of them and leading a few of my friends down into it. I really wish we had good flashlights, rope, etc. (we were all around 11 or 12). We went down two levels and looked at these huge rooms. Some of them looked to be set up for bunk beds and it looked like the beds were at some point taken out. I did wander into a random room without a flashlight and almost took a step into a darker area before realizing that it was an opening with a step ladder down to the lower level. I would have been badly hurt had I taken the step before checking. This naturally scared me enough to get out of there in a hurry. The next week this particular shelter was closed up with a better chain so we didn't get to go in again.

Edit: I tried to find some pictures of what it looked like. Surely someone has taken some photos at one point, but sadly I cannot find anything that resembled the bomb shelters I explored as a child. I do have a strange fascination with the Cold War, so now I really wish I could go back and take some pictures/explore things as an adult.

<comment><USER10> Norwegian here: While building their house, my grandparents had (by law) to build the basement solid enough to be used as a bomb shelter, this was in the fifties i believe. I now live in an apartment block from early sixties and every block here (with around 24 apartments) has their own bomb shelter in the basement.

Even some of our subway stations are (rather large) bomb shelters. In my home village, both some of the schools and sport facilities all had their own bomb shelters which also had facilities for keeping gas out.

I do believe that is no longer required to build new bomb shelters but all existing ones has to be maintained and be ready to be used within a specified timeframe.

</comment></comment><comment><USER11> Photo number 9 isn't simply a bomb shelter, its a decommissioned ATLAS F missile base - [http://www.silohome.com/](http://www.silohome.com/)

The Las Vegas shelter is awesome, almost like an art installation.

<comment><USER12> When I got to that image, that's when I completely gave up on it. This article was complete spam, and devoid of any real content.

Somebody spent five minutes googling "bomb shelter" and then posted a few photos with some asinine remarks.

</comment></comment><comment><AUTHOR> This was particularly memorable for me as during that time, my dad, a wheat farmer, during the winter would construct shelters for folks in town. None were quite so elaborate as the more extreme ones shown in the article.

A little later, the county was dotted with the Minuteman missle sites and control centers. One silo was on a friends farm. He said it was sobering to think that the several-foot thick concrete lid would blow the better part of a hundred yards away just before launch.

<comment><USER13> I'd have thought it would be even more sobering to think about what would be arriving 20 to 30 minutes after the missiles blasted off.

NB The novel "Arc Light" includes one thread where the crew from a a Minuteman control center surviving an attack and spending weeks underground then having to make an escape through the devastated landscape:

[http://en.wikipedia.org/wiki/Arc_Light_%28novel%29](http://en.wikipedia.org/wiki/Arc_Light_%28novel%29)

<comment><USER8> Even that nasty a situation is survivable with sufficient strength shelters, luck to avoid a random direct hit, and enough food and water/water supply to wait out the longer period required before evacuating the indeed "devastated landscape".

What I find weird is the attitude that it's better to die than survive with most all of your loved ones and friends, albeit at a reduced quality of life for a long time. A rather ultimate materialistic attitude, or perhaps it's as Dean Ing put it in _Systemic Shock_ :

 _The American public had by turns ignored and ridiculed its cassandras: city planners, ecologists, demographers, socialists, immigrants, who had all warned against our increasing tendency to crowd into our cities. Social stress, failure of essential services, and warfare were only a few of the spectres we had granted only a passing glance. We had always found some solution to our problems, though: often at the last moment. Firmly anchored in most Americans was the tacit certainty that, even to the problem of nuclear war against population centers, there must be a uniquely American solution; we would find it.

The solution was sudden death. A hundred million Americans found it._

<comment><USER14> "What I find weird is the attitude"

Its a manifestation of binary thinking. Well, either we "must" have no nuclear attack, or it "must" be post apocalyptic. Not allowed to be just one shipping container in LA or one maintenance accident at a missile field.

A good analogy is often heard for bicycle helmets. I'll either be squashed underneath the wheels of a bus or I won't get into an accident, either way the helmet doesn't matter so I won't wear one. Even if slipping on gravel and hitting your head on the curb is about 100x more likely than either of the binary options.

<comment><USER3> Well, with bicycle helmets in particular the odds of the particular type of injury they are effective against are very low, while wearing the helmet itself is risky (cars drive closer to people wearing bike helmets), and the helmet costs money, and it looks bad. My judgement is that the helmet's cost outweighs its benefit in most circumstances, just like walking helmets (yes, they exist!).

<comment><USER15> Do you have a citation for "the odds of a particular type of injury [helmets] are effective against are very low"?

Studies seem to suggest that helmets reduce the risk of a head injury by as much as 85%. In 2012 for nearly 2/3 of bicycle accident fatalities the rider was not wearing a helmet. This used to be an even higher percentage but fortunately the # of fatalities has decliend signficantly in the last 20 years.

[http://www.iihs.org/iihs/topics/t/pedestrians-and- bicyclists...](http://www.iihs.org/iihs/topics/t/pedestrians-and- bicyclists/fatalityfacts/bicycles)

<comment><USER3> Check out [http://www.cyclehelmets.org/](http://www.cyclehelmets.org/)

> In 2012 for nearly 2/3 of bicycle accident fatalities the rider was not > wearing a helmet.

That's irrelevant, since it indicates nothing about what fraction of those 2/3 would have survived if they'd been wearing helmets (as an example, if someone is hit my a truck at 50 mph, it really doesn't matter if he's wearing a helmet or not). Also, what fraction of cyclists wear helmets? I'd be surprised were it so much as 1/3; if it's less, then that would indicate that helmet-wearers are more likely to die than non-helmet-wearers (due to risk-tolerance on the part of both cyclists and drivers, c.f. [http://www.cnet.com/news/brain- surgeon-theres-no-point-weari...](http://www.cnet.com/news/brain-surgeon- theres-no-point-wearing-cycle-helmets/)).

There are three types of head strike injury: those with so little force that no death or severe injury would occur, with or without a helmet; those with sufficient force to inflict death or serious injury without a helmet, but which a helmet would prevent; and those with so much force that a lightweight foam-and-plastic helmet wouldn't help.

> This used to be an even higher percentage but fortunately the # of > fatalities has decliend signficantly in the last 20 years.

It looks to me from that IIHS graph that it merely reflects the fact that more people are wearing helmets now than previously.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> basements in Donetsk (Ukraine) today many people use as shelters, for example - [https://www.youtube.com/watch?v=3jY15tgcE7c](https://www.youtube.com/watch?v=3jY15tgcE7c)

A lot of the buildings there are typical USSR-style 5-9 stories concrete block apartment buildings with basements made out of concrete blocks of between 0.3-0.5m thickness.

The modern, high precision weaponry, isn't available in Ukrainian army, instead it is mostly "classic" low precision "area attack" USSR stuff, like artillery guns and multiple unguided rocket launchers. When the city is shelled from a system like this ([http://en.wikipedia.org/wiki/BM-27_Uragan](http://en.wikipedia.org/wiki/BM-27_Uragan)) using cluster munition, such basement shelters work great.

([http://america.aljazeera.com/multimedia/photo- gallery/2014/7...](http://america.aljazeera.com/multimedia/photo- gallery/2014/7/photos-donetsk-enduresshelling.html) \- starting 2nd photo - the buildings itself ok if no direct hit with basements having very good chance to be ok even in case of direct hit into the building)

<comment><USER17> > The modern, high precision weaponry, isn't available in Ukrainian army

Even if it were, it's expensive and it requires point & aim so it's not drop & forget and you're bombing a very small surface.

You're not going to use it against random civilians when shelling a city center, that's a waste of money, time, airspace, efficiency, everything anyone can care about.

<comment><USER16> in some cases when mapping the hits (especially when there is whole salvo), it is sometimes pretty clear that Ukrainians were targeting some specific target, and the civilian buildings would just happen to be either in the flightpath on the downward trajectory or just pretty close (again, it is an USSR tech - <USER2> 1000ft(300m) is pretty close when unguided missile is launched from 15-20km away) and we're talking about pretty populated city.

<comment><USER17> Misses I can believe no problem, but then precision ammunition would probably have turned it into a hit, not into a civilian building eating a bunker buster.

</comment></comment></comment></comment><comment><USER18> The balance of forces was such that it was very conceivable that most of the US could remain unscathed in a nuclear war with the Soviet Union in the 1950s. The strategic disparity was simply too huge. Therefore, shelters made tons of sense.

</comment><comment><USER19> In image 4 and 6 the woman is wearing heels. In image 6 the man is wearing a white shirt and tie. Did they just arrive from work, or are they simply intent on keeping up appearances while in the middle of a nuclear war?

In image 7 some of the glass jars are so precariously placed that they are actually hanging over the edge of the shelf.

Overall, I agree with someone else's assessment here. They said:

       Somebody spent five minutes googling
       "bomb shelter" and then posted a few
       photos with some asinine remarks.
       https://news.ycombinator.com/item?id=8811872

</comment><comment><USER6> This seems appropriate for bomb shelters and Christmas -- Weird Al's "Christmas at Ground Zero:" [http://youtu.be/t039p6xqutU](http://youtu.be/t039p6xqutU)

</comment><comment><USER20> Even if shelters had absolutely no protection from blast they would anyway did most of the job they were intended to do: protect from fallout. Just a few days of shelter time will cut number of casualties in half.

Also today's nuclear weapons are in general, weaker than those of 1950s, especially late 1950s. Weirdly, this means that shelters give somewhat LESS protection because fallout is less of a factor than blast and prompt radiation (you probably can't get in a shelter in time to avoid prompt radiation).

</comment><comment><USER21> Where are these shelters now? Were people really building a lot of these? It seems they could be all over the country in mid-century communities everywhere. I look at a lot of real estate listings but I can't say I've ever seen one mentioned.

</comment><comment><USER22> I saw one of those manual-hand-cranked blower's at a flea market , it was old and I couldn't figure out what the hell it was used for now I know it came out of a 50's bomb shelter, the guy selling it didn't know what it was either.

</comment><comment><USER23> Relevant short story by Philip K. Dick, written in 1955: _Foster, You 're Dead!_

[http://www.american- buddha.com/dick.phildickreader.17.htm](http://www.american- buddha.com/dick.phildickreader.17.htm)

<comment><USER8> Maybe.

Hmmm, in general, very much not a "Maker" attitude. Think about how much fun it would be to recast it into that sort of upbeat "survival in style" ethos. Low power micro-controllers and ARM devices would be even more popular! One form of bragging rights would go to the teens who fashion the lowest power dissipation 1080p shelter media system ^_^.

I see from Dick's Wikipedia personal life section that he is almost certainly a city boy through and through, which would explain a lot of this story.

As for particulars, ignoring the arms-race part of it, really good basic fallout shelters are _ridiculously_ low tech, the sort of thing people in flyover country are supremely competent to make without any fuss---and most of us don't get caught up in the next year model craziness (E.g. my parents bought a very high end Mercedes in ~1990 ... and sold it this year; they are, of course, members of the Silent Generation).

If you need blast protection, then the doors and vents require more serious stuff that's best produced in a factory to standard designs.

As for the new sophisticated threat from the Soviets in that story, no, I think not. Very much not in their style.

<comment><USER23> > _very much not a "Maker" attitude_

At the bottom of the page is a link labeled "Go to Next Page", which leads to _Pay for the Printer_ , written by Philip K. Dick the year after, 1956. Read that and see if you change your mind about PKD.

</comment></comment></comment><comment><USER24> Congress built themselves a massive, posh nuclear shelter with taxpayer money but when it was revealed by newspapers in the 1990s they closed it.

[http://en.wikipedia.org/wiki/Project_Greek_Island](http://en.wikipedia.org/wiki/Project_Greek_Island)

Wanna bet they built another secret one with taxpayer money?

Basically if there is nuclear war, congress should die first because it was their job to prevent that from happening in the first place. We certainly should not want them to survive their failures.

<comment><USER8> Eh, it was obsolete by sometime in the 60s, e.g. when the SS-9 got accurate enough to take it out. In the '50s and '60s we _had_ a real Civil Defense program, I still have my mother's Civil Defense Block Mother sign from the end of that, when I was in 2nd grade. It was only later that CD -> FEMA and "crisis relocation" replaced sheltering, since it was cheaper. During the same decade we gave up on missile defenses....</comment></comment>


The Optimistic Promise of Artificial Intelligence - <AUTHOR> https://www.wsj.com/articles/the-optimistic-promise-of-artificial-intelligence-1497374822 <comment><AUTHOR> Most significant takeaways [italics mine]:

"...almost anything a typical human can do with less than _one second of mental thought_ , we can probably now or in the near future automate using AI... Take a security guard looking at a video feed and saying, "Are there people in this? Are they doing something suspicious?" That task is actually a lot of _one-second judgment thoughts strung together_ , so I think a lot of it can be automated."

"...tell your children: "Learn to be lifelong learners."</comment>


Reddit's co-founder says putting work above all else could be 'toxic - <AUTHOR> <comment><USER2> This story was submitted incorrectly and doesn't have a link. Flagged.</comment>