Devops Horror Stories - <AUTHOR> https://www.statuspage.io/devops-horror-stories <comment><USER2> My Devops horror stories, one sentence each:

\- Somebody deployed new features on a Friday at 5pm.

\- Fifteen hundred machines running mod_perl.

\- Supporting Oracle - TWICE.

\- It turns out your entire infrastructure is dependent on a single 8U Sun Solaris machine from 15 years ago, and nobody knows where it is.

\- Troubleshooting a bug in a site, view source.... and see SQL in the JS.

<comment><USER3> I really hate the idea that deploying on a Friday afternoon is a bad idea. It's only bad when you have shit developers or shit processes that don't catch broken code.

Personally, I think it's better to release at 5pm on a Friday. Once people stay late a few times to fix their broken shit they'll be smarter about not checking in crap.

<comment><USER2> > It's only bad when you have shit developers or shit processes that don't > catch broken code.

Or when the bug is only triggered in specific user profiles.

Or when all the devs went on a retreat in the mountains with no cell service.

Or when a dev makes a mistake (which we know _never_ happens to even the best devs)

Or when the only developer that knows which one of the 1000 changes that were pushed could be the one breaking, turned his phone off.

Or when a flaw is discovered in the process for the first time (which we know _never_ happens because everyone's process is perfect, until it isn't)

Or how change management's requirement that the fix be tested and verified by all affected teams might have people staying a few hours after 5pm on a Friday when they just want to get their weekend started.

Or how 10 different people from 10 different teams _might_ need to be called and kept to work until 2am because the change can't be pulled because the database was already modified and the old client data is already expired from cache and a refresh would destroy the frontend servers.

Or another reason.

<comment> ryan-allen Yes, this! "Good code" and a CI box and deployment automation and some chef recipies don't spell ultimate success.

It drives me nuts when people tell me off for saying 'yeah yeah, no, automating our entire infrastructure of 5 servers isn't really worth it right now', like I'm some unprofessional bozo.

I pretty much have experience with all but one or two of your suggested scenarios, and by now I have no patience for annoying software developers who think that using chef or puppet somehow sufficiently embiggens them to run ops on their own (of course dev ops is almost a political assault on existsing ops guys, not merely a nice new solution to existing problems).

Sigh. This is why I don't work on teams these days (if I can help it).

EDIT: Though I also agree with the sub-parent, that deploying on 5pm is fine in certain teams and certain projects, the most important thing is are the guys pushing to do the deploy going to own the deployment? Are they going to hang around for another 60 minutes to check everything is OK? Are they going to be available at 10pm or on Saturday if something goes wrong and are they going to own it? If the answer is no, then nope, don't do it.

</comment></comment></comment></comment><comment><USER4> Temporarily mounted an NFS volume to a folder under /tmp.

Forgot about tmpwatch, a default entry in the RHEL cron table to clear out old temp files.

4AM the next morning, recursive deletion on anything wiuth a change time older than n days.

<comment><USER5> /mnt and /media exist for reasons. And root_squash and ...

Why no, I've _NEVER_ accidentally deleted whole file systems, I have completely earned superiority here.

Delete /proc and /dev on a running server. Thankfully not really disastrous but damn if people don't notice right away.

Thanks for the tmpwatch info btw.

<comment><USER4> Would have used /media but was thinking, if, say I forgot to unmount it or someone looked at a disk free or whatever, that it would be obvious that it was there temporarily.

Obviously that was incorrect, but the reasoning was, I think, sound.

<comment><USER6> I like /mnt/scratch/ for that. If I was using systems with lots of others, I'd make it clearer with /mnt/tmp/.

</comment></comment></comment></comment><comment><USER7> Tape Archive System: write a tape, restore it again, and do MD5sum against the original data. Then we know it can be restored correctly, and the original data is deleted.

Should be bullet proof?

Alas, the 'write to tape' scripts I'd inherited didn't warn if they couldn't load a tape into the drive.

There was a tape jammed in the drive, so the tape robot was refusing to load any new tapes, but kept on writing and restoring from the same tape over and over again.

Stupidly, we didn't do any 'check a tape from 3 weeks ago' for a while.

Lost quite a bit of data. We still have the md5sums though... _Still gets shivers thinking about it_

<comment><USER8> I know one large company where contract operators managed to destroy every copy of a very large companys payroll by loading tape after tape onto a malfunctioning tape deck.

<comment><USER7> Yes, this was why I was paranoid about the whole write/restore/compare process.

I, being mainly a software guy, didn't consider the hardware robot as something that might fail w/o error.

Now, the process looks like:

Check drive is empty. Load Tape. Write Tape Unload. Load tape "42" from anther slot. Write 'slartibartfast' to that tape. Unload. Load original tape. Restore & Compare. Unload. Load tape "42". Restore, and make sure all it has is 'slartibartfast'.

This seems to me to have removed most of the possible silent-failure situations. If anyone can think of part of this algorithm that might fail, let me know!

</comment></comment></comment><comment><USER9> We launched our brand new service into production pointing the backend at our dev instance, at the office. The entire internet showed up at our wee little DSL connection, effectively DDOS-ing our office. We had to leave, go to a cafe with public wifi to fix it.

</comment><comment><USER10> My worst horror story was a full server room shutdown. We killed servers, then the chillers, and then started work. About an hour after we started, we pulled our first floor tile to move some power cables. There was water under the floor! We spent the next few hours cleaning up all the water.

Apparently water kept flowing into the humidifier tray of the chiller, and the mechanical auto-shutoff never triggered. The pump didn't remove water from the tray because the power was off.

Facilities "fixed" the humidifier, but it still happened again when that circuit was cut off for work elsewhere in the building. No one caught the water overflow, and it flowed out down the conduits to the first floor. So we had flooding on 2 different floors from a single chiller.

</comment><comment><USER11> "you can't have more than 64,000 objects in a folder in S3 - even though S3 doesn't have folders." Is this for real, or are these stories made up? All documentation I've read about S3 suggests that it does not have any file count limitations. The timeline of Togetherville suggests that this story took place between 2008 and 2010. Did S3 have a limit back then that they lifted?

<comment><USER12> There has never been an S3 limit. Some of our customers have millions of objects in a single bucket.

When you do something like this you need to make sure that you have a good distribution of keys across the name space, and you need to think twice before you decide to write code to list the entire bucket. In most use cases at this scale, metadata and indexing are handled by something other than S3.

<comment><USER13> Guiltily admit to listing an entire bucket with 3,500,000 keys, nightly.

</comment></comment></comment><comment><USER14> The customer.io story seems like a great example of why NOT to use budget providers like OVH and Hetzner for mission-critical applications.

You get what you pay for.

<comment><USER15> Not so much an example of why not to use budget providers, more an example of why you should build highly available infrastructure. I don't believe there is any provider, "budget" or not, that guarantees a servers reliability 100% of the time.

<comment><USER14> I was alluding more to the customer support aspect of it. If a technician spends one hour troubleshooting your network problems, then they've already lost their profit for the month.

<comment><USER16> This is one thing where I find AWS shines. I'm on the lowest level of paid support, and I've had nothing but excellent service from good technicians who will try to actually replicate your problem, then contact other teams if they fail or there's follow up. Out of a dozen or so tickets, I've only had one where the response wasn't genuinely useful, and that was for an issue that may have been due to internet weather anyway.

Support is one of things that you can get along without... until you need it. Then you really, really wish you had it.

</comment></comment></comment></comment><comment><USER17> You can hardly be surprised when OVH or Hetzner go down, just consider the price. Putting every server in one location is just stupid... as always the best way to fight downtimes is to spread servers across multiple providers & DC's.

<comment><USER18> Reminds me of a downtime report with my previous shell provider. They lost all Internet connection because someone had broken into a junction under a nearby freeway and cut all the fiberoptics and cables -- while preparing a robbery (presumably trying to cut alarm and/or off-site connections to cctv).

Turned out the two "redundant" providers of fiber both had fiber going through that junction...

</comment></comment><comment><USER19> One thing I've learned: the real value of replication is how easy it makes it to handle strange events without getting stressed out.

It's 3AM. You're being paged with a high latency alert in one datacenter. You run one command to drain traffic out of that datacenter. The latency graph starts looking normal again. You go back to bed at 3:05. You look at the logs and figure out what went wrong tomorrow morning.

</comment><comment><USER20> TODO: Monday Morning: T1 install will be complete. Tuesday: Test/bootup period. Wednesday: Sales start Thursday: Sales continue, TV ad goes live Friday: Champagne!

Reality: Monday Morning: T1 did not get installed. Tuesday: Emergency ISDN solution (stolen from Chiropractors next door) Wednesday: Modem rack catches fire Thursday: TV ad goes Live Friday: T1 goes live. Champagne.

<comment><USER21> this is why given a choice between theory/plans/estimates/schedules or, say... reality and iterating and observing what-actually-happens ... I always prefer the latter. in software engineering, in human relationships, and in the physical world around me in general.

<comment><USER22> Well, sure, as logical people we know that you can't predict failures and that it's always better to play it by ear. Unknown unknowns and all that.

I have worked at several places where salespeople have sold a feature without even asking if it was POSSIBLE, much less created/deployed/tested. "We just sold [Feature X], we told them it'd be ready by [date pulled out of thin air]."

</comment></comment></comment><comment><USER23> Are there any open source load balancing solutions like what Amazon ELB does? Say, install the load balancer on to one or two Amazon VPS, proxy traffic to third party VPS/dedicated servers, Linode, OVH, etc. Wonder how feasible this approach is?

<comment><USER24> It's not about the load balancing software itself, say HAProxy or Nginx, but with ELB AWS autoscales and handles failover between availability zones. You could certainly handle spinning up your own LB instances, managing DNS/Elastic IPs to handle failovers, etc. It would be far more expensive in setup time, management time and EC2 bill, than ELB which is practically free, starting at less than $20/month.

</comment></comment><comment><USER25> At least Amazon doesn't lose your servers.

[http://www.informationweek.com/server-54-where-are- you/65055...](http://www.informationweek.com/server-54-where-are-you/6505527)

<comment><USER6> Eh, this tale of ultimate unattended service reminds me of my favorite Daniel Boone the frontiersman quote: " _I can 't say as ever I was lost, but I was bewildered once for three days._"

</comment></comment><comment><USER6> In an early phase of MIT's EECS transition from Multics (going away, Honeywell sucks) to UNIX(TM) on MicroVAX IIs, i.e. some users, but not as many as latter.

# kill % 1

Instead of %1. So I zapped the initializer, parent of everything else, logging everyone out without warning.

I had more than enough capital to avoid anything more than the deserved ribbing, but it was my Crowning Moment of Awesome devop lossage; harsh but minor screwups in the decade previous had trained me to be very careful.

I've avoided being handed the horrors of many other posters by primarily being a programmer. You full timers earn my respect.

ADDED: Ah, one big consequential goof, related to my not being a full time sysadmin but knowing more than anyone else in my startup. Buying a Cheswick and Bellovin style Gauntlet Firewall from TIS ... not realizing they'd just been bought by Network Associates, who promptly fired anyone who knew anything about supporting that product.... (At that time I didn't even know about iptable's predecessor, although given it was a Microsoft shop....)

I was fired from that job in part because I was the least worst sysadmin in the company, totally consumed with a big programming and database migration effort (Microsoft Jet -> DB2 -> DB2 on a real server), and gave opinions that others sometimes accepted and implemented without due diligence. E.g. I said "this is a competent ISP", not "you should also use their brand new email system" (which I didn't even know existed) ... visibility all the way up to the CEO is of course not always good....

</comment><comment><USER26> A few devops horror stories:

\- Someone on the hardware team deleted several VMs that were being used as build machines, there were no backups. That wasted around 2 days to get things back to normal.

\- During a show I volunteer for: a scissor lift drove over a just run (several hundred foot) ethernet line and severed it, they had to run a new line.

\- PCs running windows being set up, as point-of-sale systems, to run with static IPs on the internet, without a firewall running. Disavowed all responsibility and left them on their own for that. They would have run them unpatched too without intervention.

\- Someone checked a private key into the repository. Plan of action: obliterate from all branches everywhere, delete from all build drops (which contain source listings too), track down all build drop backups on tape and restore-delete-then-recreate them. Luckily I handed that job off to someone else.

</comment><comment><USER27> Coworker says "I'm going to do some clean-up on the server." Two minutes later, "Oh crap." He had wiped out /var/lib. And tell you what, the server kept working. We didn't dare rebooting it though.

Another fun one was coming in one morning, and cleaning up after somebody used some foul PHP provisioning scripts on a customer system and had the unfortunate idea to use a function called "archive". Turned out the function didn't so much "archive" as "delete". Henceforth deletion, especially unintended deletion, was known as "shotgun archival".

</comment><comment><USER28> alt.sysadmin.recovery lives on! albeit in a web app. wonder if usenet is still alive...

<comment><USER29> There are still some good NNTP clients out there, but I think Google Groups is the primary interface these days:

[https://groups.google.com/forum/#!forum/alt.sysadmin.recover...](https://groups.google.com/forum/#!forum/alt.sysadmin.recovery)

</comment></comment><comment><USER30> In a script: sudo chmod -R apache:apache . /

Note that space? I didn't.

</comment><comment><USER31> The first two stories are notable in how they reflect the terrible practices of the teller.

"Our distributed application produces the same type of error after the same period of time in totally different data centers. We have no idea why, but moving data centers seems to help, so we just keep doing it. #YOLO"

"We've built a product on a data store and library we don't understand even the highest-level constraints of. That ignorance bit us in the ass at peak load. We patched over the problem and continue gleefully into the future. #YOLO"

These stories should be embarrassing, but they're seemingly being celebrated, or at least laughed about. Am I off base?

<comment><USER32> your first characterization seems incorrect (did you read the story? it wasn't application errors), and your second characterization is hyperbolic at best. calling it a high-level constraint doesn't mean it's common, nor obvious.

calling them "terrible practices" is redundant, all devops horror stories can be characterized as exposing terrible practices if you're simply looking at the post-hoc view. it's a feature, not a bug, to make light of them. they're laughed about, but with the intent that they're not made again.

</comment></comment><comment><USER33> "Oh, just use keys * to work out what's there."

"No, wait, don't…!"

<site down>

</comment><comment><USER34> Mine was simple: I did a middle-mouse-button paste of "init 6" into a root window of our main Solaris server that hosted about 100 users, mid-day. Boss shrugged it off, stuff happens.

But that's because it was properly configured so a reboot was smooth and didn't have any snags or affect other systems once back online. At another data center across the hall, if their main server needed to be rebooted (not accidentally!), it was 3 days of troubleshooting to get it back up. I learned that after the boss hired one of their admins - not surprisingly, a big mistake.

</comment><comment><USER35> (worst) update table set column = 'blah' WITHOUT a where clause (thank god for backups)

(2nd worst) delete from table where created < 'old_date' WITHOUT an account (thank god again for backups)

Lesson learned, always backup and write the WHERE clause first

<comment><USER36> It is possible to tell psql to always issue an implicit BEGIN so you also have to COMMIT before your change becomes permanent.

This has saved me from paying the price for that particular class of mistake on a number of occasions.

</comment></comment><comment><USER37> Let my cofounder near the backups. Whoops.

Had a friend who recently took down his nic over ssh; he claimed he managed to get back in using some sort of serial over lan magic but I suspect he really just got someone on the other end to help.

</comment><comment><USER38> Any dibs as to what happened for customer.io at Linode/Hetzner?

</comment><comment><USER39> service network stop

<comment><USER26> ifdown eth0

</comment></comment><comment><USER40> So they hosted at Hetzner and OVH, both extremely cheap hosters, and were surprised that things did not go smooth?

Extremely professional.

</comment><comment><USER41> Academic devops horror story in one word: Ruby.</comment>


Microsoft Azure announces G-series VMs - <AUTHOR> http://azure.microsoft.com/blog/2014/10/20/azures-getting-bigger-faster-and-more-open/

<comment><USER2> Nice work on catching up! Softlayer had this for sometime now.</comment>


10 crowdfunding campaigns you should prioritise over potato salad - <AUTHOR> http://www.startupsmart.com.au/financing-a-business/10-crowdfunding-campaigns-you-should-prioritise-over-potato-salad/2014070812696.html

<comment><USER2> There is an interesting article on lesswrong about how to allocate your resources between things that make you feel good and things that actually do good.

[http://lesswrong.com/lw/6z/purchase_fuzzies_and_utilons_sepa...](http://lesswrong.com/lw/6z/purchase_fuzzies_and_utilons_separately/)

The money that I gave to potato salad guy came from a very different budget than the money that I gave to, e.g., PATH

[http://www.path.org](http://www.path.org)

If you're trading off dollars between potato salad and charity, you're arguably doing it wrong. But most likely you are not diverting chariable dollars to potato salad, which is why the premise of this article is flawed.

Or as Max Temkin put it: "Crowdfunding/capitalism isn't zero sum. We can feed the hungry and back the potato salad project."

[https://twitter.com/MaxTemkin/status/486319873207717888](https://twitter.com/MaxTemkin/status/486319873207717888)</comment>


Steve Blank's new book - The Startup Owner's Manual - <AUTHOR> http://www.stevenblank.com/startup_index_qty.html

<comment><USER2> There should be a Kindle/eBook version of this. One of the reasons why I haven't read "The Four Steps to the Epiphany" is because it's only available in paperback.

<comment><USER3> If you look at the comments on his post announcing the book, the majority are asking for a kindle/ebook version. He implies that it was easier to get going with a printed version, but I just don't understand how that's possible. I would love to hear why.

<comment><USER4> Steve Blank always justified it as saying "you want it to be on your shelf, with notes..." I think it's just a luddite reaction on Steve's part.

Apparently he hasn't done enough customer development.

<comment><USER5> Doubly ironic since I love to have reference books with me in Kindle format on my iPad - searchable, scannable (highlighted sections can be compressed together, and always with me). My paper books languish at home. With multiple offices and an apartment I don't, and I suspect this is relatively common, have "a" bookshelf.

</comment></comment></comment></comment><comment><USER6> Essentially a duplicate from just two days ago: <https://news.ycombinator.com/item?id=3577624>

</comment><comment><AUTHOR> Here's his blog post about the book [http://steveblank.com/2012/02/09/two- giant-steps-forward-for...](http://steveblank.com/2012/02/09/two-giant-steps- forward-for-entrepreneurs/)

</comment><comment><USER7> I can't wait to get my hands on this. Steve's previous book _The Four Steps to the Epiphany_ was amazing. I would put it squarely in the top 2-3 most influential books I've ever read.

I don't know that you can ever reduce "starting a startup" to a "paint by numbers" recipe, but Steve has done an amazing job of creating a rigorous methodology for going from idea -> business.

<comment><USER8> Thanks for the recommendation. I've began reading the first 33 pages by the Stanford link above, and it's very difficult to put down.

The issue of customer discovery and validation has always felt like a "large amorphous blob of negative feelings and gut unease" that I would simply ignore. This book faces it squarely and rigorously. I find that exhilarating, and I feel already more confident in tackling this issue while creating things for other people.

</comment></comment><comment><USER9> 25% off and (supposedly) free shipping everywhere in the world: [http://www.bookdepository.com/Startup-Owners-Manual-Steve- Bl...](http://www.bookdepository.com/Startup-Owners-Manual-Steve- Blank/9780984999309)

<comment><USER10> B&N has it almost 50% off.

[http://www.barnesandnoble.com/w/the-startup-owners-manual- st...](http://www.barnesandnoble.com/w/the-startup-owners-manual-steve- blank/1108343438?ean=9780984999309)

<comment><USER9> doh. I should've looked a little longer before ordering it from there on Steve's say-so.

</comment></comment></comment><comment><USER11> personally i have decided to wait until this is published in March 1st, then read reviews (if people can read through 608 pages that fast), before deciding to order this. the thought of reading 608 pages in this day of 140 character tweets makes me shudder. why couldn't this be turned into a pay per view movie? that would be faster to learn it all. with a companion login website to take notes & track your startup progress, like lean lauch pad is... just my incorrect personal opinion

</comment><comment><USER12> Lol $39.99

</comment><comment><USER13> What have you built?

<comment><USER14> <http://en.wikipedia.org/wiki/Steve_Blank>

<comment><USER15> That whole introduction needs to be scrapped and rewritten for NPOV (also for shitty writing). And I'm not sure the guy is notable enough for his own page.

That being said, his 'about' page is more useful than wikipedia in this case:

<http://steveblank.com/about/></comment></comment></comment>


I wrote the Anarchist Cookbook in 1969. Now I see its premise as flawed - <AUTHOR> http://www.theguardian.com/commentisfree/2013/dec/19/anarchist-cookbook-author-william-powell-out-of-print <comment><USER2> I was deeply touched while reading this piece - I felt that Powell was speaking almost directly to me. I was one of those children who suffered from a learning challenge. I had a very strong case of ADHD, and I'll tell you what I stuck out like a sore thumb. I was ostracized by both teachers and peers. In my isolation I found a sense of control in software development.

During my adolescence I found the anarchist cookbook. I used to read through it, fascinated with its ideology, and sometimes building various explosives and weapons. In high school I experimented with drugs and got very close to flunking out. I got in trouble with the law more times than I'd like to admit. I was very angry and paranoid, and the idiotic policies put in place during the Bush years had a lot to do with that.

Nowadays I'm a completely different person. I'm deeply religious, I try and go out of my way to say kind words and help others, and I'm a much happier person. I'm about to graduate with a degree in biochemistry from a great college. I believe the world would be a much better place if people would try harder to understand and love each other and show mercy. Reading this piece by one of my childhood mentors very much evoked a feeling of vindication.

<comment><USER3> > I had a very strong case of ADHD, and I'll tell you what I stuck out like a > sore thumb.

Yup

> During my adolescence I found the anarchist cookbook. I used to read through > it, fascinated with its ideology, and sometimes building various explosives > and weapons.

Yup

> I got in trouble with the law more times than I'd like to admit.

Yup

> I was very angry and paranoid

Still fighting the anger I have from no where. I am not paranoid though the Snowden stuff is making me somewhat.

> I try and go out of my way to say kind words and help others, and I'm a much > happier person.

These are things I try to do otherwise I find myself acting ill towards people.

I am not sure which hit me more the article or your comment. Glad to see there are others who acted the way I did in my younger days and are striving to become a better person everyday. This past May I finally graduated from a decent regional college with a degree in Information Systems and now have an awesome job doing what I love.

<comment><USER2> I am so happy to have read your comment. The Snowden stuff hit me pretty hard as well, and I actually organized my town's rally for the Restore the Fourth demonstrations. If you ever want to talk and get to know each other, my email is in my profile, please feel free to reach out :)

</comment></comment></comment><comment><USER4> > The anger that motivated the writing of the Cookbook blinded me to the > illogical notion that violence can be used to prevent violence.

I don't see why he asserts this as if it is indisputable fact. This is one of the most debated topics in all of human history. The leaders of almost every major revolution eventually reached precisely the opposite conclusion.

The book may in fact do harm in the hands of those who are rash and quick to anger. But I don't think that has anything to do with something that is inherently illogical.

<comment><USER5> Quite. Bloodlust is a dangerous thing, but total pacifism is still a fairly radical idea that I don't think many people _actually_ buy into when the rubber meets the road. Pacifism has claimed few victories, suggesting that historic incidents could have been better solved with pacifism is a good way to make yourself look very silly very quickly.

Pacifism is a tool, not a silver bullet. For revolutionaries and reformationists, there is no silver bullet.

<comment><USER6> Just to pile on here, you'll usually find that people who claim to be pacifists will say, "oh, well, yeah, of course we had to fight Hitler" when confronted. They don't actually even think of themselves that way, in other words.

<comment><USER7> That's not necessarily hypocritical. You can be a pacifist in most situations and acknowledge that sometimes violence is called for.

<comment><USER8> But then you're not a pacifist. By definition (at least my understanding / interpretation), a pacifist considers violence unjustifiable in any and all situations.

<comment><USER9> Like most things, it's a spectrum. There are pacifists who won't partake in violence. Either by their action or the action of someone on their behalf (i.e., they'd argue against their nation going to war). There are also pacifists who won't partake in violence in their own defense, but might to defend others. And others will resist throwing the first punch, but are willing to defend themselves and others with violence if forced to. And there are plenty of other variations on this theme.

And if you really want to go to the definition, wiktionary [1] has it as:

        1.One who loves, supports, or favours peace;
          one who is pro-peace.
        2.One who avoids violence.
        3.One who opposes violence and is anti-war.

Opposition to and avoidance of violence. Not necessarily an absolute rejection of it (though that is likely the ideal of all pacifists).

[1] [http://en.wiktionary.org/wiki/pacifist](http://en.wiktionary.org/wiki/pacifist)

<comment><USER10> No! Pacifism has no spectrum, like anarchism. You can't have a little bit of government with anarchism any more than you can have a bit of violence with pacifism.

<comment><USER9> Read up on pacifism and its history and various philosophies. I use spectrum to indicate that there are a variety of pacifist philosophies and I'm not wrong, because there are a variety of pacifist philosophies. They all desire a world with _zero_ violence. But some are willing to defend themselves or others against violence with violence, while others are not. Some won't even allow you to defend them, they'd rather fall on the sword than let another perform a violent act for their sake. To think that there's a singular philosophy on this issue is a totally ignorant position to take.

<comment><USER10> How can you claim pacifism if you resort to violence? Pacifism is the rejection of violence, full stop. The ambiguity lies in the definition of "violence." Tolstoy called voting, paying taxes, and engaging in commerce forms of violence, for instance. Regardless, only using physical violence in defense doesn't make you a pacifist, it just means you don't attack people.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment> vezzy-fnord It's worth noting that the author recanting his book isn't as of recent. In fact, he dropped his views all the way back in 1976.

This editorial review on the Amazon page for the book might offer some more insight: [http://www.amazon.com/dp/0974458902](http://www.amazon.com/dp/0974458902)

Oh, and might I add, _The Anarchist Cookbook_ isn't flawed so much in that it presents violence as a means to an end, but rather that actually following the recipes will likely get you killed or maimed. Hopefully people reading the book already knew this.

<comment><USER11> I'm not familiar with the book; are the recipes simply handling dangerous chemicals? Are the methods shoddy? Or are they written _intending_ to harm the preparer as a form of sabotage?

<comment><USER5> Most of them are just overly optimistic. Overestimating the size/power/destructive power of things that ultimately would be little more effective as weapons than firecrackers (though like firecrackers, more than capable of maiming the handler...)

If you wanted to make such a book properly, you would be best off describing things that you did not invent (or worse, dream up but never tried) but instead things that have actually been used. For example, describe how the Finns used storm matches to make Molotov cocktails (which they used to great effect against Soviet tanks, though modern tanks will not be vulnerable to those sort of attacks. That should also be mentioned prominently in this hypothetical _" Cookbook-done-right."_)

</comment></comment></comment><comment><USER12> I don't know. I went to a county fair recently where the local city police were proudly displaying their DHS funded heavily armored, turreted and gun ported urban assault vehicle. I was frankly utterly appalled that a bunch of local yokels ordered around by some set of corrupt local politicians would be allowed to run around with military grade hardware when there's a national guard armory not far away that should be much more qualified and better trusted with that same piece of equipment.

So with rampant stupidity and reports of massive swat team abuse cropping up in the US what tools are available to the citizens to counter this? Seems the anarchist cookbook is probably on that list.

A tool is a tool, people really need to be encouraged to use them responsibly and punished if used irresponsibly.

<comment><USER13> Why would one use The Anarchist Cookbook to defend themselves against heavily armed cops? I think a gun would be more effective than a tennis ball filled with match heads.

<comment><USER14> I think everyone needs to take a step back here and start by deciding if violent resistance is the correct way to protest the increasing weaponization of the police before we start trying to solve the problem of which tools to be violent with.

</comment></comment></comment><comment><USER15> So the violence of the ANC didn't end the violence of Apartheid, and the violence of the Allies didn't end the violence of the Axis?

edit: I'd submit that it was the violence of their respective systems upon their non-violent protests that is responsible for anything Gandhi and MLK achieved.

<comment><USER7> > _I 'd submit that it was the violence of their respective systems upon their > non-violent protests that is responsible for anything Gandhi and MLK > achieved._

You say that like it's an argument in favor of violence and against pacifism. In those two cases, the violent side lost specifically because they were violent.

I'm not convinced that violence is never called for, but pacifism clearly _can_ work in at least some circumstances.

<comment><USER15> >You say that like it's an argument in favor of violence and against pacifism.

I don't mean it to be. I mean to say that the strength of pacifism comes from the violent reaction to it.

</comment></comment></comment><comment><USER16> wow, what an interesting personification of the entire baby boomer generation's evolution from anti-establishment to the establishment.

<comment><USER15> Yes. It's the personal statement of a very privileged person who (like the rest of the 60s) was probably inspired to empathy with the civil rights movement triggered by the threat of the draft, but has now transferred his empathy to the more exotic minorities populating his new life of international adventure.

Now his personal feeling of threat comes more from the revolution than the state, so his opinion of revolution has changed.

Hopefully the consolation that no one will be able to download, take out from a library or buy his book without being put on a list by secret intelligence services will make him feel a little better about writing it.

</comment></comment><comment><USER17> "The basic premise behind the Cookbook is profoundly flawed"

I didn't think the cookbook _had_ a premise. I enjoyed it because it was full of fun sounding, curious articles that one would never find elsewhere. It gave a sense of adventure and quiet rebellion to the reader and it could be enjoyed in peaceful solitude. It could also be used as a guide for fun projects with friends ("let's try to make thermite!") or just some exciting group discussion.

</comment><comment><USER18> Newsflash - young man is rash and inflammatory & old man sees error in his ways.

<comment> dj-wonk And the reformed, wise old man wants to erase all traces of his former book? Maybe he could gather them up and burn them?

<comment><USER19> Having read the article, I don't know where you gleaned that. Desiring for new copies not to be printed is not the same as wanting previous copies or "traces" of their existence to be destroyed.

</comment></comment></comment><comment><USER20> I agree with the author. Terrorists rarely if ever achieve their stated goals, and just end up hurting innocents. But he is wrong that violence can't be used to prevent violence.

>The simple fact is that non-violent means do not work against Evil. Gandhi's non-violent resistance against the British occupiers had some effect because Britain was wrong, but not Evil. The same is true of the success of non- violent civil rights resistance against de jure racism. Most people, including those in power, knew that what was being done was wrong. But Evil is an entirely different beast. Gandhi would have gone to the ovens had he attempted non-violent resistance against the Nazis. When one encounters Evil, the only solution is violence, actual or threatened. That's all Evil understands.

-Robert Bruce Thompson

<comment><USER21> Well they managed to scare the whole world pretty good in the last decade. With the help of our governments of course but in the end everyone says, it's their doing.

<comment><USER20> They cause damage of course, but they never achieve their actual goals:

[https://web.archive.org/web/20110727223802/http://lesswrong....](https://web.archive.org/web/20110727223802/http://lesswrong.com/lw/51/terrorism_is_not_about_terror)

[http://www.gwern.net/Terrorism%20is%20not%20about%20Terror](http://www.gwern.net/Terrorism%20is%20not%20about%20Terror)

>"In a [previous study of mine] assessing terrorism's coercive effectiveness, I found that in a sample of 28 well-known terrorist campaigns, the terrorist organizations accomplished their stated policy goals 0 percent of the time by attacking civilians."

>"The seven puzzles...are: 1) terrorist organizations do not achieve their stated political goals by attacking civilians; 2) terrorist organizations never use terrorism as a last resort and seldeom seize opportunities to become productive nonviolent political parties; 3) terrorist organizations reflexively reject compromise proposals offering significant policy concessions by the target government; 4) terrorist organizations have protean political platforms; 5) terrorist organizations generally carry out anonymous attacks, precluding target countries from making policy concessions; 6) terrorist organizations with identical political platforms routinely attack each other more than their mutally professed enemy; and 7) terrorist organizations resist disbanding when they consistently fail to achieve their political platforms or when their stated political grievances have been resolved..."

</comment></comment></comment><comment><USER22> As a youth I had a copy of the Improvised Munitions Handbook. Imagine my surprise when it showed up in Toy Story!

[http://en.wikipedia.org/wiki/TM_31-210_Improvised_Munition_H...](http://en.wikipedia.org/wiki/TM_31-210_Improvised_Munition_Handbook)

</comment><comment><USER23> > I suspect that these children have taught me a great deal more than I have > taught them.

Very great statement. I've done volunteer work before and have come to the same realization.

<comment><USER24> This is an insult to the intelligence of the children. You're claiming that they are so poor at learning, and you are so superior at learning, that all of their learning combined is dwarfed by the learning of a single man--you.

Any teacher who learns more than his collective students is a failed teacher. Period. Such a teacher's life has been worthless--this is a mathematical fact.

Being proud of gaining more from your volunteer service than did the people you are supposed to be helping is not something to brag about. It is morally reprehensible and brings to mind all of the criticisms of poverty tourism that have so often been levied against your type of exploiter.

Usually, the money that wealthy poverty tourists spend on travel would be thousands of times more efficient had they just donated that money.

You are gleeful that you profited more from your volunteer work than the supposed recipients did. Your self-reported tale of exploitation is disgusting.

<comment><USER25> You're taking a little kind of cliche platitude and running in a weird direction with it ... really weird ...

<comment><USER24> Oh no! Someone on the internet says my highly upvoted analysis is.... "weird"!

What a worthless comment.

<comment><USER26> You seem to be trying very hard to start off with getting off on the wrong footing on HN given how confrontational you are being. Your "highly upvoted analysis" can't possibly be that highly upvoted given the low karma of your account.

</comment></comment></comment></comment></comment><comment><USER27> Dang right it was flawed! Especially the part about being able to smoke banana peels. No buzz and this (then) 16 year old had a sore throat for a week!

</comment><comment><USER28> I doubt the publication makes one iota difference to the individual. The person you are worried about is already in that state of mind and there are now an infinite number of other resources (the Internet) for them to draw upon. If you are truly concerned, write a new book entitled something like "If You Want to Read the Anarchist Cookbook, You Should Read This Book Instead!"

</comment><comment><USER29> "The Cookbook serves no purpose other than a commercial one for the publisher" \-- and that's precisely why it won't go out of print. Regardless of the intentions of the writer and the reader, it is a very popular book and therefore something lucrative to sell. If it were to disappear from print, ironically, it would make it even more valuable.

</comment><comment><USER30> I was an anarchist in high school too. I was an over-achieving individualist who took personal affront to the norms and rules imposed on all school kids, and people in general. I researched anarchy philosophy and policy as it presented itself over the years. It worked for me: I felt that people could be trusted to behave correctly without the imposition of external rules and laws and the violent enforcement thereof. Obviously I was wrong. But I remain basically committed to my original ideals: an ethically motivated individual does not need to be ruled by fear and force. What, however, does that same individual do when confronted by the violent - or even not so violent but still evil - application of the rule of law?

<comment><USER31> Anarchocapitalism has some interesting answers to this question. It's not a utopian variant that believes everyone can simply be trusted not to behave badly.

<comment><USER32> Anarchocapitalism, while interesting, has as little to do with historical anarchist thinking as The Anarchist Cookbook does.

</comment></comment></comment><comment><USER33> Off-topic: thank you mods for changing the original submission title, it was quite inflammatory.

Terrorism -- or (more to the point) political expression by violence -- is not at all a means to end violence, and it would be foolish to pretend that it can ever be. As a matter of fact that is the point of the submitted article. The original title claimed that the Cookbook's author felt terrorism is "a worthless strategy" but such a title contrasts so strongly with the actual article that it boggles my mind how the original submitter even came to that conclusion.

Terrorism is just another form of political expression, and calling it worthless is like calling a hammer worthless where a screwdriver would be more appropriate.

<comment><USER5> Terrorism is such an overloaded word that I think it is next to useless if you want to convey any sort of point clearly.

It is better to speak of things more specifically. Topics like assassinations, industrial sabotage, propaganda, and bombing campaigns all deserve individual treatment; talking about "terrorism" has the tendency to lump them together.

Some of the things that frequently fall under the umbrella of "terrorism" _do_ have the potential to end violence. Even things that are violent.

Because of the fuzzy and subjective nature of the word _" terrorism"_ it is always possible to assert that examples of "terrorism" working are not examples of "true" terrorism, but if we are wary of that loophole by being purposely inclusive, there are examples in history of terrorism ending violence. The show trial and execution of Nicolae Ceaușescu is an example that springs to mind immediately. Allied resistance movements during the Second World War are another (such as Operation Heads, an assassination program run by the Polish resistance: [http://en.wikipedia.org/wiki/Operation_Heads](http://en.wikipedia.org/wiki/Operation_Heads)).

</comment></comment><comment><USER34> It isn't about using violence against the government. It is about getting the government to reform itself to stop using violence to solve world and domestic problems.

We don't need violent protests now that we have the Internet and can leak documents exposing government corruption and write about truths to inform people and allow them to vote in different politicians.

<comment><USER24> The most violent thing to do is to do nothing to change the status quo because the status quo is upheld by perpetual institutionalized violence.

Today, however, the violence of the current regime goes beyond just violence against other humans. Today the current regime is committing violence against the human species, and threatening that species with extinction.

Violent acts that may stop the extinction of life on Earth are not only justified... they are morally obligatory.

We are in the last hours of humanity. The species won't be saved by squeamish old boomers.

<comment><USER35> >Today, however, the violence of the current regime goes beyond just violence against other humans. Today the current regime is committing violence against the human species, and threatening that species with extinction [Citation needed].

Who is threatening the human species with extinction. I can think of several ways that humanity could go extinct, but none of them would benefit any government. Be specific.

<comment><USER24> It's called the climate change apocalypse, and every major government is hurtling our species toward it. Hawaii University just released the best data we have so far: sea level rises are inevitable, mass destruction of coasts worldwide, hundreds of millions of refugees, famine, drought, and extinction of huge proportions of higher life forms across the planet.

The only remaining debate is whether we will experience apocalyptic horrors of Biblical degree, whether just humans will go extinct, or whether all life on Earth will go extinct. These are our three possible futures. The Permian mass extinction has oft been cited.

You are morally condemned for your ignorance. You are part of the problem. Shame on you.

<comment> dj-wonk Yes, climate change is a gigantic issue. But the existence of collective action problems does not imply conspiracy or even willful neglect. It is easy to blame one person or one entity, but that is missing the point. See: [https://en.wikipedia.org/wiki/Collective_action#Collective_a...](https://en.wikipedia.org/wiki/Collective_action#Collective_action_as_a_response_to_climate_change)

<comment><USER24> That's a non-sequitur. No one claimed there is a conspiracy or willful neglect.

Why would you even make that argument? What possible help could that be?

It doesn't matter WHY governments are causing the extinction of the human species. All that matters is that THEY ARE. If some institution, system, person, class, group is accidentally causing the extinction of the species, it matters not what their motive is. Any violence toward them that could be successful is morally obligatory. Even killing 90% of the human species is preferable to extinction--destroying all technology for instance is preferable to extinction.

Why are you making dumb arguments that we should do nothing in the face of scientifically proven extinction threat? Do you WANT the species to end?!

<comment><USER25> Eh, I think you are too wound up about it.

<comment><USER24> Yeah you're right. Extinction of our species is not really something anyone should speak passionately about. We should all just be non-chalent. Ya know, like middle schoolers trying desperately to be cool.

You are actually so threatened by what I'm saying that you feel the need to follow me around on HN and add your worthless 2 cents. It's actually rare to see your kind of empty, mindless blabber here on HN. But it does make me laugh.

<comment><USER25> No, I just comment when I see posts I want to respond. I don't always look at the usernames. It just so happens you made two weird, hyperbolic posts on the same thread. :)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER36> Nothing new in this, he's held this opinion for quite some time:

[http://www.salon.com/2000/09/18/anarchy/](http://www.salon.com/2000/09/18/anarchy/)

</comment><comment><USER37> I see another premise. Which is, without knowledge, one has fewer options to resist violence. Now as to the utility of the knowledge provided, who can say?

</comment><comment><USER38> That seemed like a well-intentioned article, but his story strikes me as paradigmatic though:

* Year 1969. Writes _the_ Anarchist Cookbook.

* Years 1973-75. Becomes a teacher.

Interpret at will.

</comment><comment><USER39> Yes:

> The anger that motivated the writing of the Cookbook blinded me to the > illogical notion that violence can be used to prevent violence.

</comment><comment><USER40> Because now you're older. Don't worry, the older, the more conservative.

</comment><comment><USER41> Violence is the physical expression of fear. When you fight something, you give power to it (while giving away your own power). All fearful things disappear - but only after you've stopped fearing them.</comment>


To win at Starcraft, machines need to figure out how to lie - <AUTHOR> http://www.wsj.com/articles/computers-that-crush-humans-at-games-might-have-met-their-match-starcraft-1461344309 <comment><USER2> I think the Starcraft pros tend to think about what separates top pros from mediocre pros. They see these things, like better 'mind games', as what makes Starcraft hard.

The thing is, a game needs to be very carefully balanced for these scissor/paper/rock dynamics to emerge. This type of balance is not the default --- it's the product of careful design, and empirical tuning.

In human vs AI, this balance will almost surely be broken. The AI will have perfect control, even with APM constraints. Its attention can be split between two attacks perfectly. It won't play at all like a human.

So, the resulting human vs AI gameplay is unlikely to be strategically deep. If the AI can learn to play Starcraft at all competently, it will probably crush. It will probably find a dominant strategy that it can re-execute exactly the same every time, which the human can't stop.

Making the AI work well for Starcraft will be very difficult. In Chess and Go the actions are discrete; in Starcraft the actions are all analog. But the gameplay requires much deeper search than the Atari games Deepmind learned in its early paper. So I think the game will be very difficult for the AI. But not for the reasons people are talking about.

<comment><USER3> Being able to have multiple battles at the same time is not terribly interesting, because humans can scale this as well. Team up two humans and they'll be able to do twice the work. Teaming two players to play a Go or a chess game against a computer wouldn't do that much good though.

I've recently read Ender's Game and there was an interesting idea in there. In the book it is hive queens controlling the Buggers fleet with exquisite control, yet Ender is able to exploit this as a weakness, as every human is a sentient being capable of thinking and acting independently, so as a commander he was only giving high-level orders to his lieutenants and they did the rest. Which is how humans have fought wars.

<comment><USER4> If you team up two humans, they need to be very well coordinated to work together effectively, otherwise they'll just get in each other's way.

Starcraft 2 has a mode like this called Archon Mode, where two players control the same units. The conflicts that can arise from this can be pretty fun to witness.

<comment><USER5> A similar mode was present in the original Starcraft, albeit without the cool name. (It was simply called "team melee".)

<comment><USER6> No, it's a completely different thing. In Archon Mode, two physical players share control of a single in-game player. A lot more coordination is required or they'll both be giving counter-orders most of the time.

<comment><USER5> It's been a long time since I played team melee, and I've never played Starcraft 2 multiplayer, but reading back, I'm pretty sure they're the exact same thing:

[http://starcraft.wikia.com/wiki/Team_Melee](http://starcraft.wikia.com/wiki/Team_Melee)

[http://starcraft.wikia.com/wiki/Archon_Mode](http://starcraft.wikia.com/wiki/Archon_Mode)

[https://www.reddit.com/r/starcraft/comments/314e28/archon_mo...](https://www.reddit.com/r/starcraft/comments/314e28/archon_mode_vs_team_melee/)

</comment></comment></comment></comment></comment></comment><comment><USER7> What people miss is that StarCraft is a physical game primarily, with tactical elements, not the other way around. There are a lot of fans who know the game well and can make all the right decisions, but will never be able to execute it in real time.

Execution of strategy is what separates the professional StarCraft players, and in that way it's much closer to professional sports than Go / Chess.

<comment><USER8> What if the pace of gameplay were slowed down 5x? Would execution still be an issue?

<comment><USER7> This is like asking what basketball would be like if you weren't required to dribble. It might be interesting to think about, but would probably result in a different game.

<comment><USER8> _This is like asking what basketball would be like if you weren 't required to dribble._

Not really. The "physical" aspect of Starcraft is an information input/output feat. It's still arguably the same abstract _game_. Arguably, it's no longer a _sport_ however. (And maybe it wouldn't be a "game" in the sense that basketball and football are "games" for people.)

<comment><USER7> Solving THAT game (where we get as close as possible to a brain -> machine interface and it becomes completely strategic) is interesting, but it's not StarCraft. Arguably it's the same abstract game, but when you bypass constraints (applicable only to humans), you probably should have targeted something else without those constraints in the first place. In our case, a turn-based strategy game that is equally complex and with limited information (Civilization?!).

</comment></comment></comment></comment></comment><comment><USER9> Having played Starcraft at a somewhat high level, I don't agree. The very best players will beat an inferior player without surprises, either just by knowing when to be aggressive and when to be defensive, with smart tactical plays, forcing an opponent into a position where they will be at a disadvantage later on, or sheer multi-tasking. What machines lack is sheer intuition.

For example, it is possible to chip away at your opponent's economy by loading units into a dropship and flying it into the back of your enemy's base. To increase your chances of doing damage, you should probably set up multiple of these at the same time, or attack the front while dropping, but you also need to know that a dropship works best when it's fully loaded (8 small units, 4 medium units or 2 large units) - if it's not fully loaded, part of your investment is wasted and your attack will be sub-optimal.

Learning this as an AI isn't going to be easy because there are lots of scenarios when you can do a drop like described above perfectly and it'll still lose you the game - maybe your opponent had some optimally placed anti- air defenses, or they were just getting ready for a huge all-out frontal attack just as you arrived, so their army is vastly superior.

<comment><USER10> Intuition can be simulated with algorithms and massive computing power.

</comment></comment><comment><USER11> It seems like most people in this thread are claiming that high APM and the ability to have perfect control over everything happening on the map will give the AI an advantage that will force a win.

Here is a video of one of the best current StarCraft bots losing to an D-rank (low skill) human player. The bot's APM is ~5500 while the human's is ~200. [https://www.youtube.com/watch?v=ztNYOnx_YQo](https://www.youtube.com/watch?v=ztNYOnx_YQo)

The fact is, no AI has ever beaten even an amateur player in a tournament. Even with great micro, if your play is too predictable then the human will learn it and exploit it.

I, for one, am very excited to see the development of new StarCraft AIs. And especially SC2 AIs so that it can challenge the current world champions.

</comment><comment><USER12> 'Eugene Kim' should be Yoo-Jin Kim aka SoS.

<comment><USER13> > An advanced human player might, for example, feign weakness on one side of > the playing field while mustering a pack of mutalisks—fire-breathing dragon- > like creatures—on the other side of the board.

Fire-breathing? I must have not been paying attention...

<comment><USER14> [http://eu.battle.net/sc2/en/game/unit/mutalisk](http://eu.battle.net/sc2/en/game/unit/mutalisk)

No fire-breathing : "Weapon: Glave Wurm"

<comment><USER13> I know. I was pointing out the... creative... journalism.

</comment></comment></comment></comment><comment><USER15> > But some of the world's biggest nerds are confident that machines will meet > their Waterloo on the pixelated battlefields of the computer strategy game > StarCraft.

Er .... sorry, but Waterloo was a decisive battle that lead to the dismissal of a pan european empire. I don't think we will see the end of the "machines" any time soon. A better comparison would have been Austerlitz (but then it isn't much celebrated in the Anglo-Saxon world).

</comment><comment><USER16> The AI had better be able to interact using a mouse and keyboard. Any time I see the "AI wins!" headline, for anything real time, it was invariably given a superior interface. Watson getting a text file sent to it instead of having to read the screen or listen to Alex read the answer for example. If the machine isn't watching a monitor and forced to use the same input methods as the players, then sure it will "win," but it will be a cheat, not a triumph of AI.

<comment><USER17> You are missing the point of man vs machine competitions. It's unbalanced on purpose. The aim is to see if an AI with all its unique advantages can best a human with their unique advantages. It is not to create a physical robot that has to mimic the disadvantages of a human.

That's not cheating, it's a agreed upon rules. Making an AI that can beat a human is already very hard. Next you'll add other arbitrary requirements, like limiting the wattage to what the human brain uses...

<comment><USER16> I'm not missing the point. If you have to modify the game itself to allow the AI to compete, then it's cheating. It would be like self driving cars that required specialised roads.

<comment><USER18> What you're suggesting is that an AI can't really drive unless it's operating the car the same way a human does, i.e. with a steering wheel and pedals rather than having a direct digital interface into the car.

<comment><USER16> No. I even said what the analogy would be for self driving cars. I'm calling out the BS of the marketing scams for investment dollars that are these contests. Point out that and a bunch of people get butt hurt because they need that investment money.

<comment><USER17> Bullshit, a human only has 2 eyes, but a self driving car can see in all directions at once. You are just making up arbitrary rules.

</comment></comment></comment></comment></comment></comment><comment><USER19> Terrible headline; machines are already great at "lying"; recent progress in poker, for example.

<comment><USER20> We changed the headline to the subtitle last night because the main title is so baity. If that's not good enough, the best way to explain that is to suggest a better title—i.e. an accurate and neutral title that preferably uses representative language from the article. When users do this, we're happy to use their suggestions.</comment></comment>


Iterating over a Javascript array at a consistent rate - <AUTHOR> http://blog.jackmontgomery.io/iterating-over-a-js-array-at-a-consistent-rate/

<comment><USER2> That's cool, but you're kind of mixing together a few different pieces of functionality. To me, this kind of thing would be much more readable using promises to wrap the asynchronous nature of the code. Assuming you used when.js, you could do something like:

      function delay(fn, ms) {
        return when(function (resolve) {
          setTimeout(function () {
            fn();
            resolve();
          }, ms || 0);
        });
      }

      function eachDelay(arr, fn, ms) {
        arr.reduce(function (prev, curr) {
          prev.then(function () {
            return fn(curr);
          });
        }, when(null));
      }

That looks more complicated on the surface, but in reality, libraries like when.js and underscore give you convenience functions that abstract away much of this boilerplate. I didn't want to presume anything but the bare `when`. If you really get used to programming "the combinator way", you get to the point where you write these cute little primative combinators that can be combined in very readable ways. If you like coding in this style, definitely check out Javascript Allongé ([https://leanpub.com/javascript- allonge/read](https://leanpub.com/javascript-allonge/read)), which makes this all look far more appealing than I've done here.

<comment><AUTHOR> That seems like an interesting way of doing it as well. I've used promises before but not enough to be comfortable with this yet. Thanks for the tip!

<comment><USER2> Nevermind, that doesn't work. I didn't even use my `delay` function. I was trying to get too cute. But you might see what I was getting at.

If you want to really understand promises, this is a good resource: [https://gist.github.com/domenic/3889970](https://gist.github.com/domenic/3889970)</comment></comment></comment>


Recruiters can't tell great programmers and flakes apart [comic] - <AUTHOR> http://blog.codeboff.in/2010/07/22/programmers-and-recruiters-are-from-different-planets-3/

<comment><USER2> Considering the way recruiters are paid, I can understand that no matter how good this guy is, he is worthless to the recruiter to recommend him

Recruiters are paid based on salary of the guy _and the fact he stays there for a certain period of time_.

<comment><USER3> Does anyone have concrete data points on how and what recruiters are paid?

The data point I heard for the finance industry was 25% of base + guaranteed bonus (but not signing bonus or discretionary bonus). I have no idea how the rules change if the employee flakes and quits within X months.

<comment><AUTHOR> I know at least one place that doesn't pay anything until an employee has stayed 3 months.

</comment></comment></comment><comment><USER4> The problem is most recruiters have absolutely no idea how to judge a candidate so they come up with their own heuristics. It's much the same way that an accountant might measure lines of code for productivity. They can only go with what makes sense to them.

</comment><comment><USER5> In my experience recuiters are used to staff long term positions. If a hiring company is looking to hire someone for a position that will take many months to learn an existing code base it is reasonable to expect that person stay with the company for some multiple of that amount of time.

That said, recruiters should ask potential candidates about their work preference. If the person is open to a long term placement they shouldn't dismiss them based on previous short term engagements. But don't get upset if "Large Corporation A" isn't interested in you because you like to do 3 month freelance projects.

</comment><comment><USER6> Assuming they could stomach doing it, how much money cold a technical person with good interviewing skills make as a head hunter? Perhaps after a year when they had time to build up a rep as being a good filter?

<comment><USER7> You would do better as an in-house recruiter at a technical company. That way, you're in more direct contact with hiring managers.

Build your rep there, and go out on your own.

My first real job was working for a recruiter who used to be a programmer. He didn't really need to use his programming knowledge, though, because recruiting means sending candidates who won't make the hiring manager look like an idiot if they don't work out. If your choices are a) brilliant dropout, or b) someone who went to Princeton but is really lazy, your incentive is to send b). If a) works out, you get no direct benefit; if b) doesn't work out, well, who could have predicted that?

</comment></comment><comment><USER8> Why should recruiters be better than anyone else at hiring? It should be common knowledge by now that there is no way to differentiate great programmers from flakes other than hiring them. Even then, a great programmer in one environment may become a helpless flake in another. It's a _relation_, not an attribute.

<comment><USER6> Great recruiters would know both ends of the equation, and knowing that, make good calls. I'm actually of the opinion that most technical companies are bad at hiring. The only strategy that can be formalized and generalized is to error on the side of caution, but this can be a problem in the current job market where it's hard to find good people and you can't afford false negatives.

</comment></comment><comment><USER9> And that`s why you should say you have worked on three well defined projects, with tight schedules and scopes, and were able to deliver the client requirements with awesome quality (insert anecdote about when a customer said you did something incredible).

But now you`re looking for a different type of work (whatever the job description says). That means you have to research the job/company/ask questions before answering the recruiter.

You can`t change an industry. Leave the pragmatism aside when trying to get money and play the game.

<comment><USER10> You mean idealism, not pragmatism.

</comment></comment><comment><USER11> If you are talking to recruiters you are doing it wrong 90% of the time.

<comment><USER12> Not my experience.

I was looking for a job out of my country and through a recruiter I was able to move and get a job with double the salary I had in my hometown.

Three months has passed and due to a stupid policy I don't like I'm switching job again. A second recruiter found me a better position in another company, with a higher salary (which is already quite high).

<comment><USER11> Sure, most recruiters can get you good money at usually large companies that may or may not actually be a good fit for you. It kind of speaks for itself that you are leaving after a couple months.

I believe the best way to get a job you will love and get paid well to do it, is to aggressively pursue companies that you want to work for. This is really pretty easy nowadays, you find an email for the company and you write a great cover letter and you send it off with your resume. Doesn't matter if they have any openings or not. You should already know the companies products, how they work, contribute to relevant open source, etc.

</comment></comment></comment><comment><USER13> Start your own company and do your short-term projects through that company. Then you can say you worked for one company over the past 14 months instead of saying you worked for three.</comment>


Copygram offers up a new, more beautiful way to browse Instagram on the Web - <AUTHOR> http://thenextweb.com/insider/2013/08/23/copygrams-redesigned-site-is-a-far-more-beautiful-way-to-access-instagram-on-the-web/?fromcat=all In response to Instagram and their emails to developers using "insta" or "gram"...<p>- Let's launch The Next Copygram, http://next.copygr.am - Focusing on the users content on our favorite device, laptops and desktops. Instagram does a good job on the mobile devices but on the web they quite don't get it.<p>So The Next Copygram makes your socks rock. <comment><USER2> Might want to change the name...

Instagram Cracks Down On Connected Apps Using "Insta" And "Gram" [[http://techcrunch.com/2013/08/19/instagram-cracks-down-on- co...](http://techcrunch.com/2013/08/19/instagram-cracks-down-on-connected- apps-using-insta-and-gram/)]

<comment><USER3> How on Earth can they try and own "gram"?

On it's own it is is a standard unit of measurement, but as a suffix is considered standard in English since time immemorial to reflect anything "written or drawn", it features in the word "telegram", and can be traced as a suffix back to ancient Greece.

I barely understand them trying to protect "insta" considering they are using Kodak's trademark there: [http://en.wikipedia.org/wiki/Instamatic](http://en.wikipedia.org/wiki/Instamatic)

But the suffix "gram"? Their lawyers may have found some powder that came in such measurements.

</comment></comment><comment><USER4> "A CLUTTER FREE ENVIRONMENT FOR YOUR PHOTOS _TOO_ SHINE" indeed...

Grids can get pretty cluttered, and by "the web" they must mean "not mobile" as the site is a mess on my phone.

<comment><AUTHOR> We'll release the responsive part of the site as a featured release due to the fact that Instagram is actually quite good at their mobile presence :-)

</comment></comment><comment><USER5> Pinstagram?!?</comment>