Analyzing the FBI's Explanation of How They Located Silk Road - <AUTHOR> https://www.nikcub.com/posts/analyzing-fbi-explanation-silk-road/ <comment><USER3> Came across a discussion on Twitter pointing out reddit users who apparently discovered IP addresses in error messages printing out in the HTTP response:

26 Mar 2013: [http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the...](http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the_silk_road_revealed_its_public_ip_last/)

26 Mar 2013: [http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_s...](http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_silk_road_users_sr_is_leaking_their/)

03 May 2013: [http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_b...](http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/)

<comment><USER4> Wow, I love how those threads are full of people who insist the OP is full of shit and that Silk Road doesn't even have a public IP address.

Maybe we'd have fewer incidents like these if keyboard jockeys were a bit more teachable. Even super 1337 h4xx0r5 make mistakes, often obvious ones. Software is complicated. Let this be a lesson to any of us who may dismiss a correction too quickly.

<comment><USER5> Yes. They all tend to talk in certainties about something that they are only speculating about:

      > you would have to go out of your way to expose
      > your public IP via Tor.

      > However, the hidden service you connect to,
      > would be the load balancer. This service would
      > have zero knowledge of its "external" ip
      > address, as it would be running in a VM
      > pointing to a port on the loopback device.

(because _obviously_ nothing else is possible)

      > - Server is almost 100% certainly running in a
      > VM and doesn't have a public IP.
      >
      > - The entire harddrive for the server (which
      > includes all the the information such as
      > private messages and addresses) is almost 100%
      > encrypted.
      >
      > This is the motherfucking SilkRoad. They've
      > been operating so successfully for so long
      > because they're not amateurs. OP is a liar.

... etc ...

<comment><USER6> the funny part is: the people i know that used the service (who were far down towards 'non' on the technical spectrum) complained regularly about it's terrible design, stability and security, and were under the impression it was administrated by parties that had little idea what they were doing. they theorized it was a big scheme to go Gox* on them once enough coins were present in the system.

it's interesting that these people, who have no idea what an ip or load balancer or VM is, were able to pick up on this. i would guess that the h4xx0r5 actually spent little time using the service or had the bit of their own necks being exposed taint their assessment.

* yeah, i know no one really knows exactly what happened, but it's fun to poke fun.

</comment></comment></comment></comment><comment><USER7> Whoever wrote this article either very poorly understands network traffic or is being intentionally obtuse to create doubt. He states that there is no way that a public IP would show up in a packet capture over a Tor network. This is blatantly false and idiotic to think.

HTTP servers, caching proxies, TLS terminators and load balancers have a habit of putting extra HTTP headers in the HTTP response to make the path of a request more clear. It's very possible that under some error condition one of them included an X-FORWARDED-FOR header or something similar that included the IP. This is all application layer data and can contain IPs.

Even ignoring headers, they could have triggered an error page that just dumps all PHP server variables, one of which includes local IP addresses.

Edit: It's infuriating that this article author doesn't understand that a packet sniffer can see contents of the HTTP protocol.

<comment><USER8> _He states that there is no way that a public IP would show up in a packet capture over a Tor network._

No, he does not. He says that there's no way a public IP would show up at layers 3 or 4 of the OSI model. The author takes the position that the IP leak must have come from layer 7 (quoting from the article):

 _[...] and it is at the application layer that the FBI uncovered the IP address._

Depending on how one interprets the FBI's wording, this impossibility contradicts their accounting of how they discovered the IP. Let's look at what the FBI said:

 _Upon examining the individual packets of data being sent back from the website,3 [sic] we noticed that the headers of some of the packets reflected a certain IP address not associated with any known Tor node as the source of the packets._

Note that they're talking about headers _of the packets_ , not _HTTP_ headers. This is the FBI, though, so it's possible they confused "packets" with "HTTP requests". Taking their words at face value, though, the FBI seem to say that they got the IP address from the IP headers on packets received from Silk Road, or at least traffic generated by their browser on behalf of Silk Road. That's what the author argues is impossible.

Nobody, least of all the author, disputes that Silk Road could have leaked its IP in HTTP headers, or via some other misconfiguration. But if they did, why didn't the FBI just say that? Why claim that they leaked it in the IP layer?

Two possibilities come to mind. The first is that the FBI got the IP from the application layer, as everyone believes is possible, and misattributed the leak to the IP layer due to terminology confusion. The second is that they got the IP from another source, attempted to deceptively misattribute the source, and accidentally picked a fake source the leak could not have come from. If the second option is the case, we're looking at an example of parallel construction.

Edit: Grammar. Gotta keep up appearances.

<comment><USER9> _Note that they 're talking about headers of the packets, not HTTP headers._

HTTP headers are _in_ the packets, and I could see myself very very very easily writing "of the packets" in lieu of "in the packets". When proofreading, I'd be much more likely to catch the extra "3" in that same sentence, than to catch the s/in/of/ as introducing confusion.

<comment><USER10> My gut reaction to that is skepticism.

I've done basic switch configuration, looked at packets, had to troubleshoot MTU and MRU misconfiguration in commercial switches, etc.

Conceptually stuff at that layer is a long ways from the actual content.

It's like pointing to a guy walking his dog at the park after noticing a parked car with it's lights on, and saying "there's the owner _in_ the car" vs "there's the owner _of_ the car".

It feels, to me, like those sorts of linguistic mistakes must be exceedingly rare?

Or maybe the person writing up the statement wasn't the person who generated the notes and they just had a transcription error. I'd buy that I guess.

<comment><USER11> Your gut reaction is to lean towards conspiracy rather than a simple typo/transposition in the mind of the FBI agent who wrote the report?

<comment><USER10> That's awful snarky, and misses the point entirely.

My gut reaction is that if I, as a civilian, were to ask an FBI agent for details about a case, I'm as likely to receive half-truths and lies by omission as anything at all though. So if you want to be pedantic I guess so.

I did point out that I'd believe it could just be a transcription error by someone unfamiliar with the subject matter though.

<comment><USER11> The point here is that this is nothing more than a simple writing mistake on the part of the FBI. Pretending like there's some conspiracy or parallel construction taking place here is just nonsense and the folks suggesting it are doing so out of irrational fear more so than any actual evidence.

Your gut reaction seems to place you in the camp of the conspiracy theorists. That should raise alarm bells for you.

<comment><USER10> > The point here is that this is nothing more than a simple writing mistake on > the part of the FBI.

You say that like it's fact. But you're assuming.

I thought it was noteworthy that I can't recall ever seeing that specific error in the wild. Maybe I just missed it. Maybe it's a lot more common than I imagine. Maybe not. I don't know. Once it was brought to my attention I thought it was noteworthy though.

> Pretending

I'm not pretending anything. I gave an additional explanation I felt might be plausible. I'm sure there are others. I just doubt the plausibility of the one brought forward.

> irrational fear

Tomato tomato. There's no evidence it was a writing mistake, but you're presenting it as fact.

I guess it all boils down to this: Given domestic spying revelations I think you're extremely naive if you think it's _irrational_ to think that the FBI might be telling less than the whole truth.

Your gut reaction puts you in the camp of people who would volunteer information to a LEO. Which I think should raise a lot more alarm bells than me shrugging my shoulders and suggesting I'd need actual evidence before taking the FBI at it's word.

The very idea that you would is blissfully naive IMO. But good luck with that.

And if you have an point to make, a little less ad-hom and snark (aka Reddittude) would be appreciated. My original comment was a few wonderings- out-loud, including posing a couple questions for discussion. Nothing in there intended to be snarky. If you want to discuss further, it would be nice if you could approach the conversation with the same level of basic respect.

<comment><USER11> Sidestepping your bruised ego entirely, the fact that _you_ haven't seen someone mistake an IP header with an HTTP header is a terrible standard with which to judge whether or not such a mistake is possible.

This isn't really about you, it's about showing the other folks reading this how nuts you are, and luckily you're doing that for me.

<comment><USER10> That's a weird argument (that I never made). Though I have to wonder about that. IPv4 headers and HTTP headers don't much look the same IME (feel free to hit up Wikipedia if you're unfamiliar).

As far as "how nuts your are". You seem to be taking this way too seriously.

"My gut reaction is to be skeptical" somehow equates to, in your words:

      * Nuts
      * Conspiracy Theorist
      * Nonsense
      * Irrational

Expressing _skepticism_ that the FBI would "tell the whole truth and nothing but", despite high ranking intelligence officials outright lying to congress, and the history of the FBI being a bit checkered... _That_ you think should raise alarm bells. Skepticism.

If that's all it takes to be a conspiracy theorist in your book, well sign me up I guess.

The outright dismissal that it might just be possible you don't have all the facts (and I'm certainly not claiming I do)... That's an idea you dismiss outright. I have to wonder how you get there. Because if anything here seems _irrational_ , that's what I'd put my finger on. But maybe that's just me.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> So we have a pretty good guess of how they did it, and while their affidavit doesn't give a lot of important details, ultimately they were just interacting with a web service in the way it is supposed to be interacted with - <USER2> sending it http requests - and the buggy site revealed itself to them because the guy who wrote it wasn't a very good web developer.

I can understand why they wouldn't want to give more details than they have to; it makes the prosecution's work harder. But I and many others have been arguing for years that there's nothing illegal about sending messages to a server and looking at the responses, if you aren't trying to damage anything. I can't see any reason why it would be less legal for the FBI to do it.

<comment><USER13> _"...they were just interacting with a web service in the way it is supposed to be interacted with - by sending it http requests"_

That line of reasoning sounds a lot like the Andre 'Weev' Auernheimer case [0], where he gathered AT&T customers' email addresses by interacting with a server simply by 'sending it HTTP requests.' The FBI made its position clear on that case, prosecuting Weev for "conspiracy to gain unauthorized access to computers" and ultimately getting the guy sentenced to three years in prison.

The overarching circumstances are clearly different but undeniably parallel. It seems curious to me that the FBI could use these some sort of apparently 'criminal' tactics (by their own precedent) as legal grounds in their case against DPR.

[0] [http://www.huffingtonpost.com/2013/03/18/andrew-weev- auernhe...](http://www.huffingtonpost.com/2013/03/18/andrew-weev- auernheimer_n_2900387.html)

<comment><USER14> The slight difference is the FBI is investigating the source of the Silk Road (not illegal), vs. trying to obtain private information of folks from ATT (illegal).

<comment><USER15> There is no difference if they did not have a warrant. The FBI has the ability to break into your house, but if they do that without having a warrant they can't use anything they found against you.

<comment><USER14> I wouldn't call this breaking in, more like surveillance.

<comment><USER16> And with some very obvious exceptions, warrants are required to commit surveillance that violate certain privacy and property rights.

<comment><USER14> The thing is I don't believe the court will find this to be violating any laws.

<comment><USER16> I think you're probably right, but the argument up-thread was that the justice system is hypocritical regarding the permissibility of actions taken by the government compared to individual people.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> All of these analyses of how they might have located the server are great. I'm pleased to see folks trying to prove or disprove that the FBI is not trying some parallel construction here.

But what worries me is that all of this will have to be eventually dumbed down to an explanation that a jury of average joes can understand. The increasingly technical complexity of evidence in some of these cases worries me that our system doesn't have an adequate way to deal with it.

<comment><USER4> I share that concern in general (and in particular with regard to software patent cases, which often cover highly technical material and where the jury has no context about the novelty of the patented inventions), but I think this case is pretty much a slam dunk however you approach it. Regardless of how they obtained the IP, they got to the box to find it running Silk Road code, they got to the box's owner to find him holding Silk Road bitcoins, etc.

This isn't a case that hinges on a technical nuance. It will be practically impossible for DPR to convince a jury that there is any reasonable doubt as to his guilt.

<comment><USER18> If the evidence was obtained using illegal methods, all that evidence can't be presented at trial and the jurors can't take that information into account when deciding guilty or not. There are many people who are acquitted for things they obviously did, but for which there is no _formal legal_ way of proving it.

<comment><USER19> _If the evidence was obtained using illegal methods, all that evidence can 't be presented_

Isn't that for the judge to decide? Judge decides what can be presented to the jury, and jury renders the verdict, right?

Then the argument doesn't need to be dumbed down for the jury. They won't see it at all.

Although the argument will still probably have to be dumbed down for the judge.

<comment><USER20> Nope. A judge cannot decide to admit an illegally obtained evidence, no matter what it is.

<comment><USER19> OK but he gets to decide if it was illegally obtained.

<comment><USER21> This is decided in pretrial hearings. There have been entire cases based on the admissibility of evidence. If the judge rules it is or is not admissible, there is some manner of recourse or escalation for the usurped party. IANAL YMMV BBQ

</comment></comment></comment></comment></comment></comment></comment><comment><USER22> Is the constitutional issue here that exploiting a security hole (SQL injection, remote execution to dump the $_SERVER variable, what have you) constitutes an illegal search?

If so, is their any guidance from case law about the difference between some sort of legal poking and prodding vs illegal hacking?

<comment><USER23> In general in order to break the law you must have a warrant. For example, a cop entering private property without a warrant or probable cause is nothing more than a trespasser. These agents likely violated the CFAA by doing what they did. Because they didn't have a warrant, IMO it _probably_ constitutes an illegal search, but the courts may well have a very different opinion. Regardless of which side loses this argument, this will likely find its way to the appellate courts and possibly the Supreme court, as it seems to be an unanswered question of great importance.

<comment><USER24> From the CFAA:

 _(f) This section does not prohibit any lawfully authorized investigative, protective, or intelligence activity of a law enforcement agency of the United States, a State, or a political subdivision of a State, or of an intelligence agency of the United States._

I'm not sure what "lawfully authorized" means there, but my current belief is that one form of it is permission from the Attorney General.

<comment><USER23> But I wonder if computer hacking (which this likely would fall under) can be "lawfully authorized" outside the confines of a search warrant. The Attorney General could order the FBI to break into the computers of a hedge fund suspected of insider trading, but that clearly wouldn't be lawful unless authorized via search warrant. This is very similar.

<comment><USER24> Well, I don't think they broke into the computer. But I guess my point was more that the CFAA doesn't have any answers, you have to figure out the scope of the authority possessed by the Department of Justice.

</comment></comment></comment></comment></comment><comment><USER25> I think it's more likely by header the FBI meant HTTP response header not IPV4 header.

Legally I don't know if we should allow "fuzzing" without a warrant. They've put people in jail for doing less.

<comment><USER26> Weev was put in prison for wayyyy less than what the FBI did.

<comment><USER27> I don't think Weev should have been tried criminally at all, but I kind of disagree with this purely from a technical standpoint.

The FBI discovered a flaw that let them see the IP address of a web server.

Weev discovered a flaw that let him see private information of AT&T customers.

Customer information is generally considered much more sensitive than an IP address.

<comment><USER28> Privacy is relative. In Weev's case, AT&T considered the information of its users to be sensible, private information, while, in SR's care, the IP address is a very sensible and private information, considering that tor was used to hide it.

</comment></comment></comment></comment><comment><USER29> I'm suspicious of anyone who seems to have trouble understanding the FBI's claims. It's clear to me that what the FBI is claiming is that some part of the CAPTCHA was linked to the server directly, rather than its TOR address.

<comment><USER30> From the article:

Even in the hypothetical case where – for some unrealistic reason – the Silk Road hidden site was including an image on an external server by referencing its IP address or hostname, the agents would still observe this traffic as having come from Tor. There is no magic way that the traffic from a real IP embedded within the HTML of a hidden service would find its way directly to a client without passing over the Tor network and through Tor nodes. Were this the case, it would be a huge vulnerability in Tor, as it would allow the administrator of a hidden site to uncover visitors by including an element that is served directly to the client over clearnet (thankfully it isn't and this doesn't work – try it).

<comment><USER31> I don't think they are presenting it accurately. I think what they are indicating is it was something like this:

        Login page
        ...
        $code_to_embed_captcha
        ...

I believe $code_to_embed_captcha was returning something along the lines of [http://REAL_SERVER_IP/captcha.jpg](http://REAL_SERVER_IP/captcha.jpg) instead of [http://ONION_ADDRESS/captcha.jpg](http://ONION_ADDRESS/captcha.jpg)

This wouldn't allow you to identify users, the request for captcha.jpg is still routed through TOR. However it does reveal the true IP of the server.

<comment><USER32> This is my guess as well. That or the captcha had a query parameter like ?nonce=1234&redirect_url=$HOSTNAME/login

</comment></comment></comment></comment><comment><USER33> We just barely have some solid ground protecting phones you have on-person from unwarranted search, and, AFAIK, there's no real laws protecting your rights when you have data residing on third party servers in foreign countries.

I'm definitely not a lawyer, but the server had an unknown provenance and it was physically located in Iceland, and that probably makes for some fuzzy ground at best as to whether or not what _anything_ the FBI did to gain some sort of access to the physical server, even if it was made possible by parallel construction, would even be illegal.

</comment><comment><USER34> Look, this is a great chance to see if anything is amiss. We can simply directly test our hypotheses:

Start up the Sill Road server with a tor circuit in front of it (isolated from the main network.) Then have the FBI demonstrate how to get the server to cough up it's IP. It's easy to prove who's telling the truth here.

<comment><USER35> not really, it could be parallel construction : they found the flaw after having seized the server.

</comment></comment><comment><USER36> Well, I can't say much about the reality of the FBI approach, y'all seems to have that better covered.

Instead, I'm led to ask at what point will people stop using "myself" as a formal version of "me"? It isn't, it actually has a completely different usage.

In this case: "the SR Server was located by myself and another member of the CY-2". Here 'by myself' means 'on my own', not 'by me', so it directly confuses the meaning of this sentence. To me the obvious meaning is that they both work independently and found the same thing.

About the only other time you can use '-self' is in the reflexive, as in "I did this to myself", "you do this to yourself". I will _never_ do something to yourself, and you will _never_ do something to myself.

So, I know this is completely out of place, but if I can get one person to not do this in the future, I will make my life a bit happier. Also if I can persuade that small group of nameless people (you know who you are) that "could care less" actually means the opposite of what they think it does; and that "alot" is not a word.

I'm all for the English language evolving and expanding, but let's weed out the bad ideas which only increase confusion and bring nothing good. And seriously, "alot"? "Lot" is a three letter word, it's not hard to spell. As for "a" ...

<comment><USER21> I understand your confusion, but this is a common usage case for the nominative first person of "me" in formal speech. This is one of a gajillion special case word choice situations. This particular sentence structure (Direct object, past perfect verb, subject) suggests "myself". However, the writer should have put in the other person's name before "myself", as is common practice, leading to confusion.

Consider: "the SR Server was located by another member of the CY-2 and myself".

I'm not claiming that "me" wouldn't parse, but that using the current rules properly would not cause confusion in the first place. Someone is trying to sound intelligent, while putting their name first so as to emphasize their contribution.

<comment><USER36> So, I'm definitely not confused :)

I do appreciate it's very common in what I'd call pseudo-formal speech, mostly used when trying to impress someone one perceives to be of higher social station. Estate agents do it a lot when talking to clients. And so it does have the "well, lots of people do it" support.

However, if you go back to the roots (Latin), "myself" is specifically the reflexive form of the first person. When the first person is the object or the dative, it is "me/to me". So to have correct agreement with the verb, your example should end in "and me".

While your example is easy enough to parse, it doesn't actually follow the rules - though maybe the common practice. And if you rearrange the two object nouns (i.e. back to the original) the sentence loses clarity. Using "me" never loses clarity, and the ordering of object nouns shouldn't matter beyond providing emphasis.

Basically, that example is ok in Perl, but not in Java ;)

And ultimately, using the reflexive form incorrectly never makes a sentence clearer, and it's longer to write and say.

Anyway, this is so far off the OP's topic, which is actually much more important & interesting, I do apologise to him/her.

</comment></comment></comment><comment><USER37> If they obtained a warrant, is fuzzing/hacking Silk Road the equivalent of kicking down a door during a raid?

<comment><USER38> They didn't obtain a warrant.

</comment></comment><comment><USER39> If I ever set up a hidden service I'll make sure it lives on a box that has no public IP whatsoever. Preferably a physical box.

<comment><USER40> Unfortunately, the technology you need to do that is not yet finished: [https://trac.torproject.org/projects/tor/ticket/9498](https://trac.torproject.org/projects/tor/ticket/9498)

ETA: I should have said "one possible technology"; there may be others, but I am pretty sure that an IP-less Tor node requires that you play the Tor Bridge game and stream the Tor protocol over a non-IP link. I have had a prototype of this design running, but have yet to have it to a point I consider robust.

<comment><USER41> praptak didn't say IP-less, but no _public_ IP. If the server has an IP in the 192.168.1.X range, that doesn't tell an attacker much, supposedly.

</comment></comment></comment><comment><USER27> While I agree that the FBI's explanation as they gave it is impossible, I see 2 possible explanations for how their explanation would make sense. I don't know why they would glaze over these details, but:

1\. They could have meant simply viewing all the HTTP response headers with something like Chrome's dev tools, Firebug, or possibly a proxy like Burp Suite.

2\. They may have set up Tor to essentially use no hops and use only their own host as an exit node. That way they would be able to directly observe traffic from a clearweb IP address in a PCAP. Or they could set just a single hop and specified another host they control as an exit node. This seems way overboard just to find an exposure like this, but it's possible that the FBI or NSA have such test environments to probe for other Tor-related vulnerabilities and misconfigurations.

</comment><comment><USER42> The thing that struck me as the most odd about this article was the assertion that keeping a hidden service securely hidden must be easy, because the TOR website's Wiki article on it is so short. I think it's clear that there isn't that much proven experience out there in keeping a hidden service secure against a determined and well-funded adversary, and what experience there is probably hasn't made it to the TOR Wiki page for various reasons. Just the last HN article on this generated dozens of comments with various best- practices for securing such a service that clearly weren't followed by Silk Road, and we're mostly amateurs at this.

</comment><comment><USER43> The guy was _not_ a good developer and they caught him because of it. It really does appear to be that simple. No parallel reconstruction needed when you're dumping server vars to the public.

</comment><comment><USER44> Slightly off topic: could you please increase the font size on your site? It's actually below 12pt. The blockquotes are more readable than the body of the post.

<comment><USER45> It is 17px for me, but most browsers support zooming into a page, that will make text easier to read on almost any site, but could make it more difficult if accidentally zoomed out.

<comment><USER46> Every browser has supported this since forever

</comment></comment></comment><comment><USER47> Why is using var_dump debugging bad practice? Or is he insinuating that using var_dump while testing on a live site is bad practice?

<comment><USER4> I think he infers that both are bad practice, but for the record I don't think there's anything wrong with var_dump debugging. Sure, a real interactive debugger is nicer, but it can be a pain to set up with PHP (I know, having set these up several times). Print-based debugging is fine as long as you don't expose sensitive information to end users.

<comment><USER48> Difficult to set up? I disagree, Xdebug with connectback is amazingly easy to set up in a Linux VM! On windows, yes, it's hard, but Linux and OS X its very simple nowadays. I should write blog post on how to do it (having done it dozens of times over the past 6 months)

</comment></comment></comment><comment><USER49> 1\. get warranty to track someone id'ed as recipient of usps drug shipment.

2\. exploit client tor missconfig

3\. repeat 1-2 until you have a net so wide of tor users that only node out is the service

</comment><comment><USER50> Nothing will keep you from getting hacked by a determined government, but using Apache/PHP isn't exactly trying to make their life difficult. But at least they didn't use Ruby on Rails. :P

<comment><USER51> So you are saying Rails might make intruders' lives less difficult? Of course, the _implementation_ is what would be more or less secure, but i'm a little curious what you meant.</comment></comment>


Ask HN: What are some missing Rust libraries? - <AUTHOR>

<comment><USER2> For me it's not that Rust would be missing libraries but that I'm too of a bash noob to get those libraries to build on windows/msys2. The rust wrapper to stb_image is one such example.

</comment><comment><USER3> [http://en.m.wikipedia.org/wiki/Simple_Network_Management_Pro...](http://en.m.wikipedia.org/wiki/Simple_Network_Management_Protocol)

Specifically SNMPv3

<comment><USER4> Thanks. I wrote a partial implementation in Go [0] and I think a Rust version would help me learn Rust, and hopefully contribute to the ecosystem.

[0] [https://github.com/PreetamJinka/snmp](https://github.com/PreetamJinka/snmp)

<comment><USER3> Oh, that's great. I've been hoping for a nice Go implementation as well.

Right now, I use net-snmp, which is featureful, but the API(s) are very poorly designed.</comment></comment></comment>


$35/Week – Eating well on $35 a week - <AUTHOR> http://erikaugust.com/thoughts/thirty-five-dollars-a-week-preface/ <comment><USER3> Anyone serious about living on little should read [http://www.theskintfoodie.com/recipes.html](http://www.theskintfoodie.com/recipes.html)

And if you're really serious about living on less than that then I can only recommend Orwell's "Down and Out in Paris and London" for the wonderful tip of smearing garlic on a knife before you put butter on bread to convince your body you've eaten a hearty meal.

I personally lived a couple of years on [http://www.complan.com](http://www.complan.com) which is about as close to Soylent as you get when you're homeless and broke in 1994 and holding on to the hope that it might stave off whatever the next illness is going to be.

My personal tip: Stale bread, old tomatoes, old cheese = pizza for less than 20p.

<comment><USER4> I usually just dice up garlic and throw it inside a partially filled bottle of olive oil. IIRC olive oil is actually cheaper than butter.

<comment><USER5> Better throw some acid in there or you run quite a significant risk of death from botulism.

<comment><USER4> Holy shit TIL. You may have just saved my life. Thank you!

</comment></comment></comment></comment><comment><USER6> 35$ are about 26€ - it's definitely possible in Germany. You can buy 1kg oatmeal for 1€, 1kg yoghurt for 1.50€. 2kg whole-grain bread for 3€, 2kg potatoes or pasta or rice for 2€, 10 eggs for 1€, Then you have plenty of possibilities and still 17€ to add vegetables/oil/spices and other ingredients and create tasty meals.

The difficult part is planning meals for 1 or 2 weeks in advance and keeping variations alive.

Also avoid soda of all kinds and only drink tap-water and coffee/tee to safe the money. Also preparing sandwiches in advance while travelling saves a lot of money...

It's surely possible to live healthy on the cheap but it takes a lot of effort and time and structure to do so..

</comment><comment><USER7> Have you tried cooking dry beans? It is really easy and cheaper than canned. I bring them to a boil, can the water and then put them in the oven. They last for days in the fridge.

<comment><USER8> I just bought an Indian pressure cooker which is great for cooking them even faster (can even skip the pre-soak if you want) and uses less energy. So far I've tried "split red lentis", "whole black urad dal", and dried kidney beans. They are super cheap and come in huge bags for $5 or so. 1 cup of these makes many servings. I started making some Indian dals using them. It's a great way to eat them, with tons of Indian spices.

The pressure cooker I bought: Hawkins 5L classic: [http://www.hawkinscookers.com/1.1.1.hawkinsclassic.asp](http://www.hawkinscookers.com/1.1.1.hawkinsclassic.asp)

<comment><USER9> I'm guessing you already know this, since you're referring to "urad dal" by its native name and buying it in large quantities, but others might not:

Here in NYC at least, foods like beans, lentils, rice, spices, etc. are much cheaper when bought in Indian food stores (where they're everyday staples sold in large packages) rather than in regular supermarkets (where they're sold in much smaller packages).

There are similarly good deals to be found in Chinese and other ethnic grocery stores.

</comment></comment></comment><comment><USER10> Pretty great cookbook I've taken a few meals from: [http://www.leannebrown.ca/cookbooks/](http://www.leannebrown.ca/cookbooks/) Rather than cheap food that happens to be edible, she tried to focus on actually good food that happens to be cheap. Really well done. Free electronic copies are always a plus too...

<comment><USER11> A book that has been fascinating me lately is How to Cook a Wolf[0] by MFK Fisher[1], which is absolutely lovely, inspiring and amazing. A sort of home ec., slice of life bunch of anecdotes, ideas and story-telling.

[0] [http://www.amazon.com/How-Cook-Wolf-M- Fisher/dp/0865473366](http://www.amazon.com/How-Cook-Wolf-M- Fisher/dp/0865473366)

[1] [http://en.wikipedia.org/wiki/M._F._K._Fisher#Books](http://en.wikipedia.org/wiki/M._F._K._Fisher#Books)

</comment></comment><comment><USER12> I am happy to see this. Coincidence is that I am on €49 budget right now (training self discipline). Thats €7 a day. BUT - I do not eat anything from cans or anything pre-made. All fresh meat, veggies and self made food - for 2 people. No bread and avoiding packaged food.

Sure - you wont get any Whiskey or Cola for it. You wont be able to go to restaurant. Or buy crisps. But thats the point - living healthy.

I dont like how author lives off cans - its not that healthy. You can make most of food much cheaper at home. Cans are mostly unnecessary. I propose to all to try this. Leaving for udner $10/€10 a day out of all healthy and self made foods.

<comment><USER13> As long as you avoid dented cans (risk of botulism) and strain off the liquid before using the contents (too much salt, risk of high blood pressure and kidney failure), canned foods are perfectly healthy.

</comment></comment><comment><USER14> Looks familiar.

I grew up eating lots and lots of rice, beans and frozen vegatables and I'm still pretty fond of them - cheap and tasty, cheaper if you do your own seasoning and cheaper still if you buy dry beans and aren't particular about your rice.

Personally, my big issue in weekly food cost is eating out at work - <USER2> $10+ per day for worse versions of stuff that could be made for 1/5 that price with the right discipline.

</comment><comment><USER15> Dang, not bad. I also buy my food weekly at Trader Joe's and have been spending twice that much ($70 a week):

7 * 1lb Grass Fed Angus Beef $6 = $42, 2 * 8oz Kerrygold Unsalted Butter $3.25 = $6.50, 2 * 12oz Kerrygold Dubliner Cheese $4.50 = $9, 2 * 12 Cage Free Eggs $3.25 = $6.50, 2 * Charles Shaw Cabernet $3 = $6

<comment><USER13> Are you on a paleo-and-wino diet?

</comment></comment><comment><USER16> Shopping at Trader Joes is a good start for keeping within budget, but you can't really make the case for eating well on $35/week if you go out occasionally and DON'T count that in your budget.

<comment><USER13> Under this system, I'd just eat out for every meal and net a budget of $0/week.

</comment></comment><comment><USER17> I imagine it is easy to eat well on $35 a week if you consider a PB&J sandwich to be a "euphoric" experience. This does not match my idea of eating well, nor that of most people I know. This seems a bit like an article on how to become rich working at McDonalds defining "rich" to mean "earns at least $5000 a year."

<comment><USER18> What are you expecting, duck confit and beef wellington?

Eating well in this case means being satisfied and healthy. A peanut butter sandwich on whole wheat bread is surprisingly healthy (although I personally loathe whole wheat). It's not classy but it's a highly cost-efficient source of calories.

Also, it turns out that even if you are willing to lower yourself to peanut butter sandwiches, getting 2500 calories of nutritionally well-rounded and healthy food for $5 a day takes quite a bit of planning (if not cooking). If you "imagine it is easy", it's only because you have not given a fig's worth of actual thought to the process.

<comment><USER17> > _What are you expecting, duck confit and beef wellington?_

I'm expecting an article about "eating well" to be about eating _well_ , not just merely subsisting.

> _Eating well in this case means being satisfied and healthy. A peanut butter > sandwich on whole wheat bread is surprisingly healthy (although I personally > loathe whole wheat). It 's not classy but it's a highly cost-efficient > source of calories._

I don't know about you, but I would absolutely not be satisfied with a diet made up mainly of things like peanut butter sandwich. Like I said, I've eaten on that level, and I didn't feel very satisfied with it.

<comment><USER19> Eating well is going to differ, but you have to have a little bit of common sense. How 'well' do you expect to eat for $35 a week?

How much further beyond PB&J do you expect to go? How much further CAN you go?

It's also going to be relative... What's the best you can do for $35? It sure as ___ won't be steak (unless you go hungry the rest of the week)...

</comment></comment></comment></comment><comment><USER20> Are there any subscription services that let me eat healthy and don't have to cook?

Better yet, feed me like a hamster :P

<comment><USER21> [http://www.soylent.me/](http://www.soylent.me/)

<comment><USER22> There is a giant modal in the way of everything when I load this site. Regardless, I've heard of this before and it seems interesting.

Has anyone tried soylent?

<comment><USER23> I rather like it. I wouldn't live on it (one tires of it quickly) but it's really nice to have healthy, cheap, easy meals available when there's no time to cook (pre-work breakfast) or when I've forgotten to shop (late night dinner).

</comment></comment></comment></comment><comment><USER24> Is "bought from a store" a requirement here? I've built a small garden on my balcony, and it seems that such could meaningfully supplement a $35/week budget; especially given an objective of "organic".

> I will likely spend up to 20% more to buy organic.

</comment><comment><USER25> That's five dollars a day, or less than $2 per meal (assuming three meals a day).

There's this product called Soylent[1] that's supposed to be pretty healthy and costs less than $4 per meal.

[1]: [http://www.soylent.me/](http://www.soylent.me/)

<comment><USER26> The subject is eating well, not avoiding eating entirely. You don't eat Soylent, you drink it. A lot of people still would like to eat.

</comment></comment><comment><USER27> Just the cupboard section would cost me more then $35 a week.

<comment><USER28> Thats his total contents, which is like 20 or so meals. Maybe not good meals, but 20 or so things.

I probably have $300-$400 bucks of "stuff" in my cupboard but I don't eat all of it every week. I do "too much" amazon subscribe and save and I like spices so maybe a "normal" cupboard only has $200 or so of stuff.

I think the idea is to promote creative thinking. So I'd fry up about a 1/10 of a bag of frozen broccoli and cook the mac n cheese and then add a can of tuna and mix all that stuff up into a tuna casserole and bake it, maybe broil it to brown the top which is WAY more than one meal for one single dude even accounting for the long term effects of food inflation (like mac n chee used to be a pound in the 90s, now its only like one cup in the 2010s). Maybe the day after tomorrow when the casserole is finally eaten up, a dash of oil in the wok with a bag of mixed veg and those meatballs and then some kind of homemade sauce, I bet you get two meals outta that at least.

Dude needs to buy more fruits and veg and more spicy stuff. I see vegetarian burritos and tacos and spring rolls in his future.

</comment></comment><comment><USER29> I look forward to your next post on recipes. Thanks for this!

</comment><comment><USER30> tldr: grains and legumes are cheap and healthy

</comment><comment><USER31> If you are taking supplements clearly you are not eating well.

</comment><comment><USER32> What a bad joke this is, only in the third world would this qualify as "eating well."

<comment><USER33> "well" is used in a different sense than you are using it. They mean enough calories with a reasonable amount of nutrition. From a feed-your-body perspective, peanut butter on whole wheat bread is pretty decent food (it's even protein complete).

To me the silliest part of the article was _Food is one of the few true necessities in life, yet overlooked in mainstream thought._ I would say that mainstream culture is obsessed with food.

<comment><AUTHOR> Good call - that was a vague statement - I've refined it a bit: "Food is one of the few true necessities in life, yet alternatives to junk food and eating out are often overlooked in mainstream thought." At least closer to what I mean, anyway. Thanks for checking out.

<comment><USER33> I think there is a good chance you are taking an observation from your life, something about lots of people being uncertain about cooking, and (falsely) translating that into a lack of consideration for cooking. Even something crassly commercial like "The Food Network" spends an awful lot of its time on real actual _cooking_ (and there are dozens of cooking shows on other channels).

And it just goes on from there. What could be more mainstream than Rachel Ray and her talk show? And she is big on ingredients and cooking them.

Or do a survey of "food section"s in print. The HuffPo, of all things, focuses on recipes:

[http://www.huffingtonpost.com/food/](http://www.huffingtonpost.com/food/)

Lots of real papers have lots of recipes and content that is not restaurant reviews.

Sorry for focusing on one small point and being cranky about it.

<comment><AUTHOR> Fair points. But I wonder why I see it this way? Maybe for my age/demographic? 30 year old urban professional?</comment></comment></comment></comment></comment>


Kids these days: the quality of new Wikipedia editors over time - <AUTHOR> http://blog.wikimedia.org/2012/03/27/analysis-of-the-quality-of-newcomers-in-wikipedia-over-time/

<comment><USER2> It's not even a new user issue. I've been an editor for going on 5 years, and I still get templated and reverted on inconsequential edits.

Case in point: I recently edited this picture: [http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.pn...](http://en.wikipedia.org/wiki/File:Turing_Machine_in_Golly.png)

It's a screenshot of the 6 octillionth generation of a Turing Machine built in Conway's Game of Life, using the GPL'd program Golly. The previous image contained Windows GUI elements (title bar, etc) which are non-free and cannot be used on WP when a free alternative exists. So I cropped it and uploaded the edited image.

Within minutes, I had received a template on my (seldom-visited) commons talk page informing me that I had uploaded a file without specifying its license, and that it would be deleted. Take a look at the "Permission" field on that image: "See LICENSE.TXT distributed with Golly for GPLv2 license"

Despite this (incredibly clear) assertion that the image was GPL'd, I received a warning that it would be deleted. Why? Because I hadn't included the "This image covered by the GPL" template that a) I didn't know existed, b) there was no mention of on the upload page, and c) is a wordier version of what I wrote in the license field.

As an experienced editor, I'm used to these stupid quibbles and time-wasting fights. I'll still contribute, although they are a large part of why I don't contribute more. As a new editor seeing this, however? I would have told them to fuck off, got banned for incivility, and never gone back.

There's an attitude among the regulars that Wikipedia is a treasured resource that must be defended against innumerable vandals, trolls, and spammers by a select cadre of noble volunteers. To an extent, they're right. But when you have such badges as "The Defender of the Wiki Barnstar" [1] being held up as the height of achievement for veteran editors, it engenders a culture that is exclusionary, if not actively hostile, towards new editors.

[1] <http://en.wikipedia.org/wiki/File:WikiDefender_Barnstar.png>

<comment><USER3> Are you sure showing title bars isn't allowed? Almost all screenshots on Wikipedia include them, even for open source software. e.g.:

<https://en.wikipedia.org/wiki/File:Firefox11.png>

<https://en.wikipedia.org/wiki/File:Winscp_screenshot.png>

<comment><USER2> I've had images with them included deleted before, which is why I modified the golly screenshot. To be perfectly honest, I'm not sure _what's_ allowed, and I doubt many people who are not active commons contributors do, either.

<comment><USER4> Some editors went through and deleted a massive number of useful Mac/Windows screenshots several years ago, but I don't think there's ever been an actual policy stating these screenshots were a legal issue.

Also, the GPL only covers code, not program output (under normal circumstances), so I don't see how it applies to screenshots anyway.

<comment><USER2> >the GPL only covers code, not program output (under normal circumstances), so I don't see how it applies to screenshots anyway.

It's an interesting question. The drawing of the window is, I would argue, indistinguishable from the code. It's like saying the HTML, CSS, and images on a site are GPL'd, but screenshots aren't. The rest of the program output is merely algorithmic, and thus not subject to copyright.

<comment><USER4> This discussion is wandering off into la-la land. The GPL isn't designed to cover things like websites or bitmapped images. There's other 'free' content- oriented licenses which should be used instead.

</comment></comment></comment></comment></comment></comment><comment><USER5> I'm glad that they're finally coming around to the realization that Wikipedia has become increasingly closed to new contributions, and that they've stopped touting the (patently absurd) hypothesis that new users just don't "get it." (The fact that they'd even think, let alone think _first_ , to blame the users is just a giant head-scratcher).

As a simple UX experiment, I would ask new users this: try to contribute substantively to any article on Wikipedia. Just try it. Make a good-faith, high-quality edit to a page, and see how long the edit is allowed to stand. More likely than not, the contribution will be automatically reverted, within milliseconds, by a bot. If it's not, it'll be hand-reverted by a hardcore Wikipedia editor -- part of the statistically small, but disproportionately powerful cadre of self-appointed content cops, who seem to see their jobs as being bulwarks against change. In its zeal for the trappings of due process -- attributions, anti-"vandalism" policework, source checks, guidelines, and so forth -- this clique has lost sight of the net effect it's had on the site, which is to calcify and close off the free exchange of information that was so crucial to Wikipedia's early growth.

IMO, Wikipedia has faced a fundamental challenge in recent years: namely, that content-quality efforts have threatened new content volume. I don't envy this strategic predicament, being forced -- quite literally -- to choose between quantity and quality. It's not an easy balance to strike, and, given the circumstances, Wikipedia's historic track record is quite admirable. Recently, however, the balance has tipped too far in the direction of quality-policing. And now it's starting to undermine the core tenets of the project. I remain optimistic that Wikipedia (and/or the Wikimedia Foundation) can right the ship. But it'll have to mean a substantial uprooting of some bad seeds that have been allowed to take hold for years now.

<comment><USER6> _(The fact that they'd even think, let alone think first, to blame the users is just a giant head-scratcher)_

Logically, I can see this. However, when you survey the world, you see a whole lot of blame directed at users. (With certain tech/business subcultures as notable exceptions.) I'll note that when our brains evolved, there was no software nor was there UX, and the only suitable targets for blame were people and critters. I'd bet this is a human cognitive bias.

<comment><USER5> Largely agreed, but if you'll permit me to get cute, I'd suggest that's there's _always_ been UX. Whether in city design, the evolution of tool usage, invention, product design, etc. UX is at the very heart of why we, as humans, always seek to improve upon our lot.

The difference with software is that, for the first time in our history, we're able to measure, isolate, quantify, and control the elements of UX better than we've ever been able to. (It helps that software is often experienced in isolation from its environment, so UI can be more closely correlated with UX than it is for other domains). UX was a fuzzy, ethereal, probably subconscious concept that only recently became a serious discipline. But it's always been important. And our brains _have_ evolved to conceptualize it, albeit intangibly until now.

<comment><USER6> _Largely agreed, but if you'll permit me to get cute, I'd suggest that's there's always been UX. Whether in city design, the evolution of tool usage, invention, product design, etc._

Permission denied. :) Everything but tool usage is in the realm of cultural evolution. For the simplest tools, most all of the error is user error. There's simply not so much functionality in a stick that isn't mostly dependent on user actuation.

 _our brains have evolved to conceptualize it, albeit intangibly until now._

That's my point. UX flaws that are independent of user error have been largely intangible until now.

<comment><USER5> Define "cultural evolution," though. I'm not sure what you mean there. Not trying to be difficult, because I rather enjoy this conversation. Just unclear about the distinction you're drawing.

I'd argue that UX design -- even if it didn't have that exact name -- has been a distinct discipline long before software. Just ask anyone in the food service industries, the retail industry (department stores were basically innovations in UX in retail; so was IKEA), the casino gaming industry, the amusement park industry, and so forth.

Casinos, in particular, are fascinating UX case studies. The person who first thought of modern casino layout, comping free drinks at table games, oxygenating the gambling floor, removing clocks from the walls, comping rooms and other amenities for big spenders and regulars, which games to place adjacent to which others, etc., was a UX designer in spirit if not in title. And those decisions were pretty rigorously tested and quantified. These things may not meet the technical definition of UX as we commonly speak of it on HN, but they certainly hold with the spirit of the discipline Don Norman would later come to articulate as "UX."

<comment><USER6> _Define "cultural evolution," though._

As for a definition, I'm talking about the evolution of individual behaviors as transmitted through culture. If one somehow rendered the entire human race sterile, but we continued to propagate ourselves for the next 2000 years through cloning, you'd still have "cultural evolution."

 _I'd argue that UX design -- even if it didn't have that exact name -- has been a distinct discipline long before software._

Again, I don't disagree. That you bring this up indicates to me you've missed my point.

 _food service industries, the retail industry..., the casino gaming industry, the amusement park industry..._

All of these predate most of the evolution of the human brain's structure and capabilities. It's somewhat true that there were "UX errors" before the stone age. I say "somewhat" because it's really hard to delineate these as entities without a certain degree of technology. When all you have are sticks, what is the error of the designer and what is the error of the user? Maybe the user's just "holding it wrong?"

It seems to me, that we're likely to assign blame to sentient and animate entities. And even if the stick wasn't "whittled correctly" according to Thag, maybe it's just fine to Ookla? It's just hard to talk about "UX errors" as quantifiable entities until we get standardized production and large sample sizes.

<comment><USER5> If I understand your original point correctly, it's this: it is human nature to attribute most things to user error, ergo, my assertion that Wikipedia's blaming its users was a "head-scratcher" was off the mark. Our evolved inclination, which predates the discipline of even _considering_ UX as a tangible -- and, more important, a controllable -- concept, is first to start with the hypothesis that the user is in error. (And, furthermore, that such a hypothesis is not necessarily unjustified by historical frequency).

I actually agree with you here, but I think this point and mine are not so much at odds, as they are orthogonal. My point is that, human nature or not, Wikipedia came about in the modern era. Even if our cognitive bias/inclination is toward blaming the user, we have tools and analytic frameworks at our disposal which exist precisely to allow a necessary check against our brains' heuristics. Those checks should have been run by the Wikimedia elite. While I'll admit that "head-scratcher" is an unfair description, rendered mostly for rhetorical effect, I believe my point still stands. We have modern tools at our disposal, precisely because we are now -- uniquely, in our history -- aware of our brains' strengths and weaknesses in pattern recognition and situational assessment.

If this is not an accurate summation of your position, then I'll freely admit that I'm missing your point.

<comment><USER6> No, that's it. Strangely enough, we were basically agreeing the whole time.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Like a lot of people here, I have sporadically read Wikipedia for years. I came on board as a Wikipedia editor in May 2010 after meeting the project's co-founder and his family in person the year before. I've since seen some of those immediate family members on another lengthy occasion. My children regularly interact with that family in an online education community. Through a web of mutual friendships, I thought I had some sense of what the community norms would be like on Wikipedia before I started. Moreover, I began editing only after reading the several published books about Wikipedia available at that time (as disclosed on my Wikipedia user page), and came on board as someone who has actually had both academic journal editing positions and paid journalism editing positions before Wikipedia even existed.

Even at that, I get a lot of well sourced edits reverted by ideologically motivated drive-by I.P. editors as part of an ongoing process of edit-warring on articles that I happen to know sources for.

[http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/...](http://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests/Case/Race_and_intelligence)

For some controversial topics, no amount of good-faith editing by editors who actually know how to look up reliable sources and how to have civil discussions of controversial issues can overcome the flood of point-of-view pushers (both I.P. editors and registered editors who are sock puppets or meat puppets of previously banned editors) who want to drag down the project to below the level of a partisan blog. There simply isn't any incentive in today's atmosphere on Wikipedia for readers who actually know what encyclopedias look like and who have actually engaged in careful research on controversial topics to devote any of their time and effort to Wikipedia.

My number of edits per month has plummeted, and mostly I wikignome to clean up copyediting mistakes on miscellaneous articles written by young people or foreign nationals who didn't write grammatical English in the last revision of the article. The way to increase participation by productive, knowledgeable, literate editors is to drive away the ideologues and enforce some reasonable behavioral norms on article talk pages and user talk pages. I see no sign of that happening over at Wikipedia, and until I do, I will heartily support anyone's effort to build a competing resource, either limited to a specialized topic or a direct attempt to build a better quality general online encyclopedia.

I think the "Lamest Edit Wars" page in project space sums up much of what is amiss about Wikipedia.

<http://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars>

</comment><comment><USER8> It's good to see real data to address this question, rather than the neverending stream of basically anecdotal information about the problem that we've been slinging around. It's especially interesting to see the percent of not just "good faith" (flawed but well-intended) edits that are reverted but also the percent of "golden" (actually contributing) edits that are reverted; this sort of hostile drive-by is discouraging even for experienced editors.

One thing I see in the graphs is that the "survival rate" of editors who made a good-faith first edit was already in relatively steep decline by 2005, but the same rate for editors whose first edit was golden remained on a high plateau through 2006 and then just categorically dropped off a cliff. What happened then?

<comment><USER9> In 2006, bots were introduced to do various clean up tasks automatically. There are currently over 800 registered bots on Wikipedia. They are generally made to enforce the rules. But if you need 800 different bots controlled by 800 different people to enforce the rules, that means you have a lot of obscure rules.

For instance, there is a list of domain names that wikipedia sees often as spam. So if an innocent user goes to edit a page, and wants to add a new piece of information referencing a domain on that list, a bot will come along and mark it as spam (and revert it) even though its a good faith edit.

It can be frustrating and confusing for new users. The bot is often quite terse and rude, and so there is no incentive to return to wikipedia to make edits. 9/10 new edits are rejected, so no wonder they can't keep new editors.

</comment></comment><comment><USER10> "What this means is that while just as many productive contributors enter the project today as in 2006, they are entering an environment that is increasingly challenging, critical, and/or hostile to their work. These latter findings have also been confirmed through previous research."

This confirms to me what I argued on HN last year: <http://news.ycombinator.com/item?id=3272926> and <http://news.ycombinator.com/item?id=3273204>

</comment><comment><USER11> NPR's Talk of the Nation recently had a segment devoted to the bureaucratic run-around that can happen on Wikipedia.

[http://www.npr.org/2012/02/22/147261659/gauging-the- reliabil...](http://www.npr.org/2012/02/22/147261659/gauging-the-reliability- of-facts-on-wikipedia)

<comment><USER12> This is a great example of my complaints elsewhere in this thread, thanks for the link.

So listening to this show, there's an expert guest, a couple call-ins (including another expert guest) and even a caller who talked about how important the crowd sourcing is, who all more or less say the same thing I'm saying, trying to contribute to wikipedia is a pain because of the overzealous editor's arbitrary and capricious application of largely nonsenical rules -- the foundation representatives? Regurgitation of the guidelines and nonanswers. Then lots of lip service to the importance of crowd sourcing an encyclopedia without acknowledging that the crowd is turning into a very small gang and "let's make new contributors feel more welcome" without any particular ideas presented of how to do it.

</comment></comment><comment><USER13> This ignores the fact that the standard for what is considered a good edit has risen dramatically over time. Most of the featured articles from the early years of the site no longer even meet that standard. There are 3,500 featured articles, and almost 1,000 that have become unfeatured as standards have risen.

<comment><USER14> No, they used the same method to judge edits over the entire sample space.

> To test the hypothesis ... we randomly sampled the first edits of newcomers > to the English Wikipedia from the earliest days of the project to the > present. With the help of some experienced Wikipedians, we hand-categorized > the edits of 2,100 new users according to a four point quality scale

<comment><USER13> That only shows that the edits have been roughly the same quality of time, what I'm saying is that edits of the same quality are now judged much more harshly than they used to be. For example six or seven years ago if you wrote something that was true but unsourced, or else the citation was formatted incorrectly, the text would generally be left up and a higher level editor would get around to hunting down the proper citation eventually. But today these kind of edits would get instantly reverted, even though they create value. The issue is that because so much of Wikipedia is already properly cited, adding new text that is correct but not properly cited temporarily lowers the overall quality of the encyclopedia even though it would ultimately create value if left up and fixed, so essentially what we have now is one of those game theory failures.

This really isn't that hard to figure out and all they'd have to do is slightly tweak the design to fix it, so I don't really see why they're having so much trouble on this one.

</comment></comment></comment><comment><USER15> Is it possible that the opportunities to make quality edits has decreased as wikipedia has matured?

What I have in mind is the possibility that topics that haven't been well written out are like low-hanging fruit and are easy to positively contribute to. As wikipedia has expanded, that low hanging fruit has disappeared to some extent, and thus new editors have fewer opportunities to actually provide quality edits.

This explanation does not place blame on either newbies or pre-existing users who are "content cops". Instead, it proposes that the difficulty of providing a "golden" edit increases as we move forward in time. If that's a problem (debatable, since it means wikipedia as a product is better than it was previously) then it strikes me as a difficult one to solve.

<comment><USER16> Those were basically my thoughts before reading this article. However, while it might be that its more difficult to create a good edit, that still shouldn't lead to more of the good edits that are made being reverted as the data shows is happening.

</comment></comment><comment><USER17> I was surprised to see that "Assume good faith" was a principle of Wikipedia - it doesn't match my experience with the community at all.

</comment><comment><USER18> There is a set of knowledge in the world. It is vast, but much of it has already been contributed to wikipedia. The knowledge of the world continues to grow, but I don't think at the rate that it is contributed to wikipedia.

Contributors are really mining the collective knowledge of the world, and so they are running out. So similar to mining an nonrenewable resource, the cost of extraction goes up with time: [http://desmond.imageshack.us/Himg521/scaled.php?server=521&#...</a><p>With difficulty in making significant contributions dropping off, you would expect to see a drop off of new users (they really don't get rewarded enough for their efforts). Back in the day, you could contribute something meaningful that you know to wikipedia. Most likely now, someone has already put it in.

</comment><comment><USER12> Thanks wikimedia for once again providing a detailed analysis of something that's been absurdly obvious for the last 5 or 6 years - "However, the rate of rejection of all good-faith new editors' first contributions has been rising steadily, and, accordingly, retention rates have fallen."

And so the conclusion? "At the Foundation level, this includes major software changes like the creation of a visual editor to lower the technical barrier to entry..."

Which completely doesn't address the human issue of entrenched editors who reject perfectly good edits by new users and delete perfectly good content. _Nothing_ about that problem is addressed by making the editing GUI simpler.

Useless.

</comment><comment><USER19> What a nice bit of research and summary presentation.

</comment><comment> cnspcs-cmplr Oh, the good old days. I went and inserted some Irish Evil into Wikipedia in commemoration.</comment>


Digging Deeper into Vivian Maier's Past - <AUTHOR> http://lens.blogs.nytimes.com/2016/01/12/digging-deeper-into-vivian-maiers-past/ <comment><USER2> Bad link. It's a just a content-free teaser for the story coming up tomorrow there, but the full story is written here: [https://vivianmaierbio.wordpress.com/](https://vivianmaierbio.wordpress.com/) if anyone cares about her bad childhood.</comment>


Would You Bet $100M on Your Pet Programming Language? (2007) - <AUTHOR> http://prog21.dadgum.com/13.html? <comment><USER3> Caveat before reading my comment: this article is from 2007 and the scene 8 years later is pretty different but the spirit still holds, I think. The author also specifically placed "pet programming language" in the article's title which makes my reaction to it a bit unsound since, for example, Haskell is not a "pet programming language".

WhatsApp built a 19B product on-top of Erlang. I'm building our startup's product on a mix of Haskell and Erlang; so far it's going well.

There's pessimism in this article that neglects to acknowledge adaptability. I agree _to a degree_ with the thought that if I chose to build our product on- top of Idris I would be spending more time building tooling and dealing with immature parts of the language than I would actually building the product.

This is not the case with Haskell and other innovative but mature languages / platforms. Once at a certain size we may need to train programmers or develop some tooling internally but I see that happening even with Java and C++.

If you can boot up the product with the language / platform effectively and quickly, it's good enough to tackle a 100M product and adaptability can take care of the long-tail.

<comment><USER4> It's funny you mention WhatsApp and Erlang, because the article actually addresses that point:

> You're more dependent on the decisions made by the language implementers > than you think ... some odd case that didn't matter at all for the problem > domain the language was created for

The language implementers (employee of Ericsson) made design decisions based on telecom constraints, exactly the same constraints faced by WhatsApp and other messaging systems. In the parlance of the article, WA is in Erlang's problem domain, so a strict interpretation of the author's words would have him agreeing with WhatsApp here

<comment><USER5> Exactly, it is silly to consider Erlang a "pet language", it is an industry hardened language designed for the a near cousin to problem domain of WhatsApp. 20+ years routing billions of calls a day under soft real-time, high reliability requirements... because you simply expect your phone to work, even phone calls from one end of the world to the other.

</comment></comment></comment><comment><USER6> Massive jobs like this tend to require a lot of inter-team dependency. The best language is also one that everyone on the current team understands, and that everyone supporting it will understand. And that will grow in the direction of such a massive project.

My 2 cents... Companies have to selectively pick where they are on the bleeding edge. Extra large projects aren't the best spot to play with the bleeding edge of new programming languages. (This is why COBOL stuck on so long for billing systems)

The place for experimenting with new technologies are smaller self-contained projects where the risk is less, and the benefit can be more clearly explained.

</comment><comment><USER7> A hundred million dollars would change my decisions, but it also changes the nature of the problem I'll be working on. The contracts I get are on a much smaller scale which means the problem I'm solving is also of a different scale. The language I chose is based on the fact that my client wants their project done on time and on budget along with the ability to change direction when new requests come in.

If my client wants something as robust as a hundred million dollar budget; they will need a 100 million dollar budget. More often then not they just want something that works.

</comment><comment><USER8> (Note: This article, while good, is from 2007 so perhaps its advice is slightly out of date.)

Personally, if I were going in completely blind I'd choose Python (with C as a backup). Numpy and Scipy are fast enough for most of the things that you need to do. It's the "it does pretty much everything" language. Is it great at very high performance games? No. But you can drop into C pretty easily (although I find Ruby + C to be easier) and most of the code you'd be writing isn't going to be core game logic.

Now as to why I'm not coding it now: Because I know what I'm building: Dynamic web apps. Ember (with EmberUI) + a pure Rails JSON backend (with xdomain) will get me there much faster than the needlessly verbose Python. It will also let me hire people that won't have to figure out my stack. It is simpler and less risky. I'd never write this in C.

<comment><USER9> What projects on the scale/complexity of Photoshop are implemented in Python? Possibly large chunks of Youtube or Dropbox?

I love Python for small scripts, and even don't regret it too much for programs with mild complexity (~5000 lines of Python), but the lack of explicit typing makes figuring out what any given function/class does pretty hard. What are the arguments? What do they do? What does the return type do? Where is this thing initialized? Finding the answers to those kinds of questions are greatly aided by an IDE/source code browsing that is anchored by types.

<comment><USER10> Yelp is a multi MLOC Python code base. AFAIK YouTube is slowly being ported from Python to Go.

There are plenty of other large codebases in dynamic languages:

      - Wikipedia and Facebook written in PHP
      - Twitter was originally written in RoR

To be honest, it's my personal opinion that good static languages (i.e. type inferenced) are better than dynamic languages. I believe it's situational compared to traditional static languages (e.g. Java, C++).

<comment><AUTHOR> Both twitter and facebook ran into some pretty hard limits on those platforms which in the case of twitter led them to abandon RoR and in the case of Facebook led them to create a compiler for the language (doable on that 100M budget but still quite an undertaking).

</comment></comment></comment></comment><comment><USER11> I have trouble accepting the general premise of the question. Not the $100m part... even a $100k job or a $10k job... doesn't matter. It implies that the program being developed has a definitive "done" moment. But software (at least software in constant use) doesn't really get finished. So you build, and you expand, and you rebuild, and new things happen that change your approach and new tech gets developed and your software changes. I can (technically) start coding in Language X and ultimately port the entire codebase to Language Y (or take bits of Language A, B, and C). It's not so much about choosing the right language as reacting to the ever-changing world in which your software is being deployed.

</comment><comment><USER12> I feel like the corollary to this is how quickly one could solve the problems in a pet programming language if $100M was on the line. If the compiler doesn't scale well, hire one of the authors for a bit as a consultant. C with Classes was once someone's pet programming language.

<comment><USER13> ... and if the baby isn't born in one month, hire nine women! A lot of the development of mature software can't be parallelized. Paying Simon Peyton- Jones twice as much, or even cloning him, won't make Haskell improve twice as fast.

<comment><USER14> Peyton-Jones is not the only person working on Haskell. In fact, if it were _only_ Peyton-Jones working on Haskell, the language wouldn't be where it is today.

Yes, the speed of development probably almost always scales sublinearly with the number of people working on it, but beware that perception may also be distorted by the old adage that "the last 20% of the work take 80% of the time". That is, when people are added to a project that has finished the first 80%, it may _seem_ like progress scales terribly sublinearly with the people added, but what is observed may simply be the fact that the same amount of work leads to a smaller perceptible change once a project has sufficiently advanced.

</comment></comment></comment><comment><USER15> A project this huge is likely to be a distributed system. I can't think of any single binary that could provide $100m in value to one customer. Maybe some really important financial database, but even then...distributed? Right? That being said, we're going to have to use way more than one language, right? Like most projects, right?

No. This question is a red-herring. I think a semi-decent point is being attempted, but I think most Software Engineers ARE good at deciding when it's appropriate to use the right tool.

Quick example: I'm a fan of NodeJS, but I would never use it to try to solve a computation heavy problem, it's good at IO multiplexing, but very little should happen in between connections. My experience has been that _most_ of the Node community is aware of this.

<comment><USER16> >I'm a fan of NodeJS, but I would never use it to try to solve a computation heavy problem, it's good at IO multiplexing

No it isn't. It is very bad at it. It just uses the most primitive event loop and foists all the complexity of that onto you as the developer using it.

<comment><USER3> You're being downvoted but you're right.

<comment><USER17> He is, and I'm seeing more of this on HN over time. Many people who use <technology x> see a bothersome comment, and while snide- he was accurate. People may use Node.js, drank the Kool-Aid, but in 5 years there's going to be an industry movement off of it and the messes being created today. I've used it, was not impressed by its technical merits, and wrote it off as yet more technological churn.

I'm not a fan of churn, and keep a keen eye eye out for true innovation. Which happens far less than people are convinced to believe. That's the biggest scam the tech world convinced everyone, that innovation is rampant and fast moving, when in reality everything moves at glacial pace.

I'm a late adopter of technology, proud of it because it's generally the smart move for most of us. I test drive shiny things I can make time for, but bringing it into my stack doesn't happen by reading a few blogs. It not only requires significant technical merit, but someone has to maintain all this shit.

</comment></comment></comment></comment><comment><USER18> The need for reliable and extensible libraries is a huge one, and that is largely why I am a huge fan of F#. Not that .NET is some silver bullet to solve all your problems, but one could use F# to develop a $100,000,000 system largely because it shares the same framework C# does.

<comment><USER19> I don't know why people always say that. How many full-stack web frameworks are there that use idiomatic F#? Haskell has at least three that were mature three years ago and still under active development. In general the .NET open- source ecosystem is weak; enterprise just waits for Microsoft to reveal the One True Way or spends thousands per seat on clunky tool-kits. That same enterprise will use hundreds of open-source Java libraries. Maybe its changed in the last few years and certainly there is enough there to work with, but I don't think it can rival Python or the JVM and I don't think F# has any edge over Haskell at all when it comes to libraries and community. You get a large, solid core to start with which is more coherent than the de facto Haskell standards but as soon as you start looking for nice idiomatic test frameworks, distributed application frameworks, embedded databases, FRP libraries or many other examples Haskell really leaves it far behind.

<comment><USER20> Which Haskell libraries interoperate with classical enterprise stacks, alongside existing code?

<comment><USER19> There are lots of libraries for building and consuming web services or using messaging products, which is the same as how any heterogeneous environment is integrated. Big companies integrate .NET, Python and Java like this all the time. There ARE also tools and libraries [1] for generating all the JNI code necessary to call directly into JVM libraries but I don't think it has seen much adoption. I'm not really arguing that Haskell is suitable for enterprise applications - it is not. For greenfield SAAS product companies though I think its as viable as F#.

[1] [http://hackage.haskell.org/package/java- bridge](http://hackage.haskell.org/package/java-bridge)

</comment></comment></comment></comment><comment><USER21> Well healthcare.gov went with Java [0], but that was about 20x this budget [1], so I think I'll have to go with Forth.

[0] - [http://www.randalolson.com/2014/05/22/programming- language-b...](http://www.randalolson.com/2014/05/22/programming-language- breakdown-for-the-healthcare-gov-website/)

[1] - [http://www.bloomberg.com/news/2014-09-24/obamacare- website-c...](http://www.bloomberg.com/news/2014-09-24/obamacare-website- costs-exceed-2-billion-study-finds.html)

</comment><comment><USER22> I was going to comment on a different article from the same author, but decided this one was better:

[http://prog21.dadgum.com/57.html](http://prog21.dadgum.com/57.html)

</comment><comment><USER23> >What would you do? And if a hundred million dollars changes your approach to getting things done in a quick and reliable fashion, then why isn't it your standard approach?

Ordinarily you try to strike a balance between "exploration and exploitation". For a small project where success isn't that hugely important, the learning that a less conservative choice offers can be more heavily weighted.

</comment><comment><USER24> Here's one idea: why don't we compile every new language into a pretty C code? This way we will be able to use every available C library with a compiler of our choice and any extra C code, necessary for our real world application.

<comment><USER25> Because if you use C libraries you will inherit their problems (and also their interface is rarely idiomatic in the new language). Usually if you want to use a different language it's because you think that language has advantages over C. E.g. a large part of the point of using OCaml is to avoid the safety problems of C, but that only applies if your libraries aren't written in C. See [https://github.com/mirleft/ocaml-tls#why-a-new-tls- implement...](https://github.com/mirleft/ocaml-tls#why-a-new-tls- implementation)

(Almost all languages do offer _some_ "FFI" to call C. But in languages that don't naturally fit, there will often be significant overhead to e.g. aligning memory management so that the same memory doesn't get freed twice, as might happen when a garbage-collected language called a C library)

<comment><USER24> That's why you would compile it into C instead of having hand-written C when safety is important and for most part of your program it probably is. But sometimes you absolutely need to be able to write assembly and use existing unsafe C libraries.

<comment><USER25> If you absolutely need a C library, almost all languages give you a way to do that. But why hobble your compilation strategy for all programs just to support this occasional use case better? You may well want to target a runtime (JVM, .net, Javascript) for which no good C compiler exists. Even if you're solely interested in building native executables, compiling to C would mean throwing away a lot of information (e.g. knowing which values are immutable) that an optimizer could use. Look at Haskell; performance-optimized Haskell performs a lot better than machine-generated C ever would (indeed, it often comes close to the performance of hand-tuned C).

(If you're worried about duplicating effort in optimizers, do what many languages do and offer an LLVM frontend. LLVM optimizations are (mostly) language-independent, and LLVM bytecode, while imperfect, often allows a better representation of language semantics than C source would)

<comment><USER24> I don't think it is so rare, that you could call it occasional. That's the author's point and mine as well. Sometimes I need sqlite, other times libjpeg, giflib, libpng and some simple image processing with at least some low-level code. And it takes a lot of effort to make these things work with other languages on different platforms. Golang tried to make it easier and it still isn't.

Optimizers is also one thing I was worried about, LLVM is not the only compiler out there. And also there is friction. Most of the systems already have C compiler installed, if you target it, there is no friction in setting up building environment and so on.

<comment><USER25> > Golang tried to make it easier and it still isn't.

How much of that is because Golang doesn't compile to C, and how much is because the languages just don't fit? E.g. you'd still need a way to mark whether Golang was supposed to garbage-collect a value that had been returned from a C call, or not.

> Most of the systems already have C compiler installed, if you target it, > there is no friction in setting up building environment and so on.

Yes there is, because you still need to build the thing that interprets to C. If anything there's _more_ friction than having a compiler that directly builds native binaries like e.g. ghc, because you need two tools - <USER2> translate-to-C tool and your C compiler. "Most systems already have a C compiler" is only really true on *nix; it will be easier for a Mac or Windows user to run your system if you distribute e.g. java bytecode.

</comment></comment></comment></comment></comment></comment><comment><USER17> If $100M were on the line you'd get a mess that met your requirements.

If I wanted good code, I'd take guys who worked on a project for free.

</comment><comment><USER26> The language is irrelevant at that price. Given an architect of sub-average skill they'll pick an acceptable one.

</comment><comment><USER27> Yes. There's risk with every technical decision. I'd probably be inclined to use Haskell, for access to a top-notch community and a level of robustness (esp. in the face of refactoring) that, while it can be achieved in a dynamic language, is hard to hit without a static type system.

An upshot of Haskell is that it probably wouldn't take a $100 million budget because you can do a lot with a small team. We're going to need far fewer people than on a Java project, and you won't need the layers of management that come with big teams, and you'll also have smaller code and more of an ability to use the Unix philosophy (systems compromised to multiple, usually small, programs). Those wins aren't additive, but multiplicative.

So, yes, I'd definitely trust Haskell (or any mature FP language) on a $100M project. I'd trust Java on one too-- if I could find someone to do all the work, and trusted that person-- but I'd rather use an FP language and really kick ass.

</comment><comment><USER25> I was nodding along until I realized that the author has a very different notion of esoteric from mine.

The last month of Fridays I've been tinkering with Idris, trying to get it to... well, do anything useful. I'd heard it was a dependently typed JVM language, which it is... kinda. Turns out you need to install a full Haskell toolchain to do anything with it. With the right esoteric option incantation, you can make it build an executable that turns out to be a shell script concatenated with a jar file. Nice. But you can't build a library with it, even within the library. You can't invoke it to build some classes to use later. You can't even call the build tool from anything remotely standard in JVM-land, and when I asked about making the compiler selfhosting the response was a kind of lukewarm "yeah, sometime". The current toolchain is in Haskell but there's not even an FFI from Idris into Haskell, so porting it would be... challenging. I tried to join the two together via their C FFI, but in between undocumented linker options and the fact that the Idris runtime is already linked to part of the Haskell runtime, I eventually gave up. If nothing else, it gave me a real appreciation for how much work went into making Scala a serious, commercially usable language, something I'd previously rather taken for granted.

But C, holy shit, C? You'd write a program in C? And expect it to work? I've seen programs go wrong in a lot of languages, but most of them can be eventually fixed. In C you get irreproducible voodoo random crashing that, sure, you can _usually_ track down with a static analyzer, valgrind, debugging, intelligence and luck. But what if you couldn't? What if the program was just broken, and you couldn't fix it? With $100,000,000 on the line, there is no way on earth I would risk letting C (or any other unmanaged language) anywhere near my codebase. It might mean more work, and less features, if I were to use e.g. that pure-OCaml SSL library, or a JVM-native multimedia library. But I'd do it in a heartbeat, all the same. "The final say on overall data sizes" is such a tiny, trivial concern compared to using a language where failure is understandable, reliably diagnosable.

(In fact I'd say the scenarios where a measly factor of 2x memory consumption makes the difference between "working" and "not working" are just vanishingly narrow. If you need to scale horizontally in OCaml, you're going to need to scale horizontally in C a couple of weeks later. Particularly with $100,000,000 on the line, sod it, buy a bigger server with more RAM if using 32GB rather than 64GB really makes the all-important difference).

Would I use a "pet language" I tinker with, like Idris? No, but I wouldn't use that for any kind of serious commercial work. With $100,000,000 on the line, I'd use the same language I use for almost all my work: Scala. And I would stay the hell away from JNI, because I don't want C anywhere near this system, lest it bring it all crashing down.

<comment><USER28> >> But C, holy shit, C? You'd write a program in C?

Most of the worlds software rides on the back of C. Pick a language, the runtime is probably written in C. Any high performance libraries are either in C or C++ (QT anyone?) and for numerics you may still find <gasp> Fortran. If performance is a requirement or running on a micro controller, you're going to have some C or you fail to get the best performance. That said, stuff like string processing sucks in C - some say that's why C++ was invented.

<comment><USER20> That is an historical accident of UNIX widespread into the industry.

I am old enough to remember C wasn't even an option unless the customer was willing to shell out money for UNIX systems.

</comment></comment></comment><comment><USER29> is the quality of HN commenters going down, perhaps?

1.) It's about strategy. re-read Sun Tze, Art of Strategy (and avoiding war). The title is amistranslation cleverly designed to fool non-native LANGUAGE cunning linguists. - english joke - cunn ing lin gui st - end joek.

2.) those who armed welll with the RIGHT language, did not need to fight or program well. that's why Haskell or Ocaml is at the 2nd stage.

3.) But it AINT COQ or provable...LANGUAGE.

4.) what's the second point U missed? Yes, the weak spot is "inter-team dependency" \- ref: mathattack post. Yes, the ??? is true and necessary.

BUT THE CONDIITION is NOT SUFFICIENT for AI or artificial intelligence.

8.) so what is the closest to ai. Seems to me prolog, but darn... most of the obscure conferences are in japanese....language barrier.

9.) We conclude the 'lesson of the day" by quote: nbardy: "client wants their project done on time and on budget along with the ability to change direction"

So, THE CLIENT wants it FREE, Yesterday (get the time machine working) and High quality and CHANGEABLE - agilie.

Yes, of course, as long as you pay up front, I THE VENDOR guarantees ALL FOUR ASPECTS in the 'game.'

No wonder, the track record of software is so BAD compared to engineering, building of the Pyramids in Egypt, farming using bio-dynamic techniques, other human endeavor - spacecraft.

addendum: fun note: just simply download basic OPEN SOURCE that includes libressl, openssl, gettext GNU, etc.

SOME OF IT DONT WORK, at least on basic systems.</comment>


PagerDuty: You Have The Power - <AUTHOR> http://blog.pagerduty.com/2012/09/you-have-the-power/

<comment><USER2> PagerDuty is a fantastic service. The flexibility in scheduling on-call rotations (they added a custom schedule for my team at my last company), the support for integrating with lots of different monitoring platforms and the generally fantastic support make it well worth the monthly cost.

</comment><comment><USER3> At Loggly, we integrated our alerting app with PagerDuty, and open-sourced the whole thing if anyone is interested:

<https://github.com/loggly/alertbirds-community-edition>

</comment><comment><USER4> We're lucky in that we have so many technically astute users, who've already built cool stuff in spite of us. I'm excited to see what they'll do now.

</comment><comment><USER5> Chiming in - PagerDuty has saved many butts at Airbnb, too. We love to hate getting pages from you guys.

</comment><comment><USER6> Datadog hearts PagerDuty. We use it every day. And sometimes nights too...

And here's its awesome API in action in Datadog -<http://www.datadoghq.com/2012/07/pagerduty/>

</comment><comment><USER7> TL;DR New Pagerduty API docs at: <http://developer.pagerduty.com/>

</comment><comment><USER8> Is PagerDuty worth it for $18 per month? For a 5 person team its $1080 for an entire year to do something that a small script + Google Calendar could take care of. Maybe I'm missing something? I'd love to hear a story about how a feature of PagerDuty saved somebody's butt.

<comment><USER9> > Is PagerDuty worth it for $18 per month?

Yes. We use them at Netflix.

The real value that they add is they take care of the edge cases you haven't thought of yet. :)

<comment><USER10> Sadly they do an embarrassingly poor job at it.

The text of their notification e-mails is ridiculously bad ("You have 1 triggered Incident", not customizable). The mails don't even contain direct links to the incidents, much less links to ack/resolve them...

There are no notifications about status updates at all, not even an e-mail. People still call each other all the time to synchronize about "Are you on it or should I take it?" and "Is it resolved yet?".

etc.

I'm still baffled how they managed to ruin such a trivial product so thoroughly.

Needless to say we went back to a simple shell-script with TSV schedule (and that amusingly works better in most regards).

<comment><USER11> Moe, at OpsGenie (<http://www.opsgenie.com>), we've attempted to solve this problem through the use of mobile apps (iPhone, Android and html5 app) on smartphones (and tablets). Apps allow us to present much richer information, with custom fields, tags, as well as attachments, and enable users to communicate, not only acknowledge or comment but also integrate with other management systems thru execution of custom actions. I'd love to have you give it a try.

Disclamer: I'm one of the founders of OpsGenie</comment></comment></comment></comment>


It's not that just MMORPGs changed, but the whole internet culture - <AUTHOR> https://np.reddit.com/r/MMORPG/comments/em62j1/its_not_that_just_mmorpgs_changed_but_the_whole/ <comment><USER2> Ironically posted on Reddit unironically.</comment>


BeagleBone Black released, 1GHz Cortex-A8, 512 ram, $45 - <AUTHOR> http://beagleboard.org/Products/BeagleBone%20Black

<comment><USER2> Very similar to $49 Cubieboard: <http://cubieboard.org/>

\- 1G ARM cortex-A8 processor, NEON, VFPv3, 256KB L2 cache

\- Mali400, OpenGL ES GPU

\- 512M/1GB DDR3 @480MHz

\- HDMI 1080p Output

\- 10/100M Ethernet

\- 4Gb Nand Flash

\- 2 USB Host, 1 micro SD slot, 1 SATA, 1 ir

\- 96 extend pin including I2C, SPI, RGB/LVDS, CSI/TS, FM-IN, ADC, CVBS, VGA, SPDIF-OUT, R-TP..

<comment><AUTHOR> Thanks for the info, the SATA in this makes me very interested.

</comment></comment><comment><USER3> This seems considerably more potent than the Raspberry Pi, especially with the real-time co-processors and proper analog in/out support.</comment>