False confessions - j-g-faustus http://www.economist.com/node/21525840 <comment><USER3> Unfortunately _not_ a lab test, I saw this on Frontline recently and it was very disturbing to see that confessions could be so easily had: [http://www.pbs.org/wgbh/pages/frontline/the- confessions/etc/...](http://www.pbs.org/wgbh/pages/frontline/the- confessions/etc/introduction.html)

Essentially, the story involves these sailors individually confessing to a crime (and allegedly tailoring their confessions to the investigator's words) when they were implicated by someone already being interrogated. With each additional person being interrogated, more people were implicated (who were previously unnamed), and previous confessions were amended accordingly.

Quite saddening indeed.

<comment><USER4> The Frontline story was very eerie. The other famous case is the West Memphis Three:

<http://en.wikipedia.org/wiki/West_Memphis_Three>

Who were jailed partly because of a police-coerced confession from one of the three. They are still in prison, but apparently close to getting a re-trial.

The case was made famous by the excellent HBO documentary 'Paradise Lost' - which is a must see:

[http://en.wikipedia.org/wiki/Paradise_Lost:_The_Child_Murder...](http://en.wikipedia.org/wiki/Paradise_Lost:_The_Child_Murders_at_Robin_Hood_Hills)

They are lucky because their case has a high-profile because of the film - <AUTHOR> there must be many, many other innocent people in prisons because of false confessions.

For example, many people believe that Cameron Todd Willingham, executed in Texas, was also innocent (he was also the subject of a Frontline story:

<http://video.pbs.org/video/1618590505/>

and a feature in the New Yorker:

[http://www.newyorker.com/reporting/2009/09/07/090907fa_fact_...](http://www.newyorker.com/reporting/2009/09/07/090907fa_fact_grann)

<comment><USER5> An interesting read is "Remembering Satan", about a small community torn apart by accusations of sexual abuse and satanic rituals. Many people confessed, some were convicted, no evidence was ever produced.

[http://www.amazon.com/Remembering-Satan-Tragic-Recovered- Mem...](http://www.amazon.com/Remembering-Satan-Tragic-Recovered- Memory/dp/0679755829/ref=sr_1_1?s=books&ie=UTF8&qid=1313411669&sr=1-1)

<comment><USER3> Wow, very creepy stories that both of you posted. Thanks for linking to them.

> They are lucky because their case has a high-profile because of the film - > but there must be many, many other innocent people in prisons because of > false confessions.

This is exactly what concerns me–so much effort and attention went into these few cases that achieved a high profile, but I'm certain that many just like it–with inconclusive or completely unmatched DNA and police hasty to close an investigation–exist, and it worries and saddens me to think how many other people are innocently sitting in prison or on death row as a result.

</comment></comment></comment></comment><comment><USER6> Rule #1: Don't talk to the police.

<http://www.youtube.com/watch?v=i8z7NC5sgik>

You're better off taking the beating.

<comment><USER7> Something I've never understood from one example from that talk: If it's your word in court vs. the police officer's regarding an unrecorded conversation, how exactly does _not talking_ help you? In the example (linked below), the police officer lied in court, saying that the defendant admitted to placing a hand on the plaintiff when the officer first approached the defendant. How does being silent help you in that example? The officer could still lie.

[http://www.youtube.com/watch?v=i8z7NC5sgik&feature=playe...](http://www.youtube.com/watch?v=i8z7NC5sgik&feature=player_detailpage#t=286s)

<comment><USER6> "Lie" in the eye of the beholder.

To the cop, stating that you touched someone means that you've opened the door -- it's a thread connecting your story to the plaintiff's. Since you are probably not routinely interrogated, your body language may reinforce that thought process as well.

<comment><USER7> In the example, the cop testifies that when he first approached the defendant, the defendant admitted to placing his hand on the plaintiff's throat, but only as a joke. The defendant claimed that he never said anything of the sort to the cop. It was the cop's and plaintiff's word against the defendant's word, and the defendant was found guilty. How would it have helped if the defendant had said nothing to the cop? The cop lied about him talking.

To me, the words of a cop and a plaintiff are "reasonably doubtful," but I guess not. Anyway, my point is that in this example, the cop lied about what the defendant said.

<comment><USER8> It's harder for the officer to lie and say that you said something when you refused to say anything.

It's easier to just "slip" something in a conversation if you're talking to him in the first place.

<comment><USER7> How so? If your interaction with the cop wasn't recorded, even if you started out by saying "I choose to remain silent" and never said another word, the cop can easily testify that you said something. I don't see how it's "easier" to change an actual conversation than it is to completely fabricate one.

<comment><USER8> > I don't see how it's "easier" to change an actual conversation than it is to > completely fabricate one.

It's easier because, when you fabricate an entire conversation rather than one sentence, you have to make up more things.

That's the purely technical argument. There are also psychological considerations on the part of the officer, people are more likely to rationalize making up one little thing than making up an entire conversation.

<comment><USER9> Yes, they can ask you all kinds of questions about what you said and make you appear quite talkative.

Say nothing and you can truthfully say repeatedly that you said nothing. You'll be able to consistently tell the truth on the stand without having to explain any of your statements, statements that the prosecution will attempt to twist into incriminating statements.

Of course, if you do the same to the prosecutor before trial, the chances of you making it to trial go down. Prosecutors like to win their cases.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> I think something like this would have a huge impact on the conviction rate in Japan where a significant amount is based in coerced confessions. See BBC article: <http://news.bbc.co.uk/2/hi/8290767.stm>

But this reminds me of the malleability of memory and also herd-like mentality. I think it was on Tv or somewhere where the experiment involved flooding a room with participants with fake smoke but confederates were set to deny it was smoke. The effect was that the fake smoke was ignored. Had it been a real fire, they'd all have perished.

</comment><comment><USER11> I used to think the whole false confession issue was bunk. I could not imagine confessing to a crime I did not commit for the obvious reason that there was simply no upside. However, this changed when someone I knew told me the story of how he found himself in a situation where he almost confessed to a crime. Here's the story

My friend, Bill, was in charge of construction projects at a real estate development. One night, he gets a phone call at 3 in the morning to let him know that one of the homes under construction is on fire. He jumps out of bed, grabs his camera and rushes out to scene of the fire. Once he gets there, he starts taking pictures in order to get a jump start on the insurance paperwork. Unfortunately, the firemen trying to put out the fire start to get suspicious. From their perspective, here's a strange guy taking pictures of burning homes at 3 in the morning (it's a development so there were no other occupied homes in the vicinity). They get to thinking maybe he is a pyromaniac who is now collecting trophy photos of his handiwork. They call the cops, Bill gets arrested and ends up in the interrogation room. Now, Bill is a pretty tough guy. Remember, he worked his way up in the construction business to become a manager so he is not a weakling by any means. However, after a couple hours of being grilled by the cops, he said he was just about ready to confess just to make it stop. They didn't beat him or anything, just the relentless application of all manner of psychological techniques (badgering, cajoling, threats of long sentences, good-bad cop, etc) completely wore him out. He was only saved when his boss found out that he had been arrested and called the station to back up his story.

From that time on, I realized that under the right circumstances, it is possible for a mentally stable, reasonable person to confess to a crime they did not commit.

More importantly, I realized that because I can't imagine something happening, does not mean it can not happen.

</comment><comment><USER12> I've always worried that if I ever became a suspect in a crime that I would confess under pressure even if I didn't do it. There is something about me feeling guilty, authority and intimidation that makes me think I might.

Then again, my coping mechanism of becoming flippant/sarcastic could flip on and I'd confess to murdering JFK, Michael Jackson and Biggie using my mind/time machine.

<comment><USER13> If that's the case, then the first words out of your mouth if you are ever interrogated are "I would like to exercise my right to remain silent." If you say that, then the police are legally required to stop the interrogation then and there. (Ironically, simply remaining silent is not sufficient: <http://en.wikipedia.org/wiki/Berghuis_v._Thompkins>)

<comment><USER14> Ironically but not unreasonably, since if silence were an indicator that you wished never to speak, people who wanted to talk would never reach that point.

(In the UK there is no such thing as the right to remain silent, unless you are in a situation in which you are denied access to legal advice (rare). In all other situations your solicitor will instruct you to say 'no comment' to any question the police asks you. Funny how it differs.)

<comment><USER15> > In all other situations your solicitor will instruct you to say 'no comment' > to any question the police asks you.

Isn't that effectively the same thing?

</comment></comment></comment></comment><comment><USER16> I always found it strange that when a guy is saying he was doing something else noone believes him but when he starts telling he was doing what people suspect him of suddenly his word is rock solid.

If someone is consistently upset with what you are saying and makes your life miserable you try behaving differently especially in such stressful situation.

It's useful to listen to people suspected of crimes but you should treat their "confessions" as a clues where to finds evidence of what actually happened.

<comment><USER17> Funnily enough, the Phoenix Wright series of games for the Nintendo DS, despite featuring an utterly ridiculous legal system, has this as a fairly central element. The basis of gameplay is cross-examining witnesses... and nearly every single one of them is either lying or misremembering things. I seem to recall at least one time where you were defending someone who had already confessed, for that matter.

The lying's not uncommon in TV dramas, but the unreliability of eyewitnesses accounts certainly is.

</comment></comment><comment><USER18> It'd be nice if [insert popular TV crime drama] would pick this up and run with it. We might start to see juries doubting confessions in the same way they've already been trained by the media to doubt non-forensic evidence.

<comment><USER15> The burden of proof has apparently been raised to ridiculous levels by CSI, according to some prosecutors. Juries believe that the sort of thing done on CSI is realistic, and they expect very definitive results and evidence, which usually doesn't happen.

<comment><USER19> And don't even start to look at court proceedings in Germany, where people are astonished that it's nothing like in the dubbed American shows they watch.

</comment></comment></comment><comment><USER20> I confess that I didn't read the article, but this reminds me of the HN story of the University professor who asked students who cheated to come in to his office, and then found some large percentage who came to the office though the software exonerated them.

The question is: did they cheat and get it past the software? Or were they, as suggested in this article, admitting to something they did not do?

<comment><USER21> They could easily have been intimidated by the prospect that they would be wrongfully accused and went to his office under the implied premise of amnesty. The game as visible to the student is something like an extra essay assignment weighed against possibly having to fight expulsion.

I suspect that there are similar factors involved in other false confessions. It's an asymmetric knowledge game. The only winning move for the person being questioned is not to play, but they may not be aware of that - and worse, they may trust the other player. With further intimidation, privation, and humiliation, it's not hard to imagine why people could break down.

</comment></comment><comment><USER22> When I was reading the article, I was wondering when they would get to the role of interrogations especially aggressive ones. I cannot find the link now but there was a study which subjected people to sustained questioning that went on for more than 3 hours without a break. After 3(?) hrs all the subjects were offered the chance to end the interview if they agreed to deposit some non-trivial amount of money for 60 days. Something like 95% of people agreed and even signed (fake but real looking) legal contracts to that effect - <USER2> tough most were students (not rich) recruited for the study.

Never underestimate the stress caused when in a room with people aggressively questioning you. Even if they do no physical harm, This is VERY stressful.

In the US - "Am I free to go?" and "I want a lawyer" should be at the top of your 2 item list when in this situation. Remember, the Police care about solving crimes certainly - they care more about clearing cases quickly.

<comment><USER23> Three hours? That's nothing for Canadian interrogations.

Here's a 2008 case (from the highest court in a major Canadian province) I found pretty quickly that's about a four and a half hour interrogation where the police lie about having DNA evidence that pins the crime on the person: [http://canlii.org/eliisa/highlight.do?text=%22The+interview+...](http://canlii.org/eliisa/highlight.do?text=%22The+interview+continued+for+approximately+four+and+one- half+hours%22&language=en&searchTitle=Search+all+CanLII+Databases&path=/en/bc/bcca/doc/2008/2008bcca127/2008bcca127.html)

Here's a 2000 Supreme Court of Canada case about an approximately eight hour interrogation (at which point the guy confessed): [http://www.canlii.org/en/ca/scc/doc/2000/2000scc38/2000scc38...](http://www.canlii.org/en/ca/scc/doc/2000/2000scc38/2000scc38.html). There are plenty more of these. There's no hard and fast limit to how long Canadian police can interrogate you for.

</comment></comment><comment><USER24> The Confessions on Frontline is one of most interesting reports from that show that I've seen, it's related to four men who confessed to a crime they didn't commit. Highly recommended.

<http://www.pbs.org/wgbh/pages/frontline/the-confessions/>

</comment><comment><USER25> While the experiment with the ALT key was interesting and can shed some light on this subject, it would be a mistake to extrapolate too much. I would be far more likely to assume "I probably did accidentally hit the ALT key" than "Oh, I probably did murder that person." Hitting the ALT key by accident is extremely easy, and the punishment in this study small relative to most criminal offenses.

</comment><comment><USER26> When an investigator tells someone that they have two choices: confess and face potentially five years in jail, or plead not guilty and face potentially twenty years in jail, what sounds like the best deal?

</comment><comment><USER27> Yea, I am well aware of the advice to not talk to the police. But I wonder why the police would go to such trouble to get confessions as to write a letter to trick people into doing so etc.

<comment><USER28> Police are generally under a lot of pressure to close cases. Getting a confession closes a case quickly and neatly.

Watch The Wire to get an idea of what kind of pressures the police face to "solve" crimes.

<comment><USER7> Also, police have power. Power has a profound impact on most people, and generally not a good one.

<comment><USER29> I think Frank Herbert had a point when he wrote that it isn't so much that having power corrupts, as the sort of people who seek power in the first place.

<comment><USER30> The experimental record disagrees with Frank Herbert: power corrupts.

It was worth doing the experiments though, so good on him.

</comment></comment></comment></comment></comment><comment><USER9> Peer pressure is a powerful thing.</comment>


Cancer patient with mastectomy denied clearance by TSA without invasive search - <AUTHOR> http://loridorn.me/post/10866768010/at-what-point-does-the-need-for-security-eclipse

<comment><USER2> Thanks for the link. I am only happy to have accepted the job offer from Europe and to have moved here instead of staying in the States. What happened to this woman is a shame, not for the agents, not for the government, but for the society... where were the other passengers? Nobody had nothing to say??</comment>


Show HN: Wishberg: Born in Mumbai. Launched in Bangalore. Made for the World. - <AUTHOR>

We've opened Wishberg (www.wishberg.com) in public beta today. Currently allowing users to share/discover wishes, reviews and experiences in their social graph.
Aim is to build social graph for product and purchase intents. Localized information for 200+ cities across 12 countries.
Appreciate reviews / feedback and suggestions. Please share on this forum or drop a note on pj(at)beingpractical.com <comment><USER2> Please change the "Wanna" - it just grates. There is nothing wrong with "Want to ....." maybe even just "Have" and "Want" buttons.

<comment><AUTHOR> Thanks for this. I agree, will get simple text.

</comment></comment><comment><USER3> Nice design. I wish that you focused on one thing - travel, cars, whatever. By doing everything, I'm not sure what to do.

It also feels a little ... lonely? It'd be better if you had some hint of user activity on here.

I was hesitant to sign up fearing that you'd spam my facebook wall, without more context.

Good luck!

<comment><AUTHOR> Thanks. Valid feedback. Yes - we're missing that bit when a user joins with email address and without friends / followers. Will introduce that soon.

Personally, I hate spam as much as you do. We have enabled sharing on Facebook / Twitter only with a context. Almost every place with explicit user permission. There are many annoying apps, we don't want to get there.

</comment></comment><comment><USER4> clickable <http://www.wishberg.com>

<comment><AUTHOR> Thanks :)</comment></comment>


The Truth on OpenGL Driver Quality - <AUTHOR> http://richg42.blogspot.com/2014/05/

<comment><USER2> [https://hn.algolia.com/?q=opengl+driver+quality#!/story/fore...](https://hn.algolia.com/?q=opengl+driver+quality#!/story/forever/0/opengl%20driver%20quality)

<comment><AUTHOR> Ugh, I swear I searched for that numerous times with various search strings, having sworn I had seen the article on here before, and only the submission with 4 points came up. Maybe I'm crazy, but I seem to have this problem fairly often with the new hn search.

That aside, there's nothing wrong with resubmitting an interesting article after a while, right?

<comment><USER2> Correct, but in HN's case "a while" means "about a year".

[https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html)

Re HN search, I usually have to select "date" instead of "forever" in order to tell what I'm looking at. Maybe that would have helped in this case.</comment></comment></comment>


When growth hacks go unethical: Plagiarizing Youtube news bots have arrived - <AUTHOR> http://snapzu.com/geoleo/when-growth-hacks-go-unethical-plagiarizing-youtube-news-bots-have-arrived

<comment><USER2> Not unethical.

The internet's a gangster's paradise. The best companies have mobsters in their ranks.

<comment><USER2> k. unethical. It's the reality of the internet.</comment></comment>


Apple demands 50% rent cuts for UK stores despite record sales - <AUTHOR> https://www.thetimes.co.uk/article/apple-demands-50-rent-cuts-for-uk-stores-k7klxmj8h <comment><USER2> How many of those sales were from the (closed) stores and how many were from "apple.com/uk"?

<comment><USER3> I think the point is that this is not a demand made out of necessity but out of opportunity by exploiting the current dire situation landlords are in.

The key is whether this is a bluff.

I note that the article says that these demands have only been made regarding specific stores, not all of them. So the question is whether Apple is seriously considering closing down stores, and how these stores are structured (i.e. whether they are separate subsidiaries), because I suppose that the effective threat carries weight only if Apple can close down subsidiaries and if they are willing to do so.</comment></comment>


Multiple Bugs in OpenBSD Kernel - <AUTHOR> http://marc.info/?l=oss-security&m=146853062403622&w=2 <comment><USER2> It sounds like one local privilege escalation (possibly?) and a series of crashers?

Honestly walking away with those being the highest severity bugs is a credit to the OpenBSD team and their focus on security. They're totally bugs and it sounds like they're getting fixed immediately, but... many kernels fix these types of things all the time and don't even consider them security bugs.

<comment><USER3> Those bugs come from syscall fuzzing. Local privilege escalation is the worst thing you can get from it, so the results are as bad as they can be (apart from finding multiple privilege escalations that is).

<comment><USER2> I see your point, but it doesn't happen to be true, a bug in recv, send or other networking syscalls certainly could be much worse than a local priv escalation.

And seriously, only one local priv escalation and some crashers is still a pretty light haul for a good fuzzing session from a competent team. The sky is not falling for OpenBSD today.

<comment><USER4> I'm not sure I follow. Certainly, bugs in the TCP/IP subsystem can be worse than syscall bugs. But what's the bug in recv(2) that is going to be worse than that?

I don't think anyone thinks the sky is falling. To me what's interesting about this is the approach Jesse and Tim took, and how quickly it generated very straightforward panics.

<comment><USER2> If there was a systematic bug in receive or send where a certain string of data corrupted a buffer allowing for code execution, exploiting it remotely would be simple if there's any code on the machine using that syscall.

I'm not saying this is a likely bug, or even something vaguely reasonable, but it could totally be found with local syscall fuzzing, so local priv escalation isn't the worst bug you can uncover.

I'm mostly being pedantic here.

</comment></comment></comment></comment></comment><comment><USER5> And for those not aware, Project Triforce, is NCC's effort to run the wonderful fuzzer American Fuzzy Lop, on everything:

[https://www.nccgroup.trust/us/about-us/newsroom-and- events/b...](https://www.nccgroup.trust/us/about-us/newsroom-and- events/blog/2016/june/project-triforce-run-afl-on-everything/)

</comment><comment><USER6> I wish someone would release something like Fuzzing At Home. I've got computer power to throw at it, but I don't really have the expertise to do the setup work...

<comment><USER7> The last time I tried to compile an executable using all the afl fuzzer magic- compile-time stuff, I gave up, so I have the impression that CPU time isn't the bottleneck here.

<comment><USER8> It took me a day to understand the tooling required to get AFL working. Now I can spin up a new test case for a library within a couple hours. Once you have the test case, then it is CPU bound. I had on one series of tests, of the baresip sip library libre, running across 4 machines(24 cores each) for a day before it was 100% sure it had tested every code path looking for a SIP protocol decoding error through fuzzing.

<comment><USER9> How did you split up the work across the machines?

<comment><USER10> AFL has documentation on how to setup that up. eg:

      https://github.com/mcarpenter/afl/blob/master/docs/parallel_fuzzing.txt

</comment></comment></comment></comment></comment><comment><USER11> It's striking how many of these issues cause panics because of assertions that were already in the code. Without good assertion use, I would assume that many of these would have been much worse.

</comment><comment><USER12> More readable archived copy: [http://permalink.gmane.org/gmane.comp.security.oss.general/1...](http://permalink.gmane.org/gmane.comp.security.oss.general/19946)

(I'm not a fan of gmane, but it did a better job with this particular mail than the alternatives.)

<comment><USER13> The gmane interface is so bad it beggars belief.

<comment><USER14> Heh -- it kind of makes sense since it's really meant to be used via NNTP. (That's the whole raison d'être for GMANE.)

<comment><USER13> Yeah, I get that, but the HTML interface is so awful I think we'd all be better off if we just used Gnus to read it.

</comment></comment></comment></comment><comment><USER15> Ted U. announced that usermount will be removed in OpenBSD 6.0

[https://marc.info/?l=openbsd- announce&m=146854517406640&w=2](https://marc.info/?l=openbsd- announce&m=146854517406640&w=2)

<comment><USER16> So I wonder what the approach will be to allow users to mount USB flash drives?

<comment><USER17> doas(1), sudo(8), hotplugd(8), ...

</comment></comment></comment><comment><USER18> Firstly glad to see these reported and fixed.

Secondly how many of these were remotely exploitable? Yes OpenBSD is limited in it's exposure with the "base system", but it seems like few of these pose as "holes" for the system? Arguably pledge(2) could factor into this, maybe? I'll let someone better qualified comment.

Again glad to see these fixed. But is the baseline free user access to the whole system for NetSec/ OpSec these days? I don't know maybe it is.

I'm just reluctant to have to read through the HN, "OMG OpenBSD had CVEs" and "C is insecure". Arguably the later has some merit but C isn't going away anytime soon, for better or for worse.

<comment><USER19> Direct remote privilege escalation exploits in kernels are relatively rare.

The usual attack path is exploiting an application bug for local user access, followed by a local privilege escalation.

</comment></comment><comment><USER20> While a bit surprising to see so many at once in OBSD kudos to the team for the rapid response and to those who found the bugs for their responsible disclosure.

</comment><comment><USER21> Fuzzing is not something I have looked at seriously (to be honest it seems like asking clients to take up running before walking) but the outcomes are ... Impressive.

<comment><USER22> How come? What better way to test your function then giving them random/unexpected input and see how that works out.

Would you not want to try out what happens if I pulled a door you built that is only meant for pushing? There will always be that one person who ignores the sign and tries pulling just like there is always that person who uses your function in an unexpected way.

<comment><USER7> The issue I have with tests, is that I feel like I'm just implementing my algorithm a second time, in a different wording (language/domain), hoping that this will catch some unknown bug(s).

If a test suite covers all aspects of an implementation, it's basically a second implementation, and then the question becomes whether the time spent writing this test code would actually have been better spent looking over the actual code that defines your algorithm. If we can't detect errors in code by reading it, I don't think tests will help us much.

Furthermore: if our algorithms require a second "test implementation", shouldn't we also write a test for the test, to confirm that the written test actually tests correctly? What good is a broken test? If you've built a plywood door, and a machine for testing that door, wouldn't it be useful to have a machine that tests that your testing machine actually tests the door properly?

All jokes aside, I really enjoy automated, high-level tests, which mimic actual user behaviour. I think this is different because we're no longer implementing an algorithm a second time around (as a test), but rather encoding user behaviour (of the algorithm) into an algorithm (new code).

<comment><USER23> > If a test suite covers all aspects of an implementation, it's basically a > second implementation, and then the question becomes whether the time spent > writing this test code would actually have been better spent looking over > the actual code that defines your algorithm. If we can't detect errors in > code by reading it, I don't think tests will help us much.

The OpenBSD code has been audited many times by very experienced and careful C experts. Fuzzing apparently found the very few things they hadn't already found by reading the code.

<comment><USER7> > The OpenBSD code has been audited many times by very experienced and careful > C experts. Fuzzing apparently found the very few things they hadn't already > found by reading the code.

That's definitely comforting, but I guess my point is that the fact that bugs can even hide in plain sight in the first place is the root cause of the problem. If we can look at a specification of what something is supposed to do - the code - and not see that it does something entirely different, then the problem is with the programming/specification language.

The very purpose of a programming language is to make the specification of computer programs readable (rather than reading assembly). If a specification language needs a a computer program to go through the specification and see if it specifies things that we didn't intend to, then that specification language misses the very point of being a language in which to specify things in the first place.

<comment><USER24> I don't think it's that simple. A lot of bugs are the result of disparate pieces of code -- each of which is fairly easy to verify by reading it -- interacting in an unforeseen way.

<comment><USER7> I would argue that this effect is not a product of code but of global state. If you have a pure function that transforms some input to an output, all you can do wrong is use the wrong tool for the wrong job, which the type system should, preferably, prevent you from doing. Using the wrong function for the wrong job means we haven't restricted well enough the input the function takes, through clearly defining the types of the arguments, not allowing for meaningless invariants.

If a specification language requires the writer/reader to be aware of the entire specification in order to verify whether an isolated piece of specification is valid, that specification language is not of much value, I would argue.

It occurs to me that the problem with traditional languages is that they do not allow the direct reference to information (values); only to variables which contain values (information). In Haskell, if you want a place to store information you create a TVar (variable), while you are forced to do this, always, in traditional languages, if you want to work with information/values properly. You're forced to keep information in a register if you want to manipulate it, and always refer to the register, when the information it contains is what you're really concerned with. Having to both think about the values/information _and_ where you stored this information is double work. Why not reference the information directly?

<comment><USER24> I agree that many bugs occur due to state. I disagree that languages, particularly imperative languages (which you seem to be attacking here), are significantly responsible for bugs.

Syntax is not really an issue for any programmer with a reasonable amount of experience. I very rarely have trouble accurately transcribing ideas to code. More often, my ideas are faulty.

Physics engines in games are a perfect example. They're a perfect use case for functional programming, there's not a lot of complex state ideally, and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands of YouTube videos of physics glitches in games.

<comment><USER7>
        > Syntax is not really an issue for any programmer with a reasonable amount of experience.

        > They're a perfect use case for functional programming, there's not a lot of complex state ideally,
        > and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands
        > of YouTube videos of physics glitches in games.

I'm not sure I'm following. Are you arguing that because physics engines are a perfect use case for functional programming, that they are implemented as pure functions? I don't know of any physics engines that are implemented this way, hence the buggyness.

If an implementations of 3D scene rasterization contains glitches, the language used is most definitely the issue, because producing glitch-free 3D animations is a solved problem. No programmer writes a physics engine to produce glitches, so if they magically appear, even though they aren't visible in the spec, the spec language is faulty.

I guess I'm arguing that the problem is that most popular languages allow you to write programs/specs that are invalid (will crash when executed). So we're writing specifications in languages where we can't even say whether the specification is valid or not (actually implementable). If humans produce buggy programs, that is proof that we're using the wrong tool, I'm arguing, because no one intends to produce bugs.

Haskell is a bit scary, at least it was for me in the beginning, because it can seem so hard just to get your program to compile. But that's because the compiler actually verifies that your program is valid - that no unhandled exception will occur. If people are having writing valid, purely functional programs, perhaps this is, in part, because their idea is unimplementable, and Haskell is telling them this via not being able to compile your spec, while other languages inform you of this via an unhandled exception that crops up a year later, after the code is in production.

<comment><USER24> > No programmer writes a physics engine to produce glitches, so if they > magically appear, even though they aren't visible in the spec, the spec > language is faulty.

I'm saying that popular physics engines generally have bulletproof, tried-and- tested code, and are even implemented in a very functional way on a conceptual level. Despite this, they exhibit glitches. The glitches don't appear magically out of the spec language, they are an inevitable result of the discrete nature of realtime physics simulations.

This is meant to be a counter-example to your assertion that bugs come from miscommunication between computers and humans. My argument is that more often than not, we communicate our ideas perfectly, but our ideas are flawed. In the case of physics engines, they are flawed by design in order to compromise accuracy for performance.

<comment><USER7>
        > In the case of physics engines, they are flawed by
        > design in order to compromise accuracy for performance.

This is an odd definition of "flaw" to me. I would call it a design choice, compromising accuracy over speed, which we're forced to do always, since we don't have infinite compute power. Would you call H.264 video and MP3 audio flawed by design because they prioritize bandwidth reduction over lossless reproduction?

        > The glitches don't appear magically out of the spec language,
        > they are an inevitable result of the discrete nature of
        > realtime physics simulations.

You seem to be arguing both that glitches in physics engines are inevitable, and that they're a design choice (sacrificing precision over speed).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> Compared to other OSes and distros this is one in a million.

</comment><comment><USER26> Multiple (!) security issues in OpenBSD (!).

OK.

We're doomed.

<comment><USER27> At least these (seem to) result in a kernel panic instead of privileged escalation or reading unauthorized memory.

But in a world where even djbdns had a (minor) security issue, I don't think it's reasonable to expect any non-trivial software—particularly software written in C—to be fully secure.

<comment><USER16> I found it interesting that most of the patches are one or two line changes. Shows how one tiny oversight can open up a DoS or security vulnerablity.</comment></comment></comment>


Doom engine code review - <AUTHOR> http://fabiensanglard.net/doomIphone/doomClassicRenderer.php

<comment><USER2> It's amazing that this would run halfway well on a 33 MHz 486. Doom had a 35 fps cap, and ran at 320x240 (square pixels):

2.7 million pixels per second at 35 fps (the cap).

1.4 million pixels per second at 18 fps (~50% of cap).

At the more realistic target of 18 fps, you have 24 clock cycles per pixel. A 486 averaged about 0.8 instructions per clock, so you're looking at 19 instructions per pixel. With a 33 MHz memory bus and the DRAM of the day, you're looking at about 5 clocks for memory latency. That looks like an upper bound of no more than 4 memory operations per pixel.

A convincing 3d renderer averaging 19 instructions and 4 memory operations per pixel. And we're not even counting blit/video delays here. Good lord is that savage optimization work. Carmack is famous for a reason.

P.S. The _really_ scary thought is that Doom would hypothetically run on any 386 machine -- can you imagine painting e.g. 160x120 on a cacheless 20 MHz 386 laptop?

<comment><USER3> If I recall correctly, you had to shrink the view port on lower-end 486s for playable framerates. Also, the status bar shrank the area to be rendered to some extent.

Your point still stands, just pointing out that the target area for rendering was often smaller than 320x240.

<comment><USER4> I remember it was very playable on the 33mhz, 8mb 486DX machines I had at my high school. A friend had a 50mhz 12mb 486SX-2 which was wonderful for Doom. I sadly only had a 16mhz 386sx with 2mb of RAM. I amazingly did get Doom to "run" on this machine through some virtual memory program for Win 3.1. I had to shrink the screen size down to the smallest size for it to run at about 1fps.

I also remember a couple of other people that had "486" upgrades for their 386 based systems and Doom was perfectly playable on those systems as well. RAM was the big limiting factor that I remember.

<comment><USER5> Why'd you run it through Windows? I had a 33Mhz 386 and 2 (4?) MB of ram. I made a boot disk, and it worked fine.

<comment><USER6> I also done that on a friends' 386SX 25MHz machine that had only 2MB of ram. Doom required at least 4MB (sounds ridiculous as I am typing) and although it used the DOS4GW dos extender that was capable of swapping, it never worked.

On the other side, Win 3.1 was also capable of swapping and ran Doom. It was absolutely unplayable though :)

<comment><USER5> You know, I still have that 386 in my mom's attic. I'm tempted to break it out and see what it can do.

Ultima VII required a 33MHz 386DX w/ 4Mb of ram, and my computer ran it fine with a boot disk. If memory serves (and I was 7 at the time, so it probably doesn't), a DOS 6.22 bootdisk contained fields for page size, which leads me to believe it had support for virtual memory. That said, I do remember the massive stink that was made about virtual memory when Windows 95 came out, so I could be wrong.

I'm highly tempted to break out that old box and play around with it. This conversation tickles my nostalgia bone.

</comment></comment></comment></comment></comment></comment><comment><USER7> The first (and only one, currently) comment brought me back years ago! A major performance trick back then was to ensure the code and data would remain into the (very small) cache, as well as preferring structures that would be read in order.

</comment><comment></comment><comment></comment><comment>==

"Because walls were rendered as columns, wall textures were stored in memory rotated 90 degrees to the left. This was done to reduce the amount of computation required for texture coordinates"

The real reason is faster memory acces when reading linearly on old machine, less cpu cache clear. It's an old trick used on smooth rotozoomer effect in demo scene year ago.

<comment><USER8> Actually, rotating a texture 90 degrees in memory when making a rotozoomer still results in large amounts of cache misses depending on the current rotation angle. In order to get a smooth rotozoomer effect you have to use a technique called block rendering. This means you divide the image in square cells and render these one by one. Because the pixels in the cell are always near eachother you can reduce cache misses. There is an old snippet from Niklas Beisert floating around the web that explains exactly how this works.

<comment><USER7> Yup :) Here's the code if someone is interested:

ftp://latvia.tucows.com/pub/mirror/x2ftp/msdos/programming/demosrc/pasroto.zip

(I managed to find it back thanks to hornet.org)

Edit: you can also check out the list at ftp://latvia.tucows.com/pub/mirror/x2ftp/msdos/programming/demosrc/00index.txt

<comment><USER9> Thanks for digging this up. Much appreciated.

</comment></comment></comment></comment><comment><USER10> Link to nice doom port at bottom: <http://www.chocolate- doom.org/wiki/index.php/Chocolate_Doom>

</comment><comment><USER11> Michael Abrash' "Zen of Graphics Programming" has a good overview of many of the tricks used during that era: [http://www.amazon.com/Zen-Graphics- Programming-Ultimate-Writ...](http://www.amazon.com/Zen-Graphics-Programming- Ultimate-Writing/dp/188357708X)

(now, of course, mainly to be read for nostalgic reasons).

<comment><USER12> ... and, to avoid similar architectural mistakes in the future.

History is useful. We're not at the point where we want CS students to memorize what happened at the Battle of Algol in 1968, but it's darned close.

</comment></comment><comment><USER13> Any qed users? I liked it for map editing most.</comment>


Ask HN: Does HN move too fast? - <AUTHOR>

It seems to me that the vast majority of comments on any given HN article come within the first 24 hours after it is posted. Once it's dropped off the first page of /news, the discussion goes away.
While this is probably fine for the vast majority of articles, I think it encourages a fire-and-forget mentality among commenters that is antithetical to the spirit of hacking. There is value both in contributing to deeper conversations, and in reading them.
Does anyone else feel this way? Perhaps I'm just nostalgic for Usenet. <comment><USER2> I suspect that moving fast keeps flame wars and inside jokes to a minimum. It also encourages people to express their ideas clearly.

</comment><comment><USER3> Needs slightly more checking against near-duplicates submissions. Sometimes i get a deja vu feeling (maybe it's just a glitch in the matrix).

Though, some important issues that stays current deserves several updated submissions. Those near-dups are good.

</comment><comment><USER4> It's the difference between a news site and a discussion forum.

There's also a very anti-user design to the submissions that I think discourages even reading the comments once there's a lot of them.

</comment><comment><USER5> Maybe, but the community has adapted to the speed of churn.

It seems like okay stories leave the front page after about half a day, while good or great stories might live for ~36 hours.

</comment><comment><USER6> There are certainly times I think it does; but unfortunately, I think this is how this best has been designed. There's no designed way to watch threads that you find interesting and so if it's not on the home page, or not in the front page of your own private comments section, at least in my experience, you're not liable to see it. Often, I'll use my own karma as an indicator of whether or not I've received a comment to my work; and, if I haven't received a karma change in a long enough time, I'll go check my comments to see if they've been responded to. Usually they haven't.

All of that said to say yes, I think it does move too fast, especially for verbose, spirited discussion.</comment>


Show HN: Over 5k sites uploaded to tiiny.host - <AUTHOR> http://tiiny.site <comment><USER2> This is quite innovative. I like deploying the static websites on Vercel because `git push` and it's live.

When I am working, I tend to deploy like 10 - 15 times a day. Any plans to sync with git?

<comment><AUTHOR> Thanks, great feedback! Yep that's on the road map. Will be releasing that soon!</comment></comment>


Introducing the Voyage G2 Autonomous Vehicle - <AUTHOR> https://news.voyage.auto/introducing-the-voyage-g2-autonomous-vehicle-5e15cca399b5 <comment><AUTHOR> We pride ourselves at Voyage on being open about our technology and process. This post reads a little like a press release, but I'd love to answer any candid questions about Voyage or the autonomous vehicle field. I'll be here all afternoon!

<comment><AUTHOR> To start, I wanted to share more about the Velodyne VLS-128 LIDAR. As far as I'm aware, this is the first 128-channel LIDAR on the market, and it really is nuts.

• 9.6M points per second

• ~300 meters of range

• 360° coverage

Long-range and high-resolution LIDAR has historically been a unique advantage for Waymo, and my feeling is that the VLS-128 dramatically closes the gap for the rest of the field to have the same quality of vision.

Some video:

1\. [https://cdn- images-1.medium.com/max/2000/1*1qQPsYeu8suHJn412...](https://cdn- images-1.medium.com/max/2000/1*1qQPsYeu8suHJn412fPl3g.gif)

2\. [https://cdn- images-1.medium.com/max/2000/1*D13g1k2Ugq0NJJ3CN...](https://cdn- images-1.medium.com/max/2000/1*D13g1k2Ugq0NJJ3CNW15mg.gif)</comment></comment>


Ask HN: Books that explains algorithms with code instead of math? - <AUTHOR> Yes as the title says. I am looking for books about algorithms that explain the algorithms with code instead of math. The reason being, I understand code better than sometimes foreign to me math symbols and formulas.
So what type of algorithms am I looking for? Graphics and machine learning mostly, but also the usual suspects, sorting, data structures, etc.
Big plus if the book is explaining things on a very understandable level, so that any average developer can grasp the concepts without too much hassle.
So, what are the best books that fall in this category? <comment><USER2> If you know perl, I highly recommend Mastering Algorithms With Perl - all code and barely any symbols :)

</comment><comment><USER3> Data Structures and Algorithms in Java by Adam Drozdek has a lot of actual code that works. I think the code is online, but the book is worth buying because things are explained well ([http://www.amazon.com/Data-Structures- Algorithms-Java-Drozde...](http://www.amazon.com/Data-Structures-Algorithms- Java-Drozdek/dp/9814239232/) ) to be fair, there's quite a bit of math supporting Drozdek's book. Stay away from Sedwick, in my experience his examples are incomplete/don't compile and are abstract. Now, while, CLRSS is the gold standard (Joshua Bloch cites it in his Java Collections code) be aware it's math heavy and lacking on implementation details. Rumor was that there was a CD of CLRSS examples somewhere...

</comment><comment><USER4> I'm not sure how to parse your request. "explain the algorithms with code instead of math" is what's tripping me up, what exactly are you looking for? Algorithms are often described as pseudocode, so you want a book that explains pseudocode with code? At some point you have to move to another level of abstraction when talking about an algorithm, which is why mathematical language is used. That said, you could try CLRS. That seems to be the gold standard in algorithms books.

</comment><comment><USER5> This approach is wrong, if you want to learn algorithms. Also, you will not find an algorithms book that combines graphics, machine learning, and the usual suspects. Also, forget about learning this stuff 'without too much hassle'. There is no 'royal road' to algorithms. Practice, practice, and practice.

</comment><comment><USER6> I would start out with Sedgwick's Algorithms in C since it has lots of code examples before moving on to CLRS</comment>


Learn You An Ruby, a mix of LYAH and LPTHW - <AUTHOR> https://github.com/krainboltgreene/learn-you-an-ruby-for-awesome-power

<comment><USER2> :horror face: It's so not ready!

Also, the temp web version should be up soon.

<comment><USER3> no worries, thanks for your work on this!

</comment></comment><comment><USER4> Relinked, because the name of the repo seems to have changed:

[https://github.com/krainboltgreene/learn-you-the-ruby-for- aw...](https://github.com/krainboltgreene/learn-you-the-ruby-for-awesome- power)</comment>


I Wrote This on a 30-Year-Old Computer - <AUTHOR> https://www.theatlantic.com/technology/archive/2019/06/why-30-year-old-macintosh-works-better-todays/591154/ <comment><USER2> Every time I consider the simplicity of computing in the early 90s (when I was a kid) I have a pang of nostalgia.

We've made some wonderful things together as an industry but the author reminded me of something poignant in his last paragraph.

When I was younger I used to strive to keep my computer running as long as possible. I relished the idea that in the future, it would never need to turn off. Now I live a life where I seem to be hoping to do the exact opposite.

<comment><USER3> > Every time I consider the simplicity of computing in the early 90s (when I > was a kid) I have a pang of nostalgia.

Nostalgia is a very subjective thing. Today, people fondly reminisce their first BASIC application or developing their first program that created a network socket. In the future, the old hats may look back longingly over setting up their first multi-server cluster or modifying an app to take advantage of parallelism or a GPU. Nostalgia isn't going anywhere.

<comment><USER4> Personally I think nostalgia is a proxy for a generalized desire for simplicity. Most people don't long for the days of swapping out 10 different floppy disks to install an operating system, or of messing around with IRQ/DMA and UART settings to get a modem configured. It's the simple things (like BASIC) that are remembered and missed.

<comment><USER5> Maybe something like a cheap Chromebook that boots into something like BASIC where it's easy to write simple PRINT and GOTO based programs, draw stuff on the screen, and beep and boop the sound card.

It's having access to simple and easily composable basic elements, like Legos, that made computing so accessible back in the day. Computer gaming was my big motivation.

So, this simple platform also needs a decent collection of fun and hackable games, and other useful applications.

<comment><USER6> Or just boot into Emacs - which one can do e.g. via [https://github.com/ch11ng/exwm](https://github.com/ch11ng/exwm)

<comment><USER7> Emacs is great. It's got a calendar and a web browser and even comes with a text editor. What else do you need from a desktop?

</comment></comment></comment></comment></comment></comment><comment><USER8> > More sophisticated than MacWrite, Apple's word processor, the program is > still extremely basic—the only reason I chose (Microsoft) Word was so I > could open the file on my modern Mac to edit and file it.

That's a pretty interesting sentence, modern Macs can't work with a document created using 1980's MacWrite, but they can work with a document created using 1980's Microsoft Word, by using a modern version of Microsoft Word.

</comment><comment><USER9> "I glance past the small form of the Macintosh and ponder that idea while this file saves. The drone of the disk and the fan seems to lull into a resonant frequency, with the desk, the chair, and my body and brain connected to them."

I remember this feeling.

<comment><USER10> Whenever my normally-silent macbook needs to fire on its fans (compilation, DB restores, or any mid-sized 3D render) I love the audible reminder of how unfathomably hard it's working, and (by extension) what cool stuff is going on behind the scenes.

</comment></comment><comment><USER11> That keyboard, the M0116 "Apple Standard Keyboard" [https://deskthority.net/wiki/Apple_Standard_Keyboard](https://deskthority.net/wiki/Apple_Standard_Keyboard) , is one of the finest mechanical keyboards ever: [https://www.youtube.com/watch?v=9Y4fueemzcM](https://www.youtube.com/watch?v=9Y4fueemzcM) . Unfortunately the ALPS Orange switches don't tend to age well at all, but while they're in good condition they're fantastic.

Meanwhile, from the everything-old-is-new-again department, note the F-keyless Mac keyboard ... and a Mac line largely bifurcated between underpowered-yet- expensive all-in-ones and multi-thousand-dollar modular systems priced to fleece the employers of creative professionals.

</comment><comment><USER12> _By today's standards the machine is a dinosaur. It boasts a nine-inch black- and-white display._

That was pretty ancient-seeming back then. At least, as a kid, I couldn't fathom what they were thinking. It made the old Apple II seem better, and certainly anything from Commodore blew it straight out of the water. That was my impression then anyway.

<comment><USER13> Atari ST had a similar hires/hicontrast black-on-white designed for text display monitor available. If Commodore sold anything similar, I never saw it.

<comment><USER14> Commodore had the A2024 monitor. 1024x800 for NTSC and 1024x1024 for PAL.

[https://www.bigbookofamigahardware.com/bboah/product.aspx?id...](https://www.bigbookofamigahardware.com/bboah/product.aspx?id=863)

<comment><USER13> Sounds like an interesting hack :) But from the picture its unclear if it had the sharp bright pixels of the old toaster Macs.

</comment></comment></comment></comment><comment><USER13> > The version I have retailed for $3,900, or about $8,400 in 2019 dollars.

Had a similar Mac SE with 1MB RAM and 20MB HD. From the university store it was student priced under $2500 with the standard ADB keyboard (iirc $130 and normally not included!), an Imagewriter II printer ($600), Microsoft Word 4 ($300), and SuperPaint ($100). Still pricy but just to give folks an idea on the retail markup on Macs back then. Plus the dealers often quoted more than sticker.

</comment><comment><USER15> I was programming ESP8266 yesterday and I noticed I felt acute distress of the program size, whopping 250 kilobytes, only 40% left. This was a latent memory from 1970's when 250 single bytes was aplenty and enough for almost anything.

<comment><USER16> 256 kilobytes was called a double mega on MSX systems and was used in very large games mostly. They would be cartridges, mostly from Konami, that had that whopping amount of game on it. Strange to think that most webpages are larger than that now.

</comment></comment><comment><USER17> _I Wrote This on a 30-Year-Old Computer_

A former colleague wrote this on a 36-year-old computer earlier this year: [http://wayne.lorentz.me/This_TRS-80/](http://wayne.lorentz.me/This_TRS-80/)

The tl;dr version is that it has a better keyboard than any modern laptop, runs forever on AA batteries, and is a distraction-free zone.

<comment><USER18> I have one myself. But it is important to know its limitations -- most notably a maximum of 32K (which like on a modern smartphone, is both RAM and storage). So you are at best going to be writing very short documents. Which is why it was very popular among newspaper journalists writing columns in the field -- they would use a modem to transfer their column (often only a few paragraphs) to their editor.

</comment></comment><comment><USER19> It's not magic. Some of us get the appeal of vintage machines, but it's not magic that these systems work. I have a large room full of them that I use daily.

[https://www.bytecellar.com/photo_pano.html](https://www.bytecellar.com/photo_pano.html)

I'm glad that this article is out there, though.

<comment><USER20> Offtopic, but that panorama made me wonder if there's a photo equivalent to fix the fisheye like they do in raycasting engines: [http://www.permadi.com/tutorial/raycast/rayc8.html](http://www.permadi.com/tutorial/raycast/rayc8.html)

</comment></comment><comment><USER21> > _This [power supply and hard drive being loud] is the experience a computer > user would have had every time she booted up her Macintosh SE, a popular > all-in-one computer sold by Apple from 1987 to 1990._

A lot of that is from the hardware being 30 years old. Fans and drives get louder.

</comment><comment><USER22> There were so many nice computers with operating systems sporting cool features and the author picked the least interesting one. We already seen this type of "look, i wrote text on my old apple" blogposts so many times. This is terribly sad.

</comment><comment><USER23> Seems pretty advanced compared to the UNIX ed command I started typing with back in 1976.

<comment><USER24> ed is not bad at all once one groks the concepts behind it. I still use it in a jiffy when I want to edit just one or two lines of text on the quick.

</comment></comment><comment><USER25> I wonder how many people who aren't traditional terminal users would benefit from using a terminal-based editor, like vim, emacs, or even nano?

</comment><comment><USER26> I don't like hyperlinks in an article. I really prefer a footnote, so I can see the link directly before clicking on it.

<comment><USER27> Firefox and Chrome both show links in the bottom left of the screen when you hover over them. Not sure about other browsers.

<comment><USER28> In Safari, Show Status Bar must be enabled to do the same thing.</comment></comment></comment>


OpenFabrics Interface Programming Guide - <AUTHOR> https://github.com/ofiwg/ofi-guide/blob/master/OFIGuide.md <comment><USER2> OFI is an evolution of OFED, the low latency messaging and RDMA stack used for Infiniband and other very fast interconnects like Omnipath, 40GbE (RoCE).

[http://downloads.openfabrics.org/WorkGroups/ofiwg/ofi_archit...](http://downloads.openfabrics.org/WorkGroups/ofiwg/ofi_architecture/ofiwg- fi-intro.pdf)</comment>


Problems with Go Get - <AUTHOR> http://0x74696d.com/posts/go-get-considered-harmful/

<comment><USER2> It's clear to me that Go didn't get package management right,go get doesn't do package management at all. I think today it is the most important thing the go team has to solve.Otherwise, people will end up using different incompatible solutions.

Projects need a manifest so that proper dependencies are declared. People who say "just use make",or "just use git submodule" or "X feature of your VCS" are just pushing for more fragmentation. I download a package.Now I have to check what VCS feature it uses, wether it uses a shell script, make , or build tool Z to fetch dependencies, do people advocating for these solutions really want to use a language where libraries aren't compatible with each others? and then some people say : just vendor and commit dependencies. Ok but let's say I find a bug in one dep. Patching all the different versions of the same dep is just not something manageable.

I think it's time go devs acknowledge the fact that there is an serious issue here, instead of resorting to the ususal "you don't need that in go".

Godeps is a first step, but we need a official way to manage packages.So everybody is on the same line.

I would really like something like composer for Go, with flat dependencies. Which means that, unlikes NPM that downloads the entire world each time one installs a library, dependencies are flat,which keeps API stables and reduce the the amount of useless libraries and forks.

<comment><USER3> _> I think it's time go devs acknowledge the fact that there is an serious issue here, instead of resorting to the ususal "you don't need that in go"._

I'm out of the loop, so honest question: does golang-dev still say that? I thought they were not in denial of the problem, but indecisive about the solution. Like with generics? Or do they consider this a non-issue?

I know Cheney recently came out with his own solution, so it can't be that bad, right?

<comment><USER4> > I know Cheney recently came out with his own solution, so it can't be that > bad, right?

Yes, and so have others. What they really need to do is say "this is our blessed solution" and bundle it. Otherwise, there's just a bunch of competing solutions floating around and nothing is really going to take hold.

And yes I'm well aware that Java doesn't have a dependency manager but Maven emerged as the de facto way to handle dependencies. I think that has a lot to do with developer attitudes. I see Go developers as much more "don't tell me what to do, I'll roll my own solution" than Java developers, and that's why I fear a third-party dependency manager won't win out in the Go world.

</comment></comment></comment><comment><USER5> I'm awkwardly patting myself on the back here since I'm one of the co-authors, but I really wish more people knew about the package manager we have for Dart[1]. Really, though, I can't take credit, since we basically just do the same thing Bundler does.

[1]: [https://www.dartlang.org/tools/pub/get- started.html](https://www.dartlang.org/tools/pub/get-started.html)

It solves every single one of the problems listed here. It has a very simple workflow:

1\. You make a pubspec.yaml file to list your package's immediate dependencies and the version ranges you allow for them.

2\. You run "pub get". It finds all of your transitive dependencies, picks versions that satisfy the constraints, and downloads them.

It also creates a pubspec.lock file that specifies which versions it picked for everything. You check that into source control and now everyone on your team will use the exact same versions of all of your package's dependencies.

3\. When you want to get a new version of a dependency, you run "pub upgrade <package>".

4\. To add or remove a dependency, just edit your pubspec.yaml file then run "pub get" again.

We have a central repository where packages or hosted, and you can also pull in packages from Git or your local file system.

Packages do not at all interfere with each other. Two packages on your machine can use different versions of the same dependency without any problem.

Bundler is an absolutely brilliant package manager. My prediction is that every language either has a Bundler-style package manager or will eventually move to one.

<comment><USER6> One of the thing Cheney says in his talk about gb is that nobody likes the .lock files[1], I see that dart is using .lock files, bundlers also uses .lock files as does rust.

But other languages seems to get away without it.. clojure with lenigen seems to do it nicely.

So what are the problems with it?

[http://go-talks.appspot.com/github.com/davecheney/presentati...](http://go- talks.appspot.com/github.com/davecheney/presentations/reproducible- builds.slide#16)

<comment><USER7> A slight tangent, but as someone who is sort of in the "respect but do not use" Clojure category, Leiningen seems pretty amazing. I really hope future language package managers look to it for inspiration; it seems to get practically everything right.

</comment></comment></comment><comment><USER8> Go's Dave Cheney is taking a different approach of solving this problem with "gb". He agrees that "go get" should not be used.

[https://github.com/constabulary/gb](https://github.com/constabulary/gb)

<comment><USER9> I think gb might work, but it needs to work in conjunction with a solution like ruby bundler or dart pub imo.

<comment><USER10> It does, unless I'm really missing something. Since I'm not familiar with pup, can you walk me through what of bundler's functionality you'd be missing? Maybe I just don't use bundler correctly.

<comment><USER9> gb doesn't know which ranges of versions for dependencies and transitive dependencies are compatible with each other.

</comment></comment></comment></comment><comment><USER11> Lately I've been working on something to address the problems that I see with go get.

[https://github.com/skelterjohn/wgo](https://github.com/skelterjohn/wgo)

It's a workspace layer on top of the go tool, with added support for repository pinning. You 'wgo get' the repo you want, and 'wgo {save,restore}' will pin/fetch it in the future. Also, 'wgo FOO' will do 'go FOO' with GOPATH set for your workspace, so all the normal goodness is still there.

It also makes it so you don't have to manage the GOPATH env var anymore, which I always found to be a pain.

</comment><comment><USER12> While I do agree that go doesn't come with built-in solution for version pinning, this hating on go get is misguided.

Most package managements solutions that are popular in other ecosystem (pip for python, npm for node, cpan for perl etc.) combine 2 functions: downloading code and versioning.

go get only does the first part: downloading the code. If you ask me they solved the problem better than pip/npm/cpan by getting rid of intermediary and going straight to the source.

go get has never claimed to be versioning solution so while I understand why people make shallow comparisons to pip/npm/cpan and assume that it does and complain that it does it badly (including this article), the reality is that go get is not a versioning tool at all.

There are good reasons why versioning is not supported by default in go: there isn't one way of doing it that is clearly superior to other ways. That's why we have so many solution (including the one advocated by this article) and none of the solutions became a de-facto community standard.

In summary: stop complaining that go get doesn't do versioning. There are plenty of working versioning solutions to choose from, pick the one you like the best.

<comment><USER13> This is the problem with most of the Go community: Blind.

There are plenty of tools that works quite well: bundler, cargo (even with some young rough edges) and at some degrees npm.

So there are tools and better ways to do it.

It is just that Sir. Pike doesn't like them and Google doesn't need them.

> In summary: stop complaining that go get doesn't do versioning. There are > plenty of working versioning solutions to choose from, pick the one you like > the best.

The reason why there are plenty of them is just one: none of them works well.

Exact opposite of what you said. You don't have such mess of pkg managers in others languages.

<comment><USER14> `go get` and `godep` have been excellent for us. It does everything we need it to and have had no issues.

It has had far less issues than bundler, npm, etc.

<comment><USER13> I really doubt that. I always had problem with godeps at the point that I hate it.

Right now for example I have a giant repo with tons of dependencies and:

       godep: unable to detect version control system for code.google.com/ path

Which dependency? Tried some and they work. -__-

</comment></comment></comment></comment><comment><USER15> It's a meta point but the semantic of `go get` is so beautiful I'm profoundly disappointed that the tool isn't usable.

Being able to dispatch a co-routine with `go ...` is beautiful. Being able to fetch a dependency with `go get` is just as succinct and expressive.

I'm not up on Go language development but imho the world would be a better place if they could fix the behaviour to preserve the syntax!

<comment><USER16> Why is the semantic of `go get` any better than `bundle install`?

There's obviously no way they can fix the behavior in a backwards compatible way because it is so fundamentally broken, and I think that doing something because of a pretty name is absolutely ridiculous when it comes to creating a technical product. You need names like 'heartbleed' if you want to talk to management, but go get is a fundamentally developer focused tool and thus its utility should come over its naming.

I almost see its naming as a bad rap against google developers because it's an un-googleable phrase and who cares if it sounds pretty if it's un-googleable and also utterly terrible.

</comment></comment><comment><USER17> True story: I was in a group project in a distributed systems class, and we had made a project that relied on a go http routing package (I think it was martini or something). For some reason, my partner's computer wasn't able to present our project, so at the last minute, we cloned it to my computer. Between then and the night before, the router changed it's API, and forced us to do some crazy last-minute adjustments. In retrospect, we probably could have manually reverted that one package's repo, but git is scary to do things with at the last second.

TL;DR: Versioning is important.

<comment><USER18> As I understood, the original idea with go gettable libraries was that:

      - All releases go on master. Dev goes on separate branches.
      - Do not make backwards incompatible changes.
        If you need to make big changes, make another project.

A lot of libraries ignored the advice. Whenever a library breaks its API (sqlx for example), I stop using the library. This has made go get usable for me.

</comment></comment><comment><USER19> What was really difficult for me to wrap my head around was how go dependencies actually work out... via gopath, etc. My first exposure was the bash scripts for a few projects, that I was trying to build in windows... and it was to say an interesting first exposure.

I kind of wish that if go is going to use github targets for dependencies, that they at least support semver pinning and require tagging in github to support this. There are other issues, but I think that would help a lot.

It's always going to be an issue...

\----

Of all the developer package managers I've used npm is probably the best... and even that has some issues.

        1. Platform binaries are problematic
        2. Nested hierarchies result in unexpected duplications
        3. Nested hierarchies result in long paths (windows issue).

2, and 3 will be resolved with NPM3, but that brings some interesting breaking changes... beyond that is the fact that npm versions are generally pinned to node/iojs version for most.

1\. I'm hoping to see a consistent implementation for tagging modules with binary dependencies, or build dependencies so that there is a build platform in place for at least common targets. Who knows if this will ever happen, and likely will be tied to paid accounts, which isn't exactly a bad thing here.

</comment><comment><USER20> A very tiny shell script: [https://github.com/joewalnes/go- getter](https://github.com/joewalnes/go-getter)

Like go get, but you specify exact versions, including transitive dependencies. One GOPATH per project. And most importantly, you don't have to use go get to install the tool in the first place.

</comment><comment><USER21> Makefiles, build: docker build

Ugh, more Linux-Only. Don't follow this please.

(This isn't necessarily aimed only at blog post, but guys and gals there are other OSs supported by Go than Linux/OSX. Most of the time just having "go get" work is enough. Trust me, I've done it a lot as an outlier. I use FreeBSD and Windows.)

What's wrong with using something like party[1], or nut[2]? Or just vendoring in a way that go build still works. Seriously, what's wrong with more source files in your project's source repository?

This way I can go get your command (package main).

I dunno, maybe I'm just grumpy.

[1] [https://github.com/jingweno/nut](https://github.com/jingweno/nut) [2] [https://github.com/mjibson/party](https://github.com/mjibson/party)

<comment> vhost- What? GNU make works everywhere. I've used it on Window, OS X, every Linux distro I've touched and all the BSDs

<comment><USER21> Yes, I know about GNU make. The point is not GNU make, the point is needing to find out how to install third-party program on my OS of choice to build an app that should be as easy as "go build" because of the Programming Language used.

Maybe once that new fancy OneGet is on Windows (and maybe even becomes as good as brew looks on OSX, which I've never used) it will no longer bother me.

</comment></comment></comment><comment><USER22> At work we have a monolithic repo containing Java, Go, Python, etc, organized like this:

      Java //src/com/yext/..
      Go   //gocode/src/yext/...

GOPATH is set //gocode. .gitignore allows us to track only our code under the GOPATH using this snippet:

      /gocode/bin/
      /gocode/pkg/
      /gocode/src/*/
      !/gocode/src/yext/

We use glock[1] to sync dependencies across the team, without having to check them in.

The whole thing works very well at 50 developers and ~100 dependencies.

[1] [https://github.com/robfig/glock](https://github.com/robfig/glock)

</comment><comment><USER23> Seems like the author is suggesting something very similar to what [gpm][1][2] provides. I use gpm for a few projects, along with a makefile that creates a GOPATH at $(pwd)/_build, and it works fairly well.

[1]: [http://java.dzone.com/articles/why-gpm-right-go- package](http://java.dzone.com/articles/why-gpm-right-go-package)

[2]: [https://github.com/pote/gpm](https://github.com/pote/gpm)

<comment><USER24> Yeah, that plus gvp. But if you dig into both they're just shell scripts under the hood (nice ones, though!) and given that I've got a makefile or shell script to build the container, run tests, etc. then adding a third-party tool is just one tiny bit more overhead.

</comment></comment><comment><USER25> "go get" works for downloading open source code to check into a monorepo [1]. This is how Google does it. But most organizations use git, which doesn't scale enough to support monorepos very well, and it's awkward if you're just using git.

[1] [http://blog.rocketpoweredjetpants.com/2015/04/monorepo- one-s...](http://blog.rocketpoweredjetpants.com/2015/04/monorepo-one-source- code-repository-to.html)

</comment><comment><USER26> would not it be easier and cleaner to use the Android's REPO tool [1] to pin down the versions and layout your workspace? At least the REPO allows you to write down the dependency information in a declarative manner instead of Makefile scripting snippets.

[1] [https://source.android.com/source/using- repo.html](https://source.android.com/source/using-repo.html)

</comment><comment><USER27> I haven't written much go, so I'm just riffing here, but could you do something similar to what the author is suggesting with git submodules?

<comment><USER28> I think this is a perfect use case for `git subtree`[0]. It actually pulls everything in-repo, but can squash their history and do some other conveniences. I've always been surprised at its relative obscurity – maybe because of a terminology conflict with "subtree merges" – because it's so useful. I haven't actually used it for go dependencies, but it seems like it would be a good fit.

[0]: [https://github.com/git/git/blob/master/contrib/subtree/git-s...](https://github.com/git/git/blob/master/contrib/subtree/git- subtree.txt)

</comment></comment><comment><USER29> I did something similar to this a while back. It's just a few lines of bash. [https://gist.github.com/thelinuxkid/c57ddaf9ddbdce2f30b9](https://gist.github.com/thelinuxkid/c57ddaf9ddbdce2f30b9)

</comment><comment><USER30> I really like the OP's structure and makefile, but it looks like a pain to work against Golang's grain here, even though the methodology makes a lot more sense to me.

Maybe I have misunderstood the documentation, but what I truly, truly don't get about Go's $GOPATH is that it wants (1) my project directory to have some kind of canonical path, and (2) to pollute my project directory with dependencies. I have done a bunch of Go development and I _still_ don't get it.

So for example, I have myproject, which I naturally want to organize in this way:

        $HOME
          Projects
            foo
              .git
              src
                main.go
                stuff.go
              assets
                photo.png
                something.xml
              scripts
                setup_database.sh
              config
                config.sample.json
              Dockerfile
              Makefile

Go wants me to throw this away and structure it like this:

        $HOME
          go
            bin
              [...]
            src
              github.com
                BurntSushi
                  toml
                    [...]
                zenazn
                  goji
                    [...]
                myaccount
                  myproject
                    .git
                    main.go
                    stuff.go
                    assets
                      photo.png
                      something.xml
                    scripts
                      setup_database.sh
                    config
                      config.sample.json
                    Dockerfile
                    Makefile

I'm supposed to work within this jungle of dependencies and generated files, in the middle of which sits my project. At least Java, for all its many faults, has the good sense to let you store dependencies nested as JAR files in a subfolder, as opposed to turning your own project into a dependency.

I have tried fixing this by symlinking my project into $GOPATH/src/whatever, but Go doesn't let me to run "go" commands on things outside the $GOPATH, which makes this rather painful to do in a shell.

Adding to the general confusion, Go doesn't manage canonical package paths, so github.com/zenazn/goji/main.go is in the package "goji" and github.com/zenazn/goji/graceful/[asterisk].go are in the package "graceful", but those package names only mean something to the importer. When "goji" wants to use the package "graceful", it imports "github.com/zenazn/goji/graceful", which, being a full URL, _must_ be resolved from a root folder in GOPATH/src.

At least Java (for all its... etc.) had the good sense to make dotted package names mirror the folder structure of a project, not URL, thus being internally consistent. Go wants everything to live in the same sea of things. Vendoring is just half the problem.

<comment><USER31> Instead of throwing away your project structure, try creating a "go" subdirectory in your project and setting GOPATH to point to it. Yes, you need a separate GOPATH for each project, but it's no worse than needing a separate Python virtualenv for each project.

<comment><USER30> That's what I'm doing these days. But it's painful.

It means I have to run full commands such as "go install <my package>" or "go get <some package>". To install all dependencies, I have to do "go get <my package>".

If I want to work on a project that depends on other projects that haven't been pushed to master yet, I have to clone those projects and symlink them in manually. It's pretty painful.

Working with certain tools that assume a certain directory structure (protoc, for example, which computes paths relative to your Go root) is also painful.

"go build ." does work.

"No worse than virtualenv" isn't exactly a rousing endorsement. It's one of the poorer package-management systems out there. Go should be as simple and easy as Bundler. There's no excuse these days, I think.

<comment><USER31> You omitted that from your comment, but I agree. I was just pointing out the option, because it didn't sound as if you had tried it.

I did not mean to endorse anything.

<comment><USER30> Yeah, sorry. I have been trying different things. Not happy. But at least it seems that I'm not alone in being dissatisfied. It's surprising because Go does a lot of things right, and disappointing because it's such a brake on productivity.

</comment></comment></comment></comment></comment><comment><USER32> I believe it is in their radar. Go 1.6+ :)

[https://groups.google.com/forum/#!topic/golang- dev/nMWoEAG55...](https://groups.google.com/forum/#!topic/golang- dev/nMWoEAG55v8%5B1-25%5D)

</comment><comment><USER33> Mods, please rename to "Go Get Considered Harmful", as per the article's title.

</comment><comment><USER34>
        >>> ('{:c}'* 3).format(0x74, 0x69, 0x6d)
        'tim'

</comment><comment><USER24> Author here. Happy to answer any questions about our setup.

</comment><comment><USER35> We should have python's virtualenv analogue in Go.

<comment><USER36> I've been using Godeps, which feels like a close equivalent - have a gander and see if it's useful:

[https://github.com/tools/godep](https://github.com/tools/godep)

Semver support was what got me using godep and I haven't looked back since (at least for larger projects).</comment></comment>


Show HN: Qwerkey, a better way to play chords - <AUTHOR> http://some1else.github.io/qwerkey/ <comment><AUTHOR> Try to play diagonally, up and to the right. It's easier to grasp the harmonic effect, when you're first trying it out. A bit more info and a gif are in the repository readme[1].

1: [https://github.com/Some1Else/qwerkey](https://github.com/Some1Else/qwerkey)

</comment><comment><USER2> <3</comment>