How the Netherlands Feeds the World (2017) - <AUTHOR> https://www.nationalgeographic.com/magazine/2017/09/holland-agriculture-sustainable-farming/ <comment><USER2> Not to talk down the Dutch achievement too much but the quality of statistics used in this article is terrible.

Total monetary value of agricultural exports is a terrible way to measure output. It doesn't take into account: 1\. Re-exports (Rotterdam is the largest port in Europe) 2\. Specialisation in the production of certain goods (i.e. high imports and exports) 3\. The high price of agricultural goods in Europe, driven by high trade barriers. 4\. Low domestic consumption compared to more populous nations.

For a better analysis of the monetary value of Dutch agricultural exports see here: [https://www.cbs.nl/en-gb/news/2018/03/dutch-agricultural- exp...](https://www.cbs.nl/en-gb/news/2018/03/dutch-agricultural-exports-at- record-high)

Note for example the 22.5 billion Euros in re-exports of agricultural goods.

90% reduction in water usage is also hard to judge without looking at changes in other advanced agricultural nations.

It would be more interesting to see the yields per-hectare the Dutch are achieving compared to other advanced agricultural nations.

<comment><USER3> One would not expect The Netherlands to export so much in the presence of "high trade barriers". But I strongly suspect that trade barriers between European Union Members to be very low.

Low transport costs also helped The Netherlands achieve this position. Numerous highways, railways and waterways connect it to it's neighbours.

<comment><USER2> The Netherlands is in the EU. They have total unrestricted access to the EU market, which is exactly where the vast majority of their agricultural exports go.

<comment><USER4> The EU isn't a free market when it comes to agricultural produce at all, thanks to the CAP (Common Agricultural Policy)[1].

E.g. just in the last year[2] central planning for how much quantity of sugar beets each country could produce was finally phased out, with the Dutch expecting in production of up to 25% as a result[3].

This was one of the grand bargains of the EU. In exchange for the likes of France, Spain etc. joining the union they wanted guarantees that these politically sensitive sectors of their economy wouldn't be subject to real competition.

So production in countries like The Netherlands is artificially low since a combination of quotas and other central planning restricts production.

1\. [https://en.wikipedia.org/wiki/Common_Agricultural_Policy#Equ...](https://en.wikipedia.org/wiki/Common_Agricultural_Policy#Equity_among_member_states)

2\. [http://europa.eu/rapid/press- release_MEMO-17-3488_en.htm](http://europa.eu/rapid/press- release_MEMO-17-3488_en.htm)

3\. [https://www.dutchnews.nl/news/2017/01/dutch-sugar- cooperativ...](https://www.dutchnews.nl/news/2017/01/dutch-sugar-cooperative- expects-a-surge-in-production/)

</comment></comment></comment></comment><comment><USER5> What is kind of funny is that people here like to pay more for 'organic' food because they think it's more healthy. But what comes from the greenhouse is so lab-grown it's more healthy than anything. Those labs also don't like to use pesticides because it's unhealthy for the workers and plants as well. They use insects to control pests.

The only problem with greenhouse food is that it tastes like nothing. A tomato grown on soil has much more taste than grown on water.

<comment><USER6> It's not the soil but the lack of sun that deprives the tomatoes of their taste.

Greenhouses in the Netherlands are heated with natural gas. The Dutch government subsidises the gas so they can keep their costs low and be competitive. So that's another reason to eat organic food; it is more sustainable because it uses natural sunlight. And tastier as well!

<comment><USER7> I'm fairly certain nobody would buy the Tomatoes if they didn't have any taste.

<comment><USER8> Depending where you live you probably already do and think that 'no taste' is the taste of tomato. In the Netherlands (where I am from) and the south of Spain (where I spend a lot of time) the difference is huge in taste buying an average tomato; the ones in NL taste like nothing compared but seems people are used to it.

</comment></comment></comment></comment><comment><USER9> Only problem is that dutch vegetables have no taste and are still insane expensive compared to meat (which is also tasteless). So yeah, awesome thinkering and efficiency here, but quality is not there.

<comment><USER10> Actually a bit of a myth (I'm Dutch). It's more a case of targeting different markets with different price and quality ratios. E.g. Germans love to complain about Dutch water bombs but buy them by the kilo anyway because the stuff is cheap. They don't realize the more expensive varieties also available to them come from the same country. You don't have to buy the cheap ones but they do; it's their poor tastes that are driving the demand for this.

Likewise, Italy imports a lot of tomatoes (more than they export according to this: [https://www.freshplaza.com/article/2158667/italian-table- tom...](https://www.freshplaza.com/article/2158667/italian-table-tomato- imports-grew-in/)). Most of those come from the Netherlands. Italians are of course famously picky when it comes to their food. Yet they seem to buy Dutch produce.

<comment><USER11> The other problem is the Netherlands are at risk of getting wiped out by rising sea levels.

Your example of Italy is bad news - in effect countries that could produce their own are buying Dutch because it's cheaper, presumably because economies of scale now work in Holland's favor.

In other words we depend on a country that's at risk of being wiped out by flooding for food _and_ that same countrys dominant market position is also holding back food production in other countries.

Sounds like the back story for a bad movie.

<comment><USER10> Sea level changes are not that much of a challenge. We have centuries of experience building infrastructure to manage that and decades more to leverage that. Much of the country is already below sea level and already well prepared for extreme water levels. Some of the infrastructure will need some upgrading of course. The reality of sea level changes is that they will be most devastating in places where they won't be able to pull together the economic resources to put infrastructure in place or where they are making the historic mistake of assuming it won't happen.

</comment></comment></comment></comment><comment><USER12> "Food Valley"? Oh, come on... they could have done better! How about "Sustenance Valley"? But that brings up a secondary point... a valley?? In Holland??

<comment><USER13> Yeah, it's nuts. Every region and a half in the Netherlands that specializes in something calls themselves "something something valley".

That said I was once in Silicon Valley and there were hills on the left and water on the right. Not really much of a valley either.

<comment><USER14> I live in the Netherlands, and I've never heard anything like that before. I live in The Hague, maybe it's a regional thing?

<comment><USER15> I used to live in the Netherlands and I'm writing this from Utrecht. The valleys are regional:

1\. [https://en.wikipedia.org/wiki/Food_Valley](https://en.wikipedia.org/wiki/Food_Valley) in Wageningen

2\. [https://www.healthvalley.nl/](https://www.healthvalley.nl/) in Nijmegen

3\. [http://www.metalvalley.eu/nl/home](http://www.metalvalley.eu/nl/home) in Brabant

4\. [https://www.immunovalley.nl/](https://www.immunovalley.nl/) in Utrecht

5\. [https://legalvalleynederland.nl/](https://legalvalleynederland.nl/) in Arnhem

6\. [https://www.energyvalley.nl/](https://www.energyvalley.nl/) in the northern provinces

If you like partying you can always go to: [https://dancevalley.com/](https://dancevalley.com/)

<comment><USER7> Coming back to the Netherlands from abroad it often feels like they're pretty good at coming up with what I would consider fairly cringeworthy names.

<comment><USER13> Yep! I myself live and work in the area (Eindhoven) that willingly calls itself "Brainport". I hear the word almost daily for ten years and I still cringe.

</comment></comment></comment></comment></comment></comment><comment><USER16> _" A farm atop a former factory in The Hague produces vegetables and fish in a self-sustaining loop: Fish waste fertilizes plants, which filter the water for the fish."_

If people should know something about how diseases & parasites appear and develop is that the existence of exactly this kind of "self-sustaining loop" is _the_ essential part. Even in the outside environment, any given organism bares the risk of taking part in development of a new parasitic creature's life-cycle, but that risk is greatly reduced by having an inconsistent pattern of interaction with other symbiotic creatures. Here the people involved are just asking for it (unless they go out their way to sterilize the substances circulating in that plant-fish loop).

</comment><comment><USER17> Impressive, to be sure, especially the technical side of things.

However I really dislike greenhouses like these for the huge amount of light pollution they cause (speaking as an amateur astronomer). Sure, it probably doesn't matter much in a densely populated area like the Netherlands, but they start to also pop up here in Eastern Europe, in (previously-) dark-sky locations.

</comment><comment><USER18> Google Earth to the greenhouse complex is southern Spain is visually entertaining. Similar giant single area in the NL?

<comment><USER19> Westland south of The Hague

[https://firstname.lastname@example.com,4.190...](https://firstname.lastname@example.com,4.1903113,13z/data=!4m5!3m4!1s0x47c5b7d4aa3838df:0x31ee3d1de87f506e!8m2!3d51.9991757!4d4.217457)

<comment><USER18> Ok. Keep scale same and look at El Ejido

[https://www.google.com/maps/place/El+Ejido,+Almería,+Spain/@...](https://www.google.com/maps/place/El+Ejido,+Almería,+email@example.com,-2.7821377,11z)

</comment></comment></comment><comment><USER20> I'm guessing that Israel and Spain might be behind the Netherlands in this trend. Absolutely fascinating.</comment>


The most dangerous code in the world - <AUTHOR> https://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html <comment><USER3> The worst example from this paper is Curl's API.

Curl has an option, CURL_SSL_VERIFYHOST. When VERIFYHOST=0, Curl does what you'd expect: it effectively doesn't validate SSL certificates.

When VERIFYHOST=2, Curl does what you'd expect: it verifies SSL certificates, ensuring that one of the hosts attested by the certificate matches the host presenting it.

When VERIFYHOST=1, or, in some popular languages, when VERIFYHOST=TRUE, Curl does something very strange. It checks to see if the certificate attests to any hostnames, and then accepts the certificate _no matter who presents it_.

Developers reasonably assume parameters like "VERIFYHOST" are boolean; either we're verifying or we're not. So they routinely set VERIFYHOST to 1 or "true" (which can promote to 1). Because Curl has this weird in-between setting, which does not express any security policy I can figure out, they're effectively not verifying certificates.

<comment><USER4> While you're attacking curl you (intentionally?) forget to mention that the default value is 2 that is perfectly safe. I don't think it's "reasonable to assume" that the parameter is a boolean value while it's well documented. If programmers are lazy, they make many kind of mistakes.

<comment><USER3> Yes, yes, I know, stupid Paypal, stupid Amazon, we get it. VERIFYHOST=1 doing nothing makes perfect sense, you just have to read the documentation. Of course.

<comment><USER4> Yes, Amazon et.al. made some mistakes due being lazy and PHP being the nice language that it is. They have fixed it. Let's go ahead and find another non- default, documented parameter value in a library and call it a vulnerability.

<comment><USER3> Ok. Your turn. Go!

</comment></comment></comment></comment></comment><comment><USER5> I agree that these APIs are fundamentally difficult to use correctly (sometimes it almost seems as if they've been designed to trick you), and that developers commonly get them wrong, but this paper is perhaps a little more inflammatory than it should be.

They cast a really wide net, looking for as many examples as possible where non-browser applications fail to do SSL validation correctly, but then conclude that this will result in a security compromise without fully examining the implications.

For instance, they point out that many SDKs for Amazon FPS don't validate certificates correctly. But I didn't see them mention that the FPS protocol does its own signature-based authentication and that credentials are never transmitted in the clear: it was essentially designed to operate over an insecure transport to begin with.

Likewise, they point out an "unsafe" construction that an Android application that I wrote (TextSecure) uses. But they don't mention that this is for communication with an MMSC, that this is how it has to be (many don't present CA-signed certificates), and that the point of TextSecure is that an OTR-like secure protocol is layered on top of base transport layer (be it SMS or MMS).

So I think the paper would be a lot stronger if they weren't overstating their position so much.

<comment><USER6> One of the authors here... The paper is accompanied by a FAQ, which among other things explains the impact on specific software:

[https://docs.google.com/document/pub?id=1roBIeSJsYq3Ntpf6N0P...](https://docs.google.com/document/pub?id=1roBIeSJsYq3Ntpf6N0PIeeAAvu4ddn7mGo6Qb7aL7ew)

For example, broken SSL in Amazon FPS allows a MITM attacker to forge instant payment notifications and defraud merchants who use vulnerable SDKs. This is a real vulnerability, acknowledged by Amazon.

<comment><USER7> Can you address his specific point about TextSecure?

<comment><USER8> please would ya?

</comment></comment></comment></comment><comment><USER9> The title should be renamed to:

Many security flaws found in commonly used SSL libraries.

Other than that, it is a great find.

<comment><USER10> Thanks for de-FUDing the title, I scanned the comments hoping someone had done that =)

</comment></comment><comment><USER11> How ironic. Even these guys hosting a paper about SSL can't host their stuff securely on an HTTPS server.

<base href="[http://crypto.stanford.edu/~dabo/pubs/pubs.html>](http://crypto.stanford.edu/~dabo/pubs/pubs.html>);

This causes the page to throw an HTTPS warning: "this page loads insecure content" due to the css loaded over HTTP.

<comment><USER3> Yes, yes, stupid Dan Boneh, extracts SSL keys from servers over the Internet using timing measurements, but can't properly write an HTML page, we get it. Credibility -> toilet.

<comment><USER12> Because being talented at one thing means you're talented at _everything_.

<comment><USER3> The two things here being "finding critical vulnerabilities in the TLS protocol and its implementations, and generally being one of the most highly- regarded academic practical cryptographers" and "writing HTML". Trenchant point.

<comment><USER12> Exactly (I think people might assume I was disagreeing), the skill sets are wildly different in many ways.

</comment></comment></comment></comment></comment><comment><USER13> From the PDF linked in the article:

"Not the most interesting technically, but perhaps the most devastating (because of the ease of exploitation) bug is the broken certificate validation in the Chase mobile banking app on Android. Even a primitive network attacker—for example, someone in control of a malicious Wi-Fi access point—can exploit this vulnerability to harvest the login credentials of Chase mobile banking customers."

<comment><USER13> Further down in the PDF, the authors present the decompiled source of the Chase app:

      public final void checkServerTrusted(X509Certificate[]
          paramArrayOfX509Certificate, String paramString)
      {
        if ((paramArrayOfX509Certificate != null) && (
            paramArrayOfX509Certificate.length == 1))
          paramArrayOfX509Certificate[0].checkValidity();
        while (true)
        {
          return;
          this.a.checkServerTrusted(
              paramArrayOfX509Certificate, paramString);
        }
      }

Good to know this isn't used for anything critical, just for, um, mobile banking.

<comment><USER14> Okay. Please tell me I'm reading that wrong.

       void foo(X509Certificate[] a, String unused) {
         if( a != null && a.length == 1) a[0].checkValidity();
         while(true){
            return;
            /* never executed stuff */
            foo(a, unused);
         }
       }

X509Certificate.checkValidity()'s javadoc: [http://docs.oracle.com/javase/1.5.0/docs/api/javax/security/...](http://docs.oracle.com/javase/1.5.0/docs/api/javax/security/cert/X509Certificate.html#checkValidity%28%29)

Selections from above link:

Checks that the certificate is currently valid. It is if the current date and time are within the validity period given in the certificate.

      Throws:
      CertificateExpiredException - if the certificate has expired.
      CertificateNotYetValidException - if the certificate is not yet valid.

There has got to be more to their code than this. It does nothing useful except make sure a random certificate is not expired. If there is no certificate, it doesn't fail or return false or anything. What is this!?

Could this be a profiler-optimized set of code that really had a LOT more code to it? But then why would the bit of random junk after the return statement exist? Did no one do a proper code-review of this???

[edit: I used JavaSE 1.5 above; but I just checked Android's version and it looks to be the same, just less verbage in the docs.]

<comment><USER13> The issue with the code above is the return statement, most likely inserted during development, with the good intention to remove it later, since the "bit of random junk" that follows would have validated the server's certificate.

<comment><USER14> But it doesn't. It's calling the exact same function. As your sibling comment remarked, it looked like someone really frustrated with why they were getting a stack-overflow error and gave up.

</comment></comment></comment></comment></comment><comment><USER15> Sounds like it might be easier to list the options that actually do the Right Thing. If you're using Python, for example, the correct way to make HTTP requests is to ignore the standard library's urllib and (shudder) urllib2, and use Requests instead:

<http://docs.python-requests.org/en/latest/>

It validates SSL certificates correctly by default. How about other languages?

<comment><USER16> Good to know. I always wondered how come none those libraries fully validate SSL certificates. On one of my projects, we were hooking into the Windows Winhttp Libraries to be able to do this (and a couple of other things), but when porting to Mac we kinda had to accept the standard libs just didn't care enough about this. It's been a while ago, so perhaps things have changed. Requests is a great example of this I guess.

</comment></comment><comment><USER17> I notice that whenever I use "wget <https://github.com/>[...]" I always end up typing wget --no-check-certificate because the first try never works.

I suppose my web browser has an extended list of CA that my OSX lion does not know about.

<comment><USER18> My HTC Droid Incredible's browser also always complained about their certificate and popped up a dialog box I had to click through. But now that I've installed Cyanogen Mod, it hasn't been a problem, so I guess it's one of several things HTC broke.

</comment></comment><comment><USER19> So of all the possible futures we could have, ones where we use computers to give us crypto, good security and privacy etc, instead we end up with Masamune Shirow's admitted guess of Ghost in the Shell where people can't properly use their arms due to 5 different version of the driver installed and people having 10 different viruses IN THEIR BRAINS and are constantly getting hacked and having their bodies taken over.

</comment><comment><USER20> they make this point in the paper, but still it surprises me - the level of testing for payment frameworks seems surprisingly minimal. it's pretty easy with openssl to roll your own certificates to test a bunch of different issues. you'd think that the people involved would have quite an incentive to test well.

i'm not saying that this would solve all the problems, or that you should develop critical financial software by having people that don't understand much writing tests. but tests are pretty much common culture now; you'd think people would have considered this. and the argument the paper makes is not that the programmers are clueless, but that they are confused by the API, so they should be able to think up some useful tests...

of course, integration testing with sockets is a bit more complicated than unit tests (perhaps something toolkit apis should support is a way to allow testing without sockets?), but it's not super-hard. [edit: hmm. although testing for unreliable dns is going to be more tricky.]

</comment><comment><USER21> The title is a bit sensationalist - there was incorrect code and it made the copy/paste rounds. Presumably all incorrect code is dangerous to some degree but I'm certain there's a more fitting title for this story.

At any rate, here is a pull request for PHP which attempts to address the issue:

<https://github.com/php/php-src/pull/221>

</comment><comment><USER7> I have only read the first two sections, but the prose in this paper is a breath of fresh air. It is clear and strong.

</comment><comment><USER22> Slightly related, link to Peereboom's rant on the OpenSSL library (a bit dated): <http://www.peereboom.us/assl/assl/html/openssl.html>

</comment><comment><USER23> I came across this issue when using node.js to make secure requests as a client and after setting up tests with bad certs found it silently worked anyway. To get it working you need to be at a certain version of node.js and make sure you set the options up carefully. Testing with a bad certificate is essential for this stuff. [http://stackoverflow.com/questions/10142431/my- node-js-https...](http://stackoverflow.com/questions/10142431/my-node-js- https-client-always-works-regardless-of-certificate-validity)

</comment><comment><USER24> So how soon until we start seeing developers fix these gaping holes? And, more importantly, how soon do we start seeing app-specific exploits that take advantage of this problem?

<comment><USER25> Probably as soon as certificate validation is more reliable.

I can see turning off validation for stuff where you're sending non-important data to a third party which may or may not be encrypted (asyncronous API pings where you say "something happened, ask us [securely] what it was") - <USER2> that's only OK if you're just as willing to send the info over an http connection as well.

If you turn off cert validation in your app to your own API, something is seriously wrong. Unfortunately, it often comes down to simply buying a more expensive certificate that's signed by a root CA with wider trust. Given spending $50 or CURLOPT_SSL_VERIFYPEER=false, a scary number of people will choose the latter.

These libraries should provide an easy way to specify an additional root CA when validating the chain of trust - that way you could even use a self-signed cert without disabling the chain of trust check, all you'd have to do is embed your self-signing public key.

Probably more important, there should be clearer error messages when the SSL chain verification. Cryptic "verify the CA cert is OK" messages don't help much; they should say "SSL CA [name] is not in the installed trusted certificates, add trust for it with CURLOPT_CAINFO, details at example.com/ca.html" or something along those lines.

It may cause issues with root CA revocation (DigiNotar anyone?), but it's still better than disabling the checks entirely.

<comment><USER26> If the SSL/TLS server is properly configured with all of the nodes in the certificate chain, you don't need to buy more expensive certs.

Now I will say that, most common apps (Apache HTTPD + mod_ssl for example), do not make this process as clear as it should be. Specifying cert order wrong in the Intermediate Chain file will still break cert validation for some apps, because it's sent out exactly as the server admin configures (and this fact is often lost of folks who maybe touch their certs once a year when they expire).

<comment><USER25> There's no standard as to which certificate issuers have their trusted root CA cert installed. If your clients don't have it on their machines (desktops, phones, servers, whatever) it doesn't matter what you paid.

My point was that the more widely-trusted issuers (meaning installed by default on the most OSes) can get away with charging more because they're more compatible.

There's nothing stopping you from setting up your own root CA and self- signing, maintaining a full chain of trust if you build the certs right (I've done it; once you have the process documented well it only takes a few minutes) but unless the clients trust the lowest link in the chain (not the case for self-signed, and some cheaper issuers) you'll still get validation errors.

</comment></comment></comment></comment><comment><USER27> And odds are the guys that wrote this paper don't have any clue that even if those writing the CLI tools/libraries/frameworks that use SSL had locked them completely down, developers and sysadmins would write scripts to agree-to-all, fake auth, etc. to get around security, because we have jobs that have to get done and security is not what we are all paid to do. Security is only critical when it fails. People claim to want security. They may even have an office of security. But even if that office of security is scanning all the apps, taking production apps down because they didn't throttle their probes, and maybe even looking at code- they cannot do the job of the developer.

It is destined to be flawed as long as insecurity is allowed. Only when every exploit is exploited continously will people be vigilant.

<comment><USER3> Yes! Yes! Stupid researchers! Who has time for security? We've got mobile banking transactions to process!

</comment></comment><comment><USER28> Anyone have an example of good cert verification in Java? The concept at <https://github.com/iSECPartners/ssl-conservatory> is great, but it needs examples in more languages. Our case is pretty weird (some self-signed certs between peers, cert pinning of sorts in that we only accept equifax as a root signer, no default signing authorities accepted), but anyone see holes in the authenticate method of our trust manager at:

[https://github.com/getlantern/lantern/blob/master/src/main/j...](https://github.com/getlantern/lantern/blob/master/src/main/java/org/lantern/LanternTrustManager.java)

? This code is intended for deployment in potentially dangerous regions for getting around government censors.

Thanks.

</comment><comment><USER29> I noticed this the other day in Rails. ActiveResource::Connection in 3.2.8 is affected in that the default OpenSSL verification mode is "OpenSSL::SSL::VERIFY_NONE". A developer has to explicitly set it for SSL validation.

You can see it here: [https://github.com/rails/rails/blob/3-2-stable/activeresourc...](https://github.com/rails/rails/blob/3-2-stable/activeresource/lib/active_resource/connection.rb)

I'm pointing it out as it was not mentioned in the paper.

Edit: It looks like it has been that way since SSL was first implemented in Connection.

</comment><comment><USER30> 2010, Certificate verification is essential to TLS.

require 'always_verify_ssl_certificates' AlwaysVerifySSLCertificates.ca_file = "/path/path/path/cacert.pem"

http= Net::HTTP.new('<https://some.ssl.site>, 443) http.use_ssl = true req = Net::HTTP::Get.new('/') response = http.request(req)

[http://www.rubyinside.com/how-to-cure-nethttps-risky- default...](http://www.rubyinside.com/how-to-cure-nethttps-risky-default- https-behavior-4010.html)

</comment><comment><USER31> Everyone who does any development should read this paper. It is not just for SSL specialists!

</comment><comment><USER32> Are Rack-based middleware affected by these vulnerabilities (or did I lose the plot)?

</comment><comment><USER33> "...bad implementations of SSL like OpenSSL..."

<falls off chair></comment>


TriggerHappy, Autonomous, and Disobedient: Nordbat and Mission Command in Bosnia - <AUTHOR> https://thestrategybridge.org/the-bridge/2017/9/20/trigger-happy-autonomous-and-disobedient-nordbat-2-and-mission-command-in-bosnia <comment><USER2> Sending in peacekeepers who cannot shoot back is absurd. It's political cowardice at it's worst. We really need to figure out this world governance thing and how to effectively rebuild rogue nation states. I'm not sure what that looks like, but probably not the UN.

<comment><USER3> After the Iraq fiasco I think a little humility should be in order. Grand schemes for "world governance" have a tendency to go wrong, and it's hard to come up with a definition of "rogue" that doesn't include too many countries.

The limitations on the peacekeepers were part of the broader context of trying not to simply invade the place and trigger conflict with Russia (e.g. [http://www.nytimes.com/1994/05/18/world/us-clashes-with- russ...](http://www.nytimes.com/1994/05/18/world/us-clashes-with-russia-over- bosnia.html) ), a pattern which continues today in Syria.

</comment></comment><comment><USER4> "Then they [the Croatian militants] issued an ultimatum: hand over the three Muslim nurses, and we will leave you alone."

Quite nauseating to think that this happened in Europe in the 90's.

Edit: even more nauseating to go on vacation there and realize that the people who did this are no different from yourself. This could happen anywhere.

<comment><USER3> Indeed. This is partly why people are so sensitive about anti-Muslim rhetoric and policy in the West today. It's an alarmingly short slippery slope from "send them back" to "send them back and burn their houses" (e.g. Rohingya) to "just hand them over so we can murder them" to "send in the genocide squads", as in Srebrenica that was mentioned at the end of the article.

</comment></comment><comment><USER5> The tough part is to find the right balance between top down management that usually results in paralyzing micromanagment, preventing quick and dynamic reaction, and a possible loss of control over the single acting units turning into rogue actors.

As the article correctly states, this is an issue of trust, but trust can only be earned with experience, which is lacking in such a complex environment.

<comment><USER6> This trust is a result of intentional training towards enabling everyone down to private to know when to make a decision and when to bother chain of command. It's not only training of management, it's more like a culture.

I guess this post in part has some traction with hn because mission command seems like a wonderful management technique. Just let the people who knows best and have first hand information make efficient decisions, fast, right? But it actually takes good people and a lot of training to work in practice. On every level there has to be a "feeling" of what's reasonable. And management, in turn, must learn when to step back, and maybe more importantly not punish afterwards when equipped with hindsight.

(Served two years in swedish army, too young for Bosnia myself but couple of my officers were in nordbat2)

<comment><USER7> Everyone involved has to know the objective, which is a lot of more work than most people understand. Besides that I think that the selection and training for the BA0X where uniq for the swedish army who could pick from a large volunteer base.

</comment></comment></comment><comment><USER8> "The culture of mission command in Sweden dates back to 1943, when senior Swedish army officers were taking note of the tactical superiority of German troops fighting Soviets on the Eastern Front."

Auftragstaktik: mission command - part of the German Army doctrine developed in the 19th C. TN DuPuy's Genius for War gives an excellent explanation.

</comment><comment><USER9> Missing a dash between "trigger" and "happy" is bothering me more than it should. Reads like a new vulnerability name...

Please fix the title as it is "Trigger-Happy" in the original.

<comment><USER10> HN titles have an 80 character limit.

The dash doesn't fit.

<comment><USER11> Remove the 2nd comma then, its useless.

<comment><USER10> I'm neither a mod nor the submitter. I can't do anything with titles.

</comment></comment></comment></comment><comment><USER12> This is precisely how i expect UN forces to behave. Bravo!</comment>


Embarrassing code I wrote under stress at a job interview - <AUTHOR> http://www.smashcompany.com/technology/embarrassing-code-i-wrote-under-stress-at-a-job-interview <comment><USER2> > Interviews are weird: the pressure of time, and not being able to look > things up, distorts the code.

In the interviews I do, I tell the candidate that:

1\. There is no time pressure. Work at a normal pace, as if you were working here. This is not a speed test. I don't expect you to finish. I mainly want to know how you think.

2\. You should look things up. Behave the same way you would when coding at home. Use Google, Stack Overflow, documentation, etc.

This probably works better for the interviews I give because the problem is not implementing an existing algorithm. It's a realistic task, something that we've actually built on-the-job.

<comment><USER3> I don't think that's good enough. It's unfair to ask candidates to write code on the spot.

<comment><USER4> I have had my share of bad interviews but I don't agree with this sentiment, why is it unfair? Haven't you ever been in a situation where you have to create a hot fix and deploy it as fast as you can. Haven't you ever gone through a crunch period where the deadline is closing in and and you have to fix that elusive bug. There is limited time for everything, the sooner we realize that the better.

Having control over your nerves when s __* hits the fan is a valuable quality, which we should all strive for.

Most of these interviews are tests to see your approach to solving a problem. The best practice is to come up with a strategy before you hit the first key, ask as many questions as you can and find as many corner cases as you can. All the sane interviewers will appreciate that even if you don't end up with the best solution. And please treat all interviews as a learning experience, you will be much better off.

<comment><USER5> I must've been working in a much different environment than you. I've _never_ had to solve a difficult problem in 15 minutes while someone who knew the solution watched over my shoulder and did their best to answer my questions without answering them too well. Ohh yeah and the results of my performance had potentially life changing consequences.

The adversarial and extreme time boxed nature of a coding interview is something that is truly outside the day to day experience of the vast majority of programmers.

>Having control over your nerves when s* hits the fan is a valuable quality, which we should all strive for.

That may be true, but think about that for a minute. Go back to college and think of how many people in the class were comfortable going up to the board and solving problems in front of everyone.

When I was taking Automata, we could get an extra point on our final grade by going to the board and correctly working out a problem that we hadn't seen before. Only myself and 2 other people ever did it--in a class of 60.

Do you think that your company is solving problems so hard and paying so well that you can can only consider 3 out of every 60 developers (or whatever the real ratio is) who are otherwise qualified who have also mastered control of their nerves far beyond what is required 99% of the time on the job? Maybe if you're Google, for the rest of us we need to find a better solution.

By the way, even though I was able to solve problems at the board, I hated every minute of it, and I refuse to work for companies that require this kind of interview.

<comment><USER6> > I've never had to solve a difficult problem in 15 minutes while someone who > knew the solution watched over my shoulder and did their best to answer my > questions without answering them too well. Ohh yeah and the results of my > performance had potentially life changing consequences.

So you've never taken an exam in your entire life?

I don't mean to be snarky. I just don't understand this extreme disdain for coding interviews, even when they're very forgiving and flexible. Yes, exams aren't fun, but they're necessary.

Even if you can look up whatever you want, use whatever language you want, take as long as you want, do it right on a real desktop (not a whiteboard), and if you're asked a reasonable, real-world problem, not an algorithmic brainteaser, you'll still have people say it's a horrible process and it's unfair. And that's already way more forgiving than any exam I ever took in school.

I mean, are programmers expected to not be able to do anything at all in an interview setting? Like you can't be expected to produce any code of any kind, as if you don't know how to program at all? I think it _is_ a bad sign if someone who is a talented programmer utterly buckles under a little bit of pressure. To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

So what, if I hire you and at some point need you to hotfix something in, is that unfair? Maybe I do know how to do it myself but don't have time because I'm busy with something else. That situation doesn't seem all that different from the interview, except it would actually be higher pressure because there's a _real_ problem, not a fake one.

<comment><USER5> >So you've never taken an exam in your entire life?

You missed the context where we're talking about a professional setting.

>And that's already way more forgiving than any exam I ever took in school.

You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

>To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

And you'd be wrong. I've worked with very talented programmers who are absolutely terrible at interviewing. The problem is you're biased by your experience. You are probably good at this kind of interview and you are probably a good programmer, so you assume that any good programmer must be a good interviewer.

I know for a fact that these 2 processes are orthogonal. Books like cracking the coding interview exist because it is possible to practice and game the system. I've met just as many people who are good at interview questions, but are terrible software engineers, as I have people who are bad at interviews and are excellent software engineers.

The fact is that our current hiring processes are broken. Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this. Newspaper writers work on deadlines, but they don't have to write an article on the spot while someone watches them type.

Studies have constantly shown that work sample tests are really the only form of interview that shows real predictive power (that and general intelligence tests). The closer you can make the work sample to real work, the better. Finding the nth item of some arbitrary sequence in 15 minutes while someone watches your every move is _not_ close enough to real work to have much predictive power.

>So what, if I hire you and at some point need you to hotfix something in, is that unfair?

If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there. I'm also probably very familiar with the domain, and it's not some artificial problem that I may have never seen before. But more than that, why don't you see the difference between a hotfix and an artificial situation where someone who already knows the answer is standing there watching your every keystroke, and judging your value as a programmer?

There are better ways to interview programmers. Here is one:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

<comment><USER7> Going on a tangent, but:

>You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Yes. Is this not common? I've had a number of teachers who, as a graded form of evaluation, would ask us to present a topic to the class, and/or ask questions about it. This was extremely common on high school, and happened once or twice in the more advanced courses in my college. I also had to defend my thesis to graduate, which was _basically_ a more lengthy version of this.

I live in Argentina. Maybe it's a cultural thing.

<comment><USER5> Were you asked to actually solve new problems you hadn't seen before in front of the class?

> would ask us to present a topic to the class, and/or ask questions about it.

This is common in the US as well, but presenting on a prepared topic and solving novel problems are 2 completely different things.

<comment><USER7> I've had to do both, but presenting on a prepared topic was much more common. I don´t recall ever being graded on a novel exercise presented to the class, but I certainly have had to solve them.

<comment><USER5> I've had professors that like to require writing on the board as well. But as in your case the key difference is, they weren't graded.

That being said, a professor student relationship isn't analogous to an employer employee relationship. Hopefully an employer wants to make the interview process as enjoyable as possible to attract as many qualified applicants as possible.

<comment><USER7> I agree, the relationship is not analogous at all. That's why I said I was going on a tangent: I was more interested in learning about the evaluation system in the US than it's possible parallels with job interviewing,

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> This is a sentiment I hear expressed a lot:

> But I wanted to appear smart, so I didn't express any confusion.

And I don't get it. When I ask questions, if anything, I get told that I seem smarter for it. Asking questions is a sign that you're paying attention and actively trying to understand things. I don't think I've ever had anyone question my intelligence for asking questions for clarification.

(I'm not trying to criticize the OP in particular for this. I've heard it from a lot of places, and wanted to point out the fallacy.)

<comment><USER9> Yep. This has been the primary source of frustration for me in my communicating with software developer peers. I'll explain the task I'm delegating carefully: why it exists, the big picture of how it should be implemented, what tradeoffs I feel like will need to be made, and what corner cases are likely to exist and they'll nod like they understand to defend their ego then go and implement roughly none of it.

<comment><USER10> It makes testing for this behavior important, and it sounds like the OP exhibited just this behavior, doesn't realize it, and blames the interview technique despite it ascertaining exactly what it needed to.

</comment></comment></comment><comment><USER11> This is a tricky problem. Here are some things I've found helpful on the other side of the table:

\- Pick a real world task (e.g. text munging).

\- Suggest in advance that they bring a laptop with their preferred work environment. Allow any reasonable language if you're willing to train them on your stack.

\- Write the problem down ahead of time. Have someone else at your company read it.

\- Do the problem yourself. Ballpark your estimate for a _great_ candidate at 3x the time it took you.

\- Tell them there is no penalty for using stack overflow over the language docs.

\- After you give them the problem, give them a few minutes in peace (leave the room) while they think it over. Talk about their approach out loud before they begin working.

\- Ask them to spend 5-10 minutes re-factoring their code once it's working.

<comment><USER12> That 3x to do it yourself time seems excessive, but we once did exactly the same thing at a previous company. We chose problems we knew, worked with tools we knew, in a comfortable environment.

Even if you aren't intentionally trying to do the opposite, which unfortunately many interview techniques, like whiteboard coding effectively do, conducting interviews where a candidate is comfortable and at their best is ridiculously hard.

<comment><USER11> Agreed, but it's worth trying to do well. You get a lot of information out of how someone handles programming live vs. what you get in a homework assignment.

</comment></comment></comment><comment><USER13> Very nice write-up, I can just feel the stress.

Regarding the interview, there's this trick I learned that presenters do when they come on the stage. Instead of focusing on the audience and their talk immediately, they first drink a glass of water slowly. It lets you set your own pace, and deal with the first high of adrenaline a little.

In this case, they asked you a question and you immediately fired up a repl and started typing away as if it's a race. But if no one is holding a stop watch it's not really a race. If you're going to have to do this kind of interview more often, perhaps practice on your generic problem solving strategy.

If it were me, I'd probably ask for a piece of paper, and then write down the components of problem on the highest level, then drill down the details of each component. At each level of detail check with the interviewer if the requirements of the component match their intentions. When you're confident you've got the algorithm down on paper, then you can start on working it out in the REPL.

Also, I'm not a LISPy guy so no idea how you usually do this, but those first three mistakes you made were horrible. Obviously you were nervous, but these kinds of mistakes don't happen if you have a good IDE and disciplined code style. You can't miss matching brackets if every opening bracket starts a new line and a new indentation level. You can always clean up the new lines when your algorithm is done.

Practice and preparation are the best countermeasure to stress.

Not that I agree with this interview style at all by the way. You are most likely an excellent developer, and if they don't decide on hiring you they probably are making a big mistake. If I was hiring Clojure people I'd be surprised I'd get someone that would work a REPL like that without internet reference at all.

</comment><comment><USER14> I feel compelled to point out, as a native English speaker familiar with this sort of mathematics, that the phrasing to "go infinitely" doesn't make much sense. Or, if it makes sense, it is at least ambiguous (with plenty of room for the interpretation "diverges to infinity"). The clarification that the author makes about looping (or even better, repeating) is much clearer.

<comment><USER15> As a non-native English speaker who also interviews candidates, I try make sure to see if the candidate understood and always say it in another way -- For example after saying "go infinitely" (which is a silly way to phrase it) I would also say "it basically loops around", "creates a cycle". Maybe even draw out a little example on a whiteboard.

</comment></comment><comment><USER16> Thanks for the great post. The author, even if he were a bad programmer, is a great writer. The text grabs the reader and makes him sweat in the interviewee's shoes.

Live programming interview tests are rubbish, because:

a) They produce a huge rate of false negatives. People who do not "interview well" flounder in a programming task with someone peering over the shoulder.

b) They can only test "algorithmic" tasks (for lack of a better definition). Most programming is not algorithmic, it's higher level: glueing already written libraries, with very light data manipulation or validation.

<comment><USER17> I'm not sure I agree with you on this.

Live programming interviews should not be the sole quality of judging someone, but if we've first given them a coding exercise to do on their own time and they passed that, then it's useful to have them come in and do some coding to show that they can walk through their code and add additions/fix bugs to it without problem.

Which on that same note, I'm not sure why you would say they can only test "algorithmic" when we can do like I just described and have them for instance code a new feature in an existing code project of theirs.

That said, I agree with everyone else that they shouldn't be rushed, should allow the interviewee any and all resources they care to look up, and should shoot for "real-world" exercises over made-up scenarios.

<comment><USER16> I was speaking about live programming, during the interview. I much prefer asking candidates to write a task on their own time, and then use that code as basis for conversation during the interview. You get all the insights on how good a programmer he is, without the sweaty palms effect.

</comment></comment></comment><comment><USER18> No matter how badly I was doing in the interview I would walk out when I saw the engineer pull out his phone and start checking messages. How embarrassing for the company to have someone that disrespectful conducting interviews. An interview should be a conversation between potential peers and him pulling his phone out is just as rude as if you had pulled out yours because you were bored of hearing about the company.

<comment><USER19> I even do this IRL. If I come to visit you or you come to visit me then that takes precedence over social media. If that's not the case I'll be more than happy to walk.

It's rude beyond insult to do this during a job interview, whoever decided that was ok to do is basically sending you a very large warning message that even hiring is a job that does not require full attention.

<comment> alok-g Counter opinion: I have done this in the past to signal the candidate that I am _not_ watching over his shoulder, just to help the person relax as he figures things out. I feel glad now that I have done this after explicitly indicating this to the respective candidates.

<comment><USER19> Do you signal that in a verbal way as well or do you simply look away and start working on your phone?

<comment> alok-g Verbal way.

<comment><USER19> I think that's a key difference then.

</comment></comment></comment></comment></comment></comment><comment><USER5> The way I see it, you can master coding interviews by practicing solving small problems quickly under pressure. But this kind of practice takes time--time that is probably better spent mastering the skills you'll use day to day.

The studies that get brought out every time this comes up are very clear. The best predictor of future success is a work sample test. Solving a bitesize problem in 20 minutes while someone watches over your shoulder is not a work sample test--it's about as far away from a work sample as you can get and still call it programming.

At the _very least_ why not let the candidate work on the problem on their own for 30 minutes and then check back in.

I've said it before, the best interview process I've ever heard of is this:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

</comment><comment><USER20> I am horrible at these kinds of interviews.

I remember once being so nervous that I completely blanked. I'd been asked to reverse a linked list. A very simple task, one I'd even practiced a few times in various languages prior to my interview and should have been able to breeze through.

But I froze. My hands were visibly shaking as the interviewer stared at my back... I shut my eyes tight, trying to think through the problem (I _knew_ this was a simple task, and I was frustrated that I somehow couldn't recall the solution). At first I tried to write out a recursive solution to the problem, but my anxiety-riddled brain just couldn't cope with recursion that morning. So I erased the board and started over trying to find an iterative solution. That went even worse.

"Um, I need to repeat this block of code, I think, maybe. There must be some way, some kind of construct that causes a block of code to be executed several times, almost as if one were _looping_ through the code." AGH! _Think!_

But I was coming up empty, I searched my memory, ready to seize on the first seemingly helpful thing I could find. And so my mind wandered back to a time when I was about 8 years old, learning to program for the first time on my Apple II. I had just discovered the magic 'GOTO' command, which was the perfect tool to cause uncomplimentary messages about my older sister's hygiene to scroll down the green mono display for all eternity (or until someone hit Ctrl-C).

So... I completed my solution on the whiteboard using a GOTO. The interviewer put his face in his hands and began laughing. Eventually, he stopped and said "Um, ever heard of a for loop?"

"Well.... yes."

</comment><comment><USER21> This sort of interview is insulting. There are real problems to solve in business and elegant solutions to the 'next number in sequence' isn't one of them.

<comment><USER22> There's not really time in a 30 minute interview to pose a real business problem, much less have any meaningful representative work done, by which to judge the candidate.

<comment><USER23> Why would you only allow 30min at the stage where the candidate is writing code? 30min is for the phone screen, in-person is when you're serious.

<comment><USER24> Agreed. If I have you in the office I may talk to you for the rest of the day and I schedule accordingly.

<comment><USER23> And if you're LinkedIn you never communicate with them again.

</comment></comment></comment></comment></comment><comment><USER25> I have ranted before about the cargo-cult style used in hiring processes across our industry. Most of the "write-code-on-whiteboard" and "how-many- golf-balls-a-school-bus" style of bullshit has been handed down through folklore from old Microsoft interview techniques from the 80's. Now that I'm running a company, I've used the opportunity to define policies by basically banning those practices from our engineering recruitment process. Instead, we give a problem to the candidate (of the sort that we might be solving in day- to-day work ... not "create this data structure and be sure to use recursion horse-manure) and ask them to take the whole day or more if they want, in their own environment (home, cafe, wherever they find focus), as if this is a real problem they might be solving for work, and produce working code. Now I know this is not perfect, but I'm convinced this is way better than the traditional coding interview.

<comment><USER26> > I've used the opportunity to define policies by basically banning those > practices from our engineering recruitment process.

Good for you but FYI, brain teasers have been banned in most companies in Silicon Valley for a solid decade. The fact that they are no indicator of the quality of an engineer candidate is a well known fact.

<comment><USER27> > Good for you but FYI, brain teasers have been banned in most companies in > Silicon Valley for a solid decade.

I think you mean "specific, but highly visible companies" instead of "most companies." Having recently gone through a round of interviews, I can tell you the brain teaser is alive and well at quite a few hiring companies in SF and SV. I think people know they don't work, but hiring is difficult and most people are lazy, especially at companies where the interviewers would rather be coding.

</comment></comment></comment><comment><USER28> I think the OP made the mistake of trying to solve the problem directly, and when he got errors, he got nervous which made him get more errors. Instead the right strategy would be to think aloud "well, how do I get the digits of a number? first, cast the number into a string then try to see if I can treat it as a list. Ok, let's try (seq (str 10)) on the repl....". Having an instant feedback and being able to no exactly if a small instruction works or not would have helped him to focus better, and avoid nervousness.

<comment><USER29> that he did not take this approach makes me think the candidate is inexperienced, and probably coming to clojure from an imperative mindset.

<comment><USER30> Yeah, that's exactly what I look for in problems like that. How does this person approach the problem? How do they decompose the problem?

For that specific problem, I'd probably be satisfied if they wrote a function that took a number like 341 and returned [3, 4, 1], and then mumbled something about a set or a hash (or a list even) for storing the already seen numbers.

</comment></comment></comment><comment><USER31> On my first job interview, despite having already quite a bit of experience with personal and free software projects in various languages and frameworks, I somehow managed to struggle with FizzBuzz. I didn't know that it's supposed to work as a basic filter so I was looking for some catch thinking that it cannot be as easy as it looks. As I couldn't find any I figured out I should go with trying to write the prettiest, most elegant and clever way, started fiddling with modulo division and managed to screw it up under nerves and the fact that I was writing it on paper with no ability to run and check. Also, despite of being told "use whatever language you want", for some irrational reason I worried about correct syntax, which probably added up to my nervousness.

Fortunately, they didn't stop there and I was able to do harder tasks better later on ;)

I reacted pretty much in the same way on my driving exam. Both that interview and that exam weren't telling you anything about how I actually program or how I actually drive - even under extreme conditions. Being judged is just a completely different experience that makes me behave in completely different way.

<comment><USER32> I had an interview a few days ago, and just punched the problem I was asked to do in Python into iPython. Immediately did it a much simpler way, and also spotted my logic error (if you want deciles you actually divide by 9, not 10, since you have 9 columns with 10 edges).

Took me about 2 minutes to do. Whereas in the interview I wrote entirely the wrong thing initially, then went "oh wait, that makes no sense".

</comment></comment><comment><USER33> I remember that my first interview was kind of similar. Though it was a phone interview on Hackerrank Codepair.

I was really nervous, and since it was a phone interview, I tried to speak my thought process, which didn't quite work out due to me constantly stammering. So I just took to typing all the time so that my interviewer wouldn't think I was taking too long coming up with a language or copying answers. I started making silly mistakes like using `int sum=0` in python. I took the input as a number, converted it to a string, and then further converted it back to a number, and did mathematical operations on it, trying to solve the problem. Every time I thought I did it, it would get stuck in another case.

Ultimately, the interviewer deleted 50 lines out of my 60-line code, and just added a print statement on a variable I was already using for storing things temporarily, which turned out to be the correct answer.

Needless to say, I blew the interview. I haven't done an interview since, and I'm gonna be up for it very soon. I'm not sure if I'd do better this time, but I'd be ready with a glass of water, and try to be as relaxed as possible.

<comment><USER34> How long do you normally wait between interviews?

</comment></comment><comment><USER35> "I'm suddenly self-conscious about the prettiness of my code — it doesn't matter if I can solve the problem, since I'm also competing against candidates who can solve the problem, so the real competition is probably about how elegant and idiomatic our code is."

This is an incorrect assumption, at least when I interview people. A live coding interview has (almost) nothing to do with the "quality" of your code, and doesn't necessarily even demand complete correctness. It's about observing your process, and trying to simulate what it would be like to work with you:

Do you approach a problem with caution until you're sure you understand it, or jump in head first and start writing code you hope is correct? Can you verbalize your thought process and communicate it to others? How good are you at reasoning about the runtime behavior and correctness of your solution? Do you consider edge cases and naturally want to test them? Things like that are won't show up in a take-home coding test, and tell me a huge amount about what kind of programmer you are, and whether I want to hire you.

See: [http://www.yacoset.com/Home/signs-that-you-re-a-bad- programm...](http://www.yacoset.com/Home/signs-that-you-re-a-bad-programmer) \- there's a lot of truth to this article, and many of these flaws will be glaringly obvious in a live coding session.

And yes, of course some people interview poorly or are just having a bad day. That's unfortunate, but I'm not about to give up a powerful interview tool just because it can have false negatives. Until I find a divining rod that will point out the best programmer in a room, asking a candidate to write some code is the fastest way to learn what kind of programmer they actually are. (And also immediately weed out people who can't actually write real code.)

<comment><USER36> > A live coding interview has (almost) nothing to do with the "quality" of > your code, and doesn't necessarily even demand complete correctness. It's > about observing your process, and trying to simulate what it would be like > to work with you

I frequently hear people say things like this. Now maybe you do think this way and I'm not saying you don't, but I think that makes you exceptional. In about 80% of interviews I've done where I successfully completed the problem, I got an offer; and in 100% of interviews where I did not complete the problem, I did not get an offer. That seems to indicate pretty strongly that solving the problem is by far the most important thing for most interviewers.

<comment><USER37> I didn't take what he said to mean that solving the problem wasn't important, but that the quality of your solution (in terms of how 'elegant' or 'idiomatic' it is) wasn't important.

</comment></comment></comment><comment><USER38> Technical interviews always get your hopes up, and then you accept the job and over the next six months not _once_ does an engineer come into your office and ask you if you know what a Collatz number sequence is, then explain it to you and ask you to write some functions that would allow him to find the longest iterations through the Collatz sequence, given a set of starting numbers. It's practically bait and switch.

( _EDIT: this was actually a joke..._ )

<comment><USER39> I recently had a good long discussion with a fellow engineer about the number of sequences of coin flips with no streaks longer than length k. (turns out the answer is a generalized Fibonacci number!) So it /can/ happen...

</comment></comment><comment><USER40> I have a dream I'll one day be asked to interview someone, and get to try out my unconventional idea... (Or have it done to me one day.) I hate standard coding exercises, and I expect the candidate to have already been vetted for being able to code by either having some open source work or if none is available being given a take-home fizzbuzz-ish problem. My idea is to pick a problem off my own personal list of things I'd like to explore, things I'd like to reinvent code for from high level descriptions if I had more time/discipline (for example drawing voronoi diagrams), and do this alongside the candidate, so together we can attack the problem with a whiteboard and a shared computer or two for wikipedia/pseudo-code/real-code/etc. We may not get very far in just an hour, we might not even get to code, but it will be good information for both myself evaluating the candidate and the candidate evaluating me about how we collaborate on a problem neither of us knows the complete solution to in advance.

</comment><comment><USER41> Part of this was nerves, part of it was insecurity, and part of it was two people being bad at communicating with each other.

Bad situation, but I don't draw any conclusion from it. There are lots of inherently hard things in life that get messed up if you are nervous, insecure and can't communicate. That isn't _necessarily_ an argument against them.

<comment><USER42> Sex immediately comes to mind too.

<comment><USER43> Regardless of the topic at hand.

</comment></comment></comment><comment><USER26> My dislike of dynamically typed languages went through the roof reading through this article.

<comment><USER44> Your dislike for an interview question that tests knowledge of number theory instead of programming is what should have increased.

Writing Clojure functions under pressure is like gardening at 55 MPH. You're trying to make something still and beautiful and overdoing it (or missing entirely) as you drive by.

<comment><USER26> > Your dislike for an interview question that tests knowledge of number theory

"Number theory" is a bit exaggerated. This is a warm up question to see if the candidate is able to decompose number 530 into digits five, three and zero. If you can't do this in a few minutes, you're probably out of the running from the get go.

</comment></comment></comment><comment><USER45> Interesting article, but ironically the interviewee's solution is still incorrect at the end of his post. You'll notice that he only squares the first digit, and then adds the second digit rather than squaring it.

E.g. when he calls (happy 97) it prints 88. However, that's 9^2 + 7, when in fact the answer is 9^2 + 7^2 = 130.

</comment><comment><USER46> 2 things that I think would make your life so much easier..

1) Agree with the interviewer before starting to code on the algorithm and problem. I.e. You could say I need function X that would be used like such, and function Y that would loop through such and such. Even better is drawing a high-level example on the board. That way, the interviewer knows that you understood the problem and can be systematic about it. (If you don't have a whiteboard, just write in comment at the top, whatever)

2) Once you start coding, start small.. especially in the REPL. I.e. instead of starting with the whole function, start with the string, then separate it in digits, then sum it up, then work from there. That way, if something doesn't compile, it's perfectly clear what step it was and once you have your function, you're 100% sure it works. TDD style.

</comment><comment><USER47> Going on a tangent, but that Clojure code looks incredibly convoluted. Compare to this:

        next = (n) -> "#{n}".split('').reduce ((p,c) -> p + c*c), 0

        happy = (n, set = []) ->
            return false if n in set
            return true  if n is 1
            happy next(n), set.concat(n)

        console.log(n, happy n) for n in [1..100]

That complexity certainly makes it harder to solve the problem under pressure. This is really an example of an interview process that focuses on specific language skills and not actual programming abilities.

<comment><USER48> What language is this?

<comment><USER49> CoffeeScript.

</comment></comment></comment><comment><USER50> This kind of thing is why the best "code interview" I ever had involved being given a problem (in this case, a small message sending/receiving library) and told to work on it over the weekend.

<comment><USER51> This type of work-intensive interview gets old real fast when the umpteenth company declines. I fully understand the pros of looking at a more comprehensive, realistic body of work. But if that's what you're looking for; I got a whole Github profile full of stuff to discuss. Spending every weekend on yet another throwaway application until you get hired; I'd rather not.

EDIT: removed some bitterness. :)

<comment><USER52> I would assume that a longer programming task would only take place in the very final rounds of interviews, where it's clear that the candidate is a potential hire and the company just wants to be sure that the candidate can do non-trivial work, walk through his code, have it reviewed and discuss the problems found.

Starting with a programming problem that takes hours is simply being lazy from the company's part. It's easy to hand out and review but a lot of work for the candidate.

<comment><USER53> that's the dream but there are some companies which send out their pre-baked "weekend projects" after the initial hr screening and I'm talking about large corporations (the new york times for example) which isn't terrible at first but results in lots of trying to look up what practices the company in question follows and prefers and what they are actually looking for

personally I really need human feedback rather then 8 hours at my desk guessing and searching for what's being looked for

it's one of those situations where I see the value for the interviewer but as the applicant it's completely sucks especially when you have 4 sample projects to write at home while juggling phone interviews and you know having a life/job

</comment></comment></comment></comment><comment><USER54> I have very little love for code interviews of this kind. Here's an alternative I would propose to hiring managers:

Give the candidate a research paper in PDF, then ask them to implement the algorithm described there in code.

You get the same feedback on whether they can actually code, and what kind of code they write, but it takes the pressure off having to think of what kind of algorithm to use, what the big O complexity of it is, and a thousand other factors that will probably never relate to what their day job will consist of.

<comment><USER55> For some companies/teams/hiring managers/interviews, having someone who can reason about big-O complexity and design their own algorithms is important and necessary. There won't always be problems that have a preexisting algorithmic solution.

</comment></comment><comment><USER10> The important mistakes that the author here made were assuming the interviewer wanted him to work quickly, without looking anything up, without asking for clarification, and without collaboration. The author seems to still be making this mistake by assessing his performance by the code he wrote, rather than the (lack of) dialogue he had with the interviewer.

Consider that the interviewer has now seen that this person will choose to, without clear understanding of either the goals of the task or the task itself, work alone as though he must solve the problem in isolation in order to prove himself.

The most important thing that a candidate needs to be able to do on the job is discuss expectations; failing to do this in the interview is more egregious than any compiler error. Another important thing is to understand the problems they are working on; failing to even discuss the problem is more egregious than any compiler error. Another is collaboration and humility; you can demonstrate these skills in a coding interview as well.

This interview demonstrated many more important indicators of a poor fit than typing ones. Just because you pose a coding problem doesn't mean you're unable to assess candidates on the things that matter.

</comment><comment><USER56> Awesome post. You have some really good memory! I wish I would see more post interview analysis in this depth here in HN. Great read.

P.s. You are not a bad programmer... Pressure can mess your mind. My secret to finding great developers that everyone else misses is by making them feel calm (giving them an easy problem and giving them encouragement so interview stress becomes less a factor, best way to find good developers who just are bad at interviews)

<comment><USER34> Not OP, but I have going through a string of interviews. Hopefully I can post a couple soonish.(In the middle of studying for another one)

</comment></comment><comment><USER57> I took 2 different tests for 2 different companies.

One was a very simple task, setting up a server as a customer would at an ISP using WebMin type tool. Really simple stuff. Guess what. I couldn't figure it out in time. Maybe because I had been consciously staying away with gui stuff to stay sharp with cli. Or maybe the proprietary tool the ISP was using was very unintuitive. Maybe the suit and tie I put on for the first time in 2 years got to me. Either way, had I NOT been in an interview setting, I would've figured it out. But with this interview, I couldn't figure it out.

In the 2nd company, I was giving an online test using [https://trueability.com/](https://trueability.com/), to complete from home. Set amount of time to complete some test scenarios. Far more difficult stuff than setting up a small web server at an ISP. And I would say it was a far more realistic test of my skills.

IMHO, interview should really only focus on compatibility, not actual tech skills. If your company is really spending time on tech skill test DURING interview (especially onsite), you are doing it wrong. just my 2 cents.

<comment><USER58> I hate TrueAbility. It gives you so little time to do the test, and for example the Linux skills test is really about your intimacy with the dark corners of an outdated version of CentOS instead of anything current. Then, if you bomb a test, it doesn't allow you to try again.

<comment><USER57> I love TrueAbility. You are not expected to finish 100% or even 80% of the given tasks completed during the test. I read online somewhere that he/she got barely 50% done but still got an interview. The recruiter giving out the test on TrueAbility is looking for how you solve the problem, not how much of it you solve.

Also, the way I understand it, you can keep a personal profile on TrueAbility and point recruiters to it for them to review your tech skill level.

</comment></comment></comment><comment><USER59> I'd be more comfortable coding something more concrete and familiar than Collatz and Happy number sequences personally.

<comment><USER19> You're liable to see Collatz sequences and such in interviews because the interviewers can look up the answers to project Euler problems online to see how your solution compares to those coded up by others.

</comment></comment><comment><USER60> I realize that the interviewee was in time pressure, but really should've made smaller helper functions - probably would have helped ease the stress a lot. Possibly also unfamiliar with Clojure/functional programming as someone mentioned, given that his "happy" function uses loop unnecessarily. Here's my solution.

         (defn int-to-digits [n]
           (map #(Character/getNumericValue %) (str n)))

(I had to Google Character/getNumericValue - doubt anyone expects you to memorize that).

        (defn next-happy [n]
          (reduce +
            (map #(* % %) (int-to-digits n))))

        (defn happy? [n]
          (loop [n n seen #{}]
            (if (contains? seen n)
              (== n 1)
              (recur (next-happy n) (conj seen n)))))

A lot simpler to reason about, even if the initial implementation did have bugs.

<comment><USER43> _" really should've made smaller helper functions"_

I couldn't agree more. Especially when you're under pressure, bite off smaller pieces. Not only is it possibly better practice in general, but in a time of diminished cognitive capacity (such as when much of your bandwidth is being eaten up wondering how you're coming off) they will be easier to get right, and small successes will help ease your stress as much as they help solve the problem.

</comment></comment><comment><USER61> I related to this a lot. Having somebody watching over my shoulder (especially in a judgmental capacity) means that when I inevitably make some stupid minor mistake, instead of just laughing it off as I normally would, I get anxious. This then increases the chances that I will make another mistake, and so on.

</comment><comment><USER49> CoffeeScript solution: (sorry, couldn't resist trying!)

        happy = (n) ->
          sum = 0
          while (sum += Math.pow n%10, 2 ; n = 0|n/10) then
          sum

        happyness = (n, s=[]) ->
          (n is 1) or (n not in s) and (happyness (happy n), [ s..., n ])

        console.log n for n in [1..100] when happyness n

Edit: and a JavaScript one:

        function happy(n) {
          for (var sum = 0; n; sum += Math.pow(n%10,2), n = 0|n/10);
          return sum
        }

        function happyness(n, s){
          s || (s=[])
          return n==1 || !~s.indexOf(n) && happyness(happy(n), s.concat(n))
        }

Edit 2: And the golf version:

        function h(n,s){for(s=0;n;s+=Math.pow(n%10,2),n=0|n/10);return s}
        function H(n,S){return n==1||!~(S=S||[]).indexOf(n)&&H(h(n),S.concat(n))}

</comment><comment><USER62> I think the author may be missing the point of this exercise. Solving this problem requires the following things:

1) Decompose a number into digits. Less experienced candidates will likely convert the number into a string and then the characters back into numbers while more experienced candidates will tend towards a small loop using div and mod.

2) Use some kind of library to insert and test membership in a set data- structure. Less experienced candidates may not be able to do this without stackoverflow, big red flag.

3) Some basic flow control wrapping it all together. Less experienced candidates may fail here similar to fizzbuzz.

The interviewer likely thinks these things are a good proxy for the quality of a candidate. For example, the author would have failed #1 even without miscommunication/stress/time constraints.

</comment><comment><USER63> An interview that forces you to do something that is otherwise not part of your daily routine and measures how good you are based on that is bad. I understand that you need to get insight into the skill of the candidate but forcing her to work on an editor she does not know, giving limited time etc. is probably not the best way to get that. I used to interview software engineers for a big company and we usually gave them a problem that they could solve in 48 hours using anything they want (internet, library etc.) and we just asked them to explain their solution in person during the final round of interviews. You are interested in how good the candidate is not how fast she can fail a synthetic test that has nothing to do with the actual work you are doing.

</comment><comment><USER64> It's weird. I have the opposite experience. I do not have a CS degree. I do not know formal algorithms or data structures. But I have technical aptitude and significant _stage time_ , including in front of drunk audiences from another life that didn't exactly lead to stardom. So I _enjoy_ technical interviews because what limited knowledge I have is instantly accessible and I thrive in front on an audience.

So it seems here is a an instance of where life was not fair... in my favor. But now I'm in a position to hire other people. So I don't want to judge them in the same way I was. I want to find out who's got the chops completely apart from who can dance on cue.

</comment><comment><USER53> I don't fully understand the direction you were going with some of the solutions, something like this would work no?

(defn digits [num] (let [ x (map str (str num)) s (map read-string x)] (apply + (map #(* % %) s))))

I do think the interviewer should have pointed you in the right direction or offer some sort of hint to gain an understanding of how comfortable you are with the language rather then just having you fumble around but depending on the role they were hiring for these questions and this practice don't seem bad to me

</comment><comment><USER65> Looks like more of an advertisement against Clojure than anything else.

<comment><USER66> At least like an advertisement against coding anything directly in a REPL.

Typing in an editor (probably with Send-to-repl functionality) would have caught a lot of errors. But yes - doing it with a proper IDE or even a statically typed language would really had helped to focus in the problem and not on getting-it-to-run.

</comment></comment><comment><USER67> Out of curiosity, what's the best solution to this? One thing I noticed is that every loop will pass through an entry less than 100, so you only need to remember the last 99 numbers below 100 when checking what's been seen-so-far.

You could also precalculate or cache the answer for 1 through 100 to save rerun time.

But maybe a little bit more math would reveal much better optimizations.

</comment><comment><USER68> I once had to implement a simple API during an interview. I'd done it many times, but the stress of having an interviewer breathing over my shoulder made it a painful experience and I completely flopped. Not to mention there was another interview trainee in the room.

I can write white board code with ease as those are almost always two way conversations

</comment><comment><USER69> Hey man, I get terribly nervous too. I've gotten just a bit better with time, if you ever want to talk about it let me know :)

Skype n_goles

</comment><comment><USER70> In scala:

        def happy(n: Int) = n.toString.map(_.asDigit).map(Math.pow(_, 2)).sum.toInt

        def find_happiness(n: Int, seen: List[Int]): Boolean = {
            if (happy(n) == 1) true
            else if (seen.contains(happy(n))) false
            else find_happiness(happy(n), seen :+ happy(n))
        }

I think OP's answer is wrong

</comment><comment> bra-ket that's why the top topcoder coders use java under pressure

</comment><comment><USER71> Are there any good IDEs for Clojure? If so, most of the stumbling could have been avoided. Coding without one tends to fall into the "ball-of-mud" problem where wrong code is built on top of other wrong code. Or alternatively, is there any way to continuously compile Clojure?

<comment><USER72> Emacs is good for this. A buffer of clojure code in cider-mode can be (re-)compiled easily when you're jacked into a running session (as he does above).

Of course, some of the issues the author faced can't be solved by Clojure's compilation step. When his code fails at runtime in the repl, it compiles successfully.

</comment></comment><comment><USER73> A lot of the trouble here comes from Clojure's cryptic function names and non- static-typing making it hard to get work done. The only real mistake I see a unprofessional choice of language. The mistakes and successful debugging are totally fine.

</comment><comment><USER74> I won't put a candidate in a situation which I myself won't want to be in.

While interviewing, I tell the candidate the problem and then leave the room. He is alone and comfortable (I hope). I check back after 30 min. If not done I still give him more time.

<comment><USER58> Pay me a fair wage for my trouble, especially if I succeed at the problem, and maybe I'll feel a lot better about the coding exercise.

</comment></comment><comment><USER75> What helped me I think was going to meetups at the local hackerspace and preparing presentations involving live coding without previous preparation.

The cost of screwing up there is small and yet its excellent exercise in coding in front of an audience.

</comment><comment><USER76> Another idea, if you really hate the pressure of interviewing, is to ask for a medium-sized project that you'd hand in the next day. That's more real-world scenario, anyway. Try it - I've done this!

</comment><comment><USER77> I find the % operator (div and mod) essential - and not only in interview questions.

As far as this interview question is "Do you %?", I like it as a test. For the rest, I feel sorry for the author. Good luck next time :)

</comment><comment><USER78> once i interviewed at a company because my current employer was under aggressive take over from Microsoft.

in the interview i fluked a question about js scope because the whiteboard handwrite of the interviewer made it hard to see which variable was i and which was j... but after he explained why i got wrong and i realized the letter mistake i was too stressed and dumbfounded to make a case, so i just left.

6 mo later, my current employer is still independent while that company is now owned by Microsoft...

</comment><comment><USER79> It's quite clear that a lot of your pain was caused by Clojure's type system. Those exception messages are something you'd not expect in the 21st century.

</comment><comment><USER80> Your code isn't embarrassing (its just rushed/nervous). Their interview technique is embarrassing.

</comment><comment><USER81> Euler addicts.

Although some function decomposition of the problem would have been nice to see!

</comment><comment><USER82> When we do coding at our interviews we do ping-pong pair programming. One of the things we want to see is if the person can deal with actually doing TDD. Usually there are 2 of us and one candidate. We explain the problem to the candidate and ask, "Do you want to write the first test, or shall we?". We've done the problem hundreds of times so we know a very, very simple first test that will lead forward. This allows the candidate to "warm up" a bit before they have to start thinking hard about the problem. Sometimes they hit the ground running and are keen to write the first test, which is also fine.

The idea is that we switch back and forth. One of us writes a failing test, then the candidate writes the production code. The candidate then writes the next failing test and one of us writes the production code and the next failing test. If the candidate gets stuck, we usually try to talk through the situation and if they are _really_ stuck, one of us will offer to take over.

The candidate is allowed to look things up on the internet and even ask questions. Often we will suggest things. For instance, after the candidate writes some production code, we might say, "That works well. But how about refactoring it using inject," for example.

I often use a pomodoro timer set to 20 minutes. We'll do as much as we can in the first set. After that first set we'll ask the candidate how they think it is going. We'll offer some advice about technique and such. If it was an absolute disaster (happened a couple of times), we'll just leave it there and thank the person for their time. Otherwise we'll ask if they are up for another round. Usually the first set is pretty bad because the person may never have done real TDD before, or may find the ping-pong awkward at first, or may be very nervous. Talking about how they are feeling at the break usually helps and the second set is much, much better.

After the second set we'll ask the candidate how they think they did. I think this is important because if the candidate says, "It went really well" and it was actually awful, then we have a pretty good idea of their skill level. If they say, "I wasn't really happy with my performance, but I enjoyed the style of programming", then we will have to take into consideration a variety of different factors to see if we believe that the person can do better. If the person says, "I really didn't like working that way", then we can rule them out as a candidate regardless of their skill level (also happened a few times unfortunately).

The feedback we've gotten from candidates on the process seems to indicate that they really enjoy this interview style. I think it removes a lot of the spotlight from the person and allows them to relax a bit. Especially having us come in and help out in sticky situations allows the candidate to get over stupid blind spots (like the converting between string and integer in the article). So far, I think it has helped us spot potential in a few people that I'm sure we would have overlooked using a more standard, "Here, solve this problem" approach.

On the downside, there is still considerable room for improvement. Language lawyers (know every detail about the programming language but are still poor programmers) can often look very, very good in this kind of test. Similarly, inexperienced but very talented people can often be frustratingly mediocre because the unfamiliar method of development can hamper their style. You need to be very aware of the amount that the person improves during the interview.

One final comment about this approach. We have occasionally been in a situation where we said, "I'm not sure how good this person is. They seem like their skills are fine, but whether they will be really excellent, I don't know. However, it was _really_ fun programming with them." In those cases we have hired the person and never regretted it.

</comment><comment><USER83> Also, the interviewer didn't even know his own question.

As I understand it, you are taking the sequence

n -> (sum of the squares of the digits of n)

First, the sequence never diverges to infinity. For example, 9999999999 -> 9*81.

Second, not all numbers converge to 1. There is a stable cycle that does not include one.

Consider

4 -> 16 -> 37 -> 56 -> 61 -> 37 -> ...

So, any positive integer either

1\. Converges to 1

2\. Converges to the stable cycle I just mentioned above.

So the interviewer should be fired for not understanding the question he was asking.

<comment><USER84> Interviewer said it would converge either to one or infinitely:

 _' The next sentence of English is where the problem started. I _thought_ he said:

"The sequence will either go to one or to infinity."

but he actually said:

"The sequence will either go to one or infinite_ly_."'_ [My emphasis]

(Firing's also a bit harsh for getting an interview question wrong)

<comment><USER83> That also is incorrect - the sequence either converges to one or a stable cycle - not infinitely.

That's also a horrible question for illustrating functional programming style. It would be much more efficient to have a simple loop caching the array n->f(n) and then finding the ones that converge to 1.</comment></comment></comment>


The Do-It-Yourself Economy - <AUTHOR> http://www.nytimes.com/2009/12/13/opinion/13friedman.html?_r=1

<comment><USER2> The marketing agency owner claims that no jobs were created by his 80% off video production. How many are employed by voices.com? How many voice actors can now find steady work instead of waiting tables while they wait for that occasional $500/hour gig (of which an agent likely takes a huge cut)? How many semi-pro photographers are making some money to pay for their gear by shooting stock for iStockPhoto on weekends? Previously, a photographer would have to find representation to sell stock work -- a hard thing for somebody who doesn't produce a lot of work. I could go on and on. Creative destruction brings longer-term happiness.</comment>