Ask HN: How to prepare for tech interviews? - <AUTHOR>

Over the next three months, I plan to start interviewing for software engineering positions. I have read Steve Yegge's long rant on the art of interviewing. In the interest of acquiring more information, I was wondering if folks in HN could share their previous experiences in how they prepared for a tech interview?
Note that I am fresh out of school with a rather sparse github profile which means I am likely to be grilled on algorithms and data structures rather than anything else. I have done a few interesting internships that I can talk about though...
One specific question I had for folks who have been interviewers is: I kind of suck at coding on a white board. I feel that I can think better on paper and pencil rather than on a white board with someone staring at me. How would you feel if a candidate came in with a bunch of papers and offered to sit and write code next to you (instead of coding on the white board) <comment><USER2> The best way to prepare for interviews is to understand completely what you have worked on; be it internship or hobby project. Communicating clearly what you have learned is the best thing you can exhibit. Rest can be learned</comment>


Taking Back Memory from Vmmem/WSL - <AUTHOR> https://blog.simonpeterdebbarma.com/2020-04-memory-and-wsl/ <comment><AUTHOR> How to reclaim RAM from Vmmem when you're not using Windows Subsystem on Linux)WSL.

 __TL;DR __Takeaway

WSL does use a really low amount of RAM but it is allocated about 4GB by default when it is started. If you think that's too much and would like to decrease it, Windows Insider Build 18945 brough customizable settings for wsl.

Create a %UserProfile%\\.wslconfig file and use it to limit memory assigned to WSL2 VM.

```

[wsl2] kernel=<path> # An absolute Windows path to a custom Linux kernel.

memory=<size> # How much memory to assign to the WSL2 VM.

processors=<number> # How many processors to assign to the WSL2 VM.

swap=<size> # How much swap space to add to the WSL2 VM. 0 for no swap file.

swapFile=<path> # An absolute Windows path to the swap vhd.

localhostForwarding=<bool> # Boolean specifying if ports bound to wildcard or localhost in the WSL2 VM should be connectable from the host via localhost:port (default true).

# <path> entries must be absolute Windows paths with escaped backslashes, for example C:\\\Users\\\Ben\\\kernel # <size> entries must be size followed by unit, for example 8GB or 512MB

```

Therefore, in your situation, all you need to include in your file is ⤵

`

[wsl2] memory=1GB

`

Customize it as necessary with the available options.</comment>


Message Queue Evaluation Notes - <AUTHOR> http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes

<comment><USER2> N.B.: this page wasn't updated heavily since April 2009; a slight revision was made in January 2010.

<comment><USER3> +1 to this, a lot has changed since this was written.

</comment></comment><comment><USER4> I'm curious what the motivation for the potential "hundreds of millions" of messaging queues is here. having a queue per-registered user seems, I don't know, maybe overkill? Per group chat makes a lot of sense, but I'm having trouble reconciling the numbers based on what was posted. What am I overlooking?

<comment><USER5> Second Life involves a lot of per-user message sending. IM, group chats, friend requests, marriage proposals, dialogs with in-game merchant systems, payment notifications, persistent announcements, and a lot that I'm forgetting. They're all roughly the same thing -- messages that need to be delivered to either a specific user or a group of users, with flags for whether they get discarded when the user is offline or whether the user has to explicitly acknowledge them.

It was probably overreaching for us to try to find a system that would handle all this for every user. But even hundreds of thousands of current users, the bare minimum we could expect to be concurrent, were not well supported by the existing solutions.

</comment></comment><comment><USER6> Can someone more familiar with these products explain why they decided against ZeroMQ even though it apparently offers by far the best performance?

The ZeroMQ "note" part sounds like there's a bug in the Python library, but that hardly seems to be grounds for eliminating it completely from the "race".

<comment><USER5> I'm the primary author (formerly Which Linden). Surprised to see this pop up here after all this time!

We wrote this almost three years ago, and at the time ZeroMQ was substantially less mature than it is now. In particular, the Python bindings were broken, weren't maintained by the ZeroMQ organization, and there was no indication they'd be production-ready by the time we needed it (which would have been within the next month or two).

More importantly, we were looking for something that handled our use case a little bit more fully. Our message-rate requirements were laughably small, so what we really wanted was software that gave us an "inbox" for each of millions of users. We also weren't looking forward to implementing failover, consistency, and sharding from scratch.

Edit: corrected timeline.

<comment><USER7> Do you know what they ended up settling on?

<comment><USER5> No out-of-the-box solution was used. The system remained a menagerie of different messaging techniques (database, udp, in-memory storage), and the company just spent time improving the reliability and performance of these existing systems individually.</comment></comment></comment></comment>


Ask HN: I want to build a better content discover platform.  - <AUTHOR>

Tell me what you hate, what you love and if you think i'm crazy.<p>Appreciate all your feedback on this, it's extremely valuable. <comment><USER2> I love:

\- insightful personal stories

\- thoughtful reviews and perspectives

\- great visualizations of interesting/compelling concepts

I hate:

\- Clickbait ("You're never going to believe THIS! Please, don't do this, it's insulting to the reader. Tell us what it's about.")

I wish:

Things were curated more effectively. Think about all the /top and /gilded sections of subreddits. I would like all the best information about all the interesting topics, curated, analysed, interpreted, centralized, maybe in a wiki-ish model.

Cheers.

<comment><AUTHOR> Thanks for your 0.02, there's a couple gems in there.

</comment></comment><comment><USER3> I'm interested mainly in tech articles (libraries, software projects) that I might use. Restricting to articles that contain (or link to) python or js code could be very useful for me.

I'm also interested in particular industries and "topics" which could be identified using standard topic modelling techniques (e.g. LDA, which is used by getprismatic)

</comment><comment><USER4> What I would want is a WAAAY more decentralized platform as opposed to something like Medium or Svtble where its all on their infrastructure and to their site. Something like HN, except where it would be more acceptable to post blog posts (with curation of course)

</comment><comment><USER5> I've recently been toying with a few ideas and was thinking about building a discovery platform. Would love to co-found or work on one. Feel free to email me - firstname.lastname@example.com</comment>


Ask HN: Let me solve your problem #1 - <AUTHOR>

What: I will solve any technical problem you have and give it to you for free (within reason, for example, if it requires any purchase on my behalf I would expect to at least recoup it, but I will not ask anything for my development time). It can be anything, hardware, software, service, etc…
Why: I want to increase my Computer Science / Electronics skill-set by doing things that matter instead of just "toy projects". I want to solve an actual pain for someone.
How: Post a problem you face and elaborate a bit - I will try to better understand the cause of it. Users that also have the same problem please upvote so that I can see it's not a single-man case (see the "why"). I will chose one to work on based on the popularity and my ability to deliver on it.
When: I will work part-time during workdays and full-time one day over the weekend to deliver a solution. <comment><USER2> It's good that you want to improve your skills, but this is not the way to do it.

You should work on an open source project.

There are plenty of large projects that would be thrilled to have a developer who wants to tackle something more than a small bug, but you have to go find them.

Moreover, open source projects are much more deserving of your talent than the people posting on this thread trying to get some free work out of you. Don't undervalue yourself.

What are you interested in?

Personally, I have contributed to Python, Django and AngularJS because I use all three on a daily basis. Think of something that interests you and I'd be happy to help you find a project that fits your goals.

</comment><comment><USER3> There's plenty of tickets on Snowdrift.coop ([https://snowdrift.coop/p/snowdrift/t](https://snowdrift.coop/p/snowdrift/t)), any of which could present a problem.

If you want something particularly challenging, that's off the main development path (so if you fail or take a long time, no one's waiting on you), but that would be useful, one thing I'd like to see is the Snowdrift comment system peeled out as a separate library that provides a Yesod subsite, with sufficient hooks to retain existing integration.

Happy to point you at resources and explain what's currently going on.

</comment><comment> alok-g I am interested in using Deep Learning for several Windows Desktop projects but am struggling a bit due to dependencies in various open-source libraries which are often built on Linux platforms.

I have attempted to use Caffe, which has an unofficial Windows port, which nevertheless is failing to compile for me due to some issue. I have looked at DeepLearning4j, but I am again confused by the setup on Windows since it uses some native libraries for BLAS and provides little help on how to set it up. I am now looking at CXXNET, which looks promising since the source code looks very nicely organised.

Another issue I face is that most libraries come with MNIST, ImageNet, etc. examples where the training data is already pre-processed, and I find practically no examples of how to plug my own data. (In fact, the specific problem I am interested in may require modifying the library a bit.)

I am unsure if this work will be of help to you, though I can imagine other folks on Windows who may benefit from this.

</comment><comment><USER4> I have a number of clients who need to use up their budget for the year at the end of the year for technical goods and services. Each has a "wish" list of technical goods and services that they would like to pay for, some of which have taxes and other fees includes with them. There are no hourly fees, all of the services and goods are fixed cost, so that's good at least. But the lists can be really long and they aren't in a particular order. The problem is that my clients lose their budget if they don't use all of it exactly. So basically I need a solution that can pick the goods and services and calculate the taxes and fees so that the amount comes out to the exact left over of their budget. Of course, sometimes the budget will get changed by some annoying accountant mid day, so they have to be able to re-run it regularly. It's a real pain!

<comment><AUTHOR> >I have a number of clients

Could you tell me who are your clients or at least what sector they are in?

>Each has a "wish" list of technical goods and services

Could you give an example of the goods and services?

This sounds very much like a knapsack problem! ([https://en.wikipedia.org/wiki/Knapsack_problem](https://en.wikipedia.org/wiki/Knapsack_problem))

<comment><USER4> Ha! It is :) well done. I put it in as a thought experiment and a joke. However, believe it or not, I actually do get these types of questions a lot from clients mostly in the publishing industry. They mostly want to spend all their budget down to the penny on things like new fonts and fixed price graphics sets.

At any rate, good job on catching my joke :) and I don't expect you to work on it. I hope someone gives you a real problem to try out!

</comment></comment></comment><comment><USER3> Another thing I'd like to see is a notification daemon with two features missing from all the existing notification daemons I've played with:

1) Ability to pull up a history of notifications

If I go away from my computer for a bit, I'd like to quickly review what happened while I was gone. Also, sometimes I want to reference something that appeared in a notification.

2) Batching of notifications by priority

When I'm trying to get work done, I don't want to be bugged every time anything happens. I'd like to be able to say, "low priority stuff, don't inform me more often than every two hours; middling, every half hour; high priority stuff send through right away". Ideally, some sort of pattern matching to adjust priorities.

This could take the form of a new notification daemon (would need dbus and some UI library) or a patch to an existing one.

</comment><comment><USER5> Make Arc use all cores. Although it's already multi-threaded, all threads use a single processor. Or make Arc scale out on multiple machines by modifying its webserver to shard based on username.

If this works it can run HackerNews.

</comment><comment><USER6> What about a way to come up with the best lineup in Fanduel (daily fantasy sports game)? It is also a Knapsack problem. To give you a little background, each night you get to build a fantasy team (2 PGs, 2 SGs, 2 SFs 2 PFs, 1 C) within a salary cap. Each player is given a salary for the night and they are given points based on their stats like getting rebounds or assists. To determine what players should perform the best on a certain night, you can use a homebrewed algorithm or get a projection from sites online that project points.

<comment><AUTHOR> To clarify, do you mean something like this: [http://www.rotowire.com/daily/nba/optimizer.htm](http://www.rotowire.com/daily/nba/optimizer.htm)

</comment></comment><comment><USER7> \- A free chrome extension that lets you screenshot a page and copy to clipboard or upload to imgur (without watermarks etc) \- app that monitors flights for very flexible weekends (eg. a weekend stay in boston for anytime this spring) \- app that tells you which airplane flights are visible in the sky above you right now \- chrome extension that lets you clip images directly to your google drive/dropbox \- an in-browser way to send people images securely with client-only encryption/decryption

</comment><comment><USER8> Is it really a good idea to undervalue yourself like this? I feel like problems that "matter" couldn't be solved by a single person in a single day (or even a few days), otherwise they would have already been solved.

Just a thought.

<comment><AUTHOR> By problems that matter I don't necessarily mean big technical problems, just something that you would like to have and would make your life easier, but don't want to invest time yourself to make it.

</comment></comment><comment><USER9> My LL(k) recursive descent parser cannot handle left recursion without incurring in a O(n³) worst case time-complexity. OP please fix. I expect a solution until next week. Thank you.

</comment><comment><USER10> The posters do not seem to understand you. Don't worry. At least I think I do.

Can you post which are your preferred tools, which technology you'll use to solve the problems?</comment>


Meet the supermouse bred by geneticists that can't get cancer - <AUTHOR> http://www.dailymail.co.uk/pages/live/articles/health/healthmain.html?in_article_id=496751&in_page_id=1774

<comment><USER2> Also [http://www.biologynews.net/archives/2007/11/27/cancerresista...](http://www.biologynews.net/archives/2007/11/27/cancerresistant_mouse_discovered.html)

I haven't found the original paper or much information about the gene in question.

</comment><comment><USER3> Soon my pet mice will have nothing to worry about but stroke and heart disease.

Actually, I think cancer has been cured in mice several times over. The problem is that mice have a two to three-year lifespan and several orders of magnitude fewer cells than a human. So cancer in mice is very, very different from cancer in humans. Mice don't even _get_ the kind of cancer that concerns us -- the kind that develops after dozens of years, and that can evade a human's anti-cancer mechanisms (which are more powerful than those in mice, because protecting humans from cancer is a much harder task!)</comment>


HTML5 game: Adslife - <AUTHOR> http://adslifegame.com

<comment><USER2> Make "HTML5" game, only present the video with flash

I get a 'plugin missing' message when I try to watch the trailer

</comment><comment><USER3> I'm a fan of that style video trailer. Thoughts on the tools that are used for that sort of video production? Figured by now there's specific apps that are focusing on those visual styles. Oh, and, the game looks interesting as well. HTML5 FTW.

</comment><comment><AUTHOR> See also this "museum" of user contributed ads: <http://adslifegame.com/m>

</comment><comment><USER4> Hey! but this is a Mad Men history!</comment>


Ask HN:  Why don't ads put you inside them (e.g., as the driver in a car ad)? - <AUTHOR>

Wouldn't that get your attention?
Facebook for example could use pictures/videos of you and some advanced image analysis to construct such commercials automatically.
Taking it further, Facebook could use pictures/videos of your friends for other characters in the commercial.  Logos for organizations that you have been associated with could be used as well (e.g., as stickers on that car). <comment><USER2> A similar marketing campaign was attempted by a bank here in Brazil once and backfired badly.

When a client logged into his online account page, instead of showing "Banco do Brasil" (the bank's name) it'd show "Banco do <Client's-First-Name>" (Bruno's Bank -- if the client was called "Bruno", obviously).

A few hours after this seemingly clever personalization, there were _several_ national news sites talking about the new feature and how people were reacting to it: Some people couldn't understand this would only be visible to their session, they thought their name was showing up to everyone and wanted it removed ASAP.

Bottom line is: marketing acts should take into account how savvy the target audience is.

<comment><USER3> Tech savvy or not, that campaign was absolutely terrible idea, from beginning to end, online and offline (they even changed the bank name on the entrance of the agencies!). That said, I wouldn't say the online bit was such a big culprit for its failure...

</comment></comment><comment><USER4> Not quite as advanced, but they did this in the past and there was a huge backlash against it, so they stopped doing it: <http://www.wired.com/epicenter/2008/01/facebook-ads-ma/> .

From the wired article, it looks like there were a number of legal issues with reusing photos without permission [http://blogs.law.harvard.edu/infolaw/2007/11/08/facebook- soc...](http://blogs.law.harvard.edu/infolaw/2007/11/08/facebook-social-ads/) not to mention it's kind of creepy.

</comment><comment><USER5> Probably for the same reason athletic apparel companies sponsor athletes rather than average joes. I don't aspire to my own running talent; I aspire to Usain Bolt's talent. It makes the brand something to look up to.

</comment><comment><USER6> The best example of this I've seen is the Arcade Fire's "interactive film" called The Wilderness Downtown, which asks for your hometown address and… well, you'll see:

<http://www.thewildernessdowntown.com/>

</comment><comment><USER7> a.) The cost to get these adverts looking better than terrible flash games would be pretty huge, and very difficult to automate.

b.) The impact would be one of novelty, viewers wouldn't pay any attention to what the advert was trying to sell.

c.) Why bother, advertising is already able to sell products without placing the viewer inside the advertisements

<comment><USER8> Wow, you probably couldn't be more wrong.

a.) It's trivial to make it look good in fact it have been for quite some time.

b.) The impact is normally pretty big exactly because people pay attention once their friends or themselves are involved.

c.) Advertising as you seem to refer to it doesn't work nearly as well. Why do you think adwords work? They work because they communicate better by being closer to what you care about.

I shall see if I can make a list of examples but there is tons of great great examples looking stunning and having a huge impact.

</comment></comment><comment><USER9> I think you'd be fighting people's visual cortexes.

On the bad-end, you see a mostly-accurate copy/paste of your head in a random expression slapped onto a car. On the good-end, you start fighting with the uncanny valley[1], and people start rejecting it because there's just something _off_ about it. Then you've got people who won't know how you did it and will freak, as caioromao points out.

Will it happen _eventually_? I'd say absolutely, definitely. Minority Report shows a half-decent means of doing this, loathe as I am to cite sci-fi movies as future predictors. But doing it _correctly_ , at all times, will take _far_ more sophisticated visual analysis tools than we have right now.

[1]: <http://en.wikipedia.org/wiki/Uncanny_valley>

</comment><comment><USER10> The film "Daybreakers" did a promotion which used this technique in late 2009. The film's premise is that a vampire virus has swept through humanity, converting most people to vampires and leaving only a few surviving humans. These remaining humans are hunted and captured to provide blood for the rapidly starving vampires.

The ad was a site <http://www.capturehumans.com> , which pulled your Facebook picture and pictures of your friends. The site then showed a short video on the origins of the vampiric plague, with you as Patient Zero. Your friends showed up as other early vampires or as humans to capture.

The site isn't loading for me now, but this description gives this idea: [http://www.movieviral.com/2009/12/15/daybreakers-gets-a- vira...](http://www.movieviral.com/2009/12/15/daybreakers-gets-a-viral-site- that-claims-youre-patient-zero/)

</comment><comment><USER11> They do :-)

<http://www.renault.fr/espace-experience/index.jsp>

an awesome licensing deal we made with our panoramic photos.

-Jeffrey Martin, Founder www.360cities.net

</comment><comment><USER12> A Swedish advertising agency did just that. You could upload a photo of someone and this picture would appear all over the place in a video.

<http://www.draftfcb.se/info/thehero/en>

</comment><comment><USER13> Beacon, which was a much watered-down version of your idea, backfired. So I'm not sure if this would fly. Although it would be pretty whacky.

</comment><comment><USER14> Legally don't don't own the rights to your likeness. So they can't use your picture without your permission.

<comment><USER15> Not really a problem for facebook:

"For content that is covered by intellectual property rights, like photos and videos ("IP content"), you specifically give us the following permission, subject to your privacy and application settings: you grant us a non- exclusive, transferable, sub-licensable, royalty-free, worldwide license to use any IP content that you post on or in connection with Facebook ("IP License"). This IP License ends when you delete your IP content or your account unless your content has been shared with others, and they have not deleted it"

<comment><USER16> Out of interest, what would this mean if I don't have a Facebook account but my friend posts a picture of me?

</comment></comment></comment><comment><USER17> Will it not be violating my privacy by taking me as a object for advertising?

<comment><AUTHOR> The ad starring you would only be shown to you, although it may contain images of your friends.

<comment><USER18> So does it violate my friends' privacy?

<comment><AUTHOR> Perhaps it might be safer to not include your friends in the ad although the ad would be cooler if you do.

</comment></comment></comment></comment><comment><USER19> I made something similar to what you are describing. <http://www.socialadmanager.com> I kind of abandoned it though. What I found is that people hate ads no matter what and most people dont recognize people e.g.: Facebook friends in pictures.</comment>


How to Get a Job at Google - <AUTHOR> http://www.nytimes.com/2014/02/23/opinion/sunday/friedman-how-to-get-a-job-at-google.html <comment><USER3> Ok I'll bite.

I interviewed with Google about 2 years ago. First technical phone interview, the guy sounds really grumpy and irritated, and starts off with "I see that you have a Master's in CS. Are you an expert?". I replied something to the effect of "well I think it'd be pretentious for me to call myself an expert, I like to let my work speak for itself.". He responded "well we only hire experts at google".

The interview went downhill from there; several times he told me my solution to his question was incorrect, and I stepped through the code with him and showed him that he was mistaken and my solution was valid. He was just super grumpy the whole time.

Who knows, maybe my name reminded me of his ex wife or something. Regardless, it was a terrible experience.

I don't think that I sucked that hard because 2 weeks later I interviewed with apple, made it all the way to the onsite, and their questions felt harder. Their process was much more pleasant and efficient overall.

<comment><USER4> Yeah, same experience. I interviewed a long time ago (something like 5-6 years ago), and the interviewer sounded bored and grumpy the whole time. I told them before I interviewed that I didn't know of any of their core languages well enough to interview in them (Python being the only one I could possibly perform well enough with), but they said doing it in Ruby was fine since I was going for a testing position where they used Selenium and Ruby (no clue if that was true since the guy who interviewed me said it wasn't). The interviewer was just severely adversarial the whole time, constantly pointing out "you can't do that" syntactically or "that won't work" even though I'd step through and show them links or irb runs that proved otherwise.

And what's the deal with doing code interviews in Google Docs? I don't know if that's how it's done now, but it was extremely high-pressure. Given that he could see every keystroke and was pretty un-cooperative, it made for a super unpleasant interview with the constant feeling that I couldn't think or explore the problem given.

Needless to say, I didn't progress.

<comment><USER5> Yes, Google Docs was still the whiteboard for screening less than a year ago.

Thankfully, the person I dealt with was generally pleasant and made for one of the better interview experiences I've had.

<comment><USER6> Google docs here too the last time. Very pleasant, kind and helpful interviewer.

</comment></comment></comment></comment><comment><USER7> I somewhat enjoy interviewing, and I thought about writing up my narrative of the _best_ interview I ever had (it was very exceptional), and at the same time I thought about writing about the _strangest_ interview I ever had, a distinction that belongs to Google. I decided not to because I didn't want to be unfair to them based on my experience which may well be out of the ordinary (I could recount it solely for this crowd if someone is interested).

I didn't get far in that interview (I excused myself), but the entire process was _downright Kafkaesque_ and I get the same eerie feeling reading this article.

> Traditional leadership is, were you president of the chess club? Were you > vice president of sales? How quickly did you get there? We don't care. What > we care about is, when faced with a problem and you're a member of a team, > do you, at the appropriate time, step in and lead.

I'm sorry, _what?_ This definition doesn't actually describe anything. It's more useful as an example of _begging the question_ [1] than it is of describing what they mean by leadership. And how do they go about deciding if you step up to the plate at the right time in an interview, anyway?

It seems like their "leadership" criteria wouldn't actually select for people with good leadership skills, it would just select for people with good storytelling. Par for the course for interviews, but let's not kid ourselves.

[1] This phrase is commonly used to mean _raising the question_ but I mean it here in the traditional sense, like the comical observation: "If such actions were not illegal, then they would not be prohibited by the law."

<comment><USER8> I would love to hear the story.

<comment><USER7> Sorry! I dozed off.

PLEASE remember that my anecdote may not be indicative of Google's typical recruiting and I am especially not passing judgement about anything they do (except maybe recruit). I like Google and it's probably a great place to work.

Here's what happened:

Back in 2011 a Google recruiter emailed me saying I might be a good fit for a position called:

> 'Performance Engineer' for which we're looking for a candidate with a > combination of compiler, high performance software design and computer > architecture experience.

Now I'm not certain how he came to the conclusion I might be a good fit, because I'm a simple JavaScript farmer from a backwards asteroid and the only thing remotely close to that job description in my short life were a few articles about JavaScript Canvas performance that I've written. I relayed that there must be some sort of misunderstanding about my skill-set.

He says ah thanks, sorry for the confusion, have you got any friends? Standard recruiter stuff. We part ways.

Fast forward to 2013 and I've just spent a year of my life slogging through writing a book on HTML5. It got published in July and I was busy resting on newly-built laurels when another Google recruiter contacts me, and I entertain.

Now the timing is not particularly weird but this recruiter contacts me a week before a conference I was invited to. You see, one month before the recruiter email I got an invitation to a conference (EdgeConf), which happened to be invite only, even for the audience members, and which was to take place at Google HQ. This is puzzling if only because I'm still a relative nobody, save for perhaps the just-published book, and I couldn't fathom who might _invite me_ to such a thing, but I figured it was someone in Google.

The recruiter email a week before the event made this theory much more plausible, so I figured there may be some hint of who invited me awaiting at the conference.

Alas, during the talks and the after-party nobody approached me or mentioned who invited me. I did meet the ever-recognizable Paul Irish for a second - <USER2> exchanged a handshake and I swooned (who wouldn't?), but no more. The puzzle remained.

But how could it be an invite from anyone other than someone in Google? I don't know anyone. I'm from New Hampshire. I have very little in the way of programmer friends. That's a big part of why I love HN so much, it makes me feel like I'm part of a programmer community that I don't get to have offline.

After the conference I talked to the recruiter on the phone, twice, where I relayed that no I was not interested in leaving New Hampshire, I like my slow life among the moose and other friends I have.

Well that leaves Cambridge, he says, I'm not sure what we do there, he says, I'll get back to you. Cue dial tone. I guess that meant remote is out of the question, but that's not hard to imagine.

On the second call, he talks to me about Cambridge and what they do there, but he was oddly uneasy talking about it, like he knew something I didn't (he certainly must). He made it sound like there was little (web?) developer presence there, and he wasn't totally certain what the projects were, but, and this is the part that really blew my mind, _it shouldn 't matter._

He didn't literally say those words, but he heavily implied them by telling me that _I would not know what kind of a spot I would be interviewing for._ Not until after I'd been accepted, at any rate. There was no "You'll work on fixing Google Finance, which can't be hard since we haven't changed it in 6 years and it still uses Flash."

There was almost no discussion of what products I might work on, and it was pointless to have, because I'd be interviewing to be a _developer_ , and I'd be placed afterwards.

So I could interview, but what I'd be doing was really up for grabs, and I wouldn't be doing the grabbing, and my options-that-were-apparently-not-mine- to-make were pigeonholed because of location. It sounded like teams only have people on-site, at least It was implied I couldn't work for a team based in Mountain View if my butt sat in Cambridge.

So I asked if I could be sure of the position and team that I'd be working for before I interviewed properly, and he said no. At this point I had to cut things short.

I don't understand the concept of interviewing not for a job but for a slot in a pool of talent to be assigned only after I am accepted. That sounds like an insurmountable amount of work to find out if a job is something I'd want to do or not. What if I don't like the app? Or the team? It seems like a very stressful way to begin at a company.

So there you have it. That's everything I remember at the moment.

To this day I never found out who invited me to that conference.

I'm sure I'm remembering some of the details wrong. For instance, cell phones do not have dial tones to cue. But you get the idea.

<comment><USER9> > To this day I never found out who invited me to that conference.

Pretty sure I did, dude. :) Also, while I sometimes offer names of friends recruiters should talk to, I didnt' around this time. So that part's still a mystery. Additionally, it was a pleasure to meet you.

<comment><USER10> This might just be one of the greatest replies I've ever read.

Also, I aspire to attain your stature. I'm a fan.

</comment></comment></comment></comment></comment><comment><USER11> There's a big (and common) error in statistical reasoning Google is making with the decision to down-weight GPA based on their data: That GPAs do not predict performance among those it hired does _not_ imply that Google should not use GPA when hiring any more. Rather, it means that Google used GPA to exactly the right extent among those it hired under its old policy - there was no information left in GPA they didn't use, and therefore they should leave whatever policy they have in place as is.

Explanation: suppose there are only two things Google observes, GPA and coding ability, and that Google uses some correct decision rule to only hire those people where the sum GPA + coding ability > some threshold. Those who have lower GPAs will thus tend to have higher coding ability, otherwise they wouldn't have met the threshold to make it into the pool of hires they're analyzing - and, therefore, comparing "those with low GPAs that Google hired" and "those with high GPAs that Google hired" is not an apples-to-apples comparison.

In order to assess whether GPA should be used at all, they would need to look at how the people they didn't hire because of their existing policy would have performed.

More reading: [http://beerbrarian.blogspot.com/2013/11/the-subtle-joys- of-s...](http://beerbrarian.blogspot.com/2013/11/the-subtle-joys-of-selecting- on.html) and [http://www.jamesmahoney.org/articles/Insights%20and%20Pitfal...](http://www.jamesmahoney.org/articles/Insights%20and%20Pitfalls.pdf)

</comment><comment><USER12> These are great ideals. But there are a lot of problems with how Google does an interview process.

The most known problem is recruiter quality. Google's recruiters are notoriously non-diligent and screw up in big ways. It's not unheard of for eng candidates to be booked with a non-eng interviewer, and vice versa.

In any case, here is how it works in practicality:

1\. You apply for job X and do a phone screen (or two).

2\. Random people throughout the company who a) have had interview training and b) are (supposedly) related to X will be tapped to interview you, but nobody from the actual team you are applying to is selected to minimize biases.

3\. Each person interviews you and fills out a form giving feedback. Feedback is qualitative, as well as quant (scale ratings with previous entries exposed to make biases clear).

4\. Feedback goes to an unbiased committee (again, not related to the hiring team) for a decision. Feedback is permanently saved in case the person applies again.

The problem with (2) and (3) is that it sometimes results in an unrealistic process - depending on the prowess of the recruiting coordinator in setting up interviewers. A front-end dev candidate for Shopping might get interviewed by an deep algos person from Search, and tank the interview because the interviewer had higher expectations for the candidate's theoretical knowledge.

Good interview training alleviates these problems somewhat, but there are still huge gaps.

<comment><USER13> I have interviewed with Google before and I'm currently going through the process of getting an internship with them. Let me tell you I've had very disparate and different experiences in these months.

I was contacted exactly one year ago for a fulltime position and my contact/recruiter was really excellent, very friendly, helpful and informal, we also talked a lot about non-Google related stuff, a definite great guy. Then I was moved to onsite interviews and I failed.

Fast forward to last October, I was contacted for an internship, I said I was interested and the recruiter said my application was forwarded. I never heard from them again (even after pinging the guy again). Then I talked to an engineer at Google and I was told that since I had already done the required interviews with them a year before (I failed for fulltime but they apparently qualified me for internship), I didn't have to do them again so the process should have been faster.

With that in mind I managed to get in touch with another recruiter and explained my situation, she decided to give me a chance and we had a phone screening (although it shouldn't have been necessary, but no big deal). After that she told me I'd have to go through all the interviews again although that's not what I was told. She passed on my application and nobody contacted me again.

That was beginning of January. I pinged her a few weeks ago and luckily I was told that my application had been magically forwarded to another recruiter again (aka she forgot, probably). This time I was told that I qualified to skip the interviews so now I'm at a later stage (project matching for internship) so I can't complain, although I was told there's not much choice because I am "applying late" even though it's been going since October.

Tl;dr - it really depends on the recruiter, it's a shame because that can really reflect negatively on the whole company.

Ps: sorry for the lengthy post, I needed to vent

</comment></comment><comment><USER14> Here is a much better and quite popular (in engineering circles) post by Steve Yegge:

[http://steve-yegge.blogspot.com/2008/03/get-that-job-at- goog...](http://steve-yegge.blogspot.com/2008/03/get-that-job-at-google.html)

This of course applies only to engineers.

<comment><USER15> You mean developers. Programmers are not engineers.

<comment><USER14> Of course I meant software engineers (SWE). There is no job title 'Programmer' at Google.

</comment></comment></comment><comment><USER16> Oh great, Thomas Friedman.

[http://www.salon.com/2012/07/25/the_value_of_tom_friedman/](http://www.salon.com/2012/07/25/the_value_of_tom_friedman/)

[http://nypress.com/flathead/](http://nypress.com/flathead/)

[http://www.rollingstone.com/politics/blogs/taibblog/no- kiddi...](http://www.rollingstone.com/politics/blogs/taibblog/no-kidding-the- most-incoherent-tom-friedman-column-ever-20120725)

[http://jilliancyork.com/2011/12/14/the-definitive- collection...](http://jilliancyork.com/2011/12/14/the-definitive-collection- of-thomas-friedman-takedowns/)

<comment><USER17> Obviously an expert on meritocracy.

<comment><USER16> Thomas Friedman is a billionaire by marriage. Or were you being sarcastic?

<comment><USER17> I take my sarcasm bone-dry.

</comment></comment></comment></comment><comment><USER18> It's unfortunate that the skills they claim to value are ones that can't be measured, especially in the limited time of an interview: "leadership, humility, collaboration, adaptability and loving to learn and re-learn." While these are great qualities, I wonder what they actually base their decisions on in lieu of the above information.

<comment><USER19> They measure that over time. Coding skills they can measure quite well.

<comment><USER20> >If it's a technical role, we assess your coding ability, and half the roles in the company are technical roles.

Which means that the other half are not technical roles.

<comment><USER21> This has always baffled me. Is coding the only technical skill?

It's especially bewildering when a company seeks a software engineer with expertise in a highly technical domain and populates the interview primarily with programming questions. If they can find the subject matter expert who also happens to be a master software craftsman, congrats to them, but I'm guessing that person doesn't come along very often.

(Maybe I'm bitter, but with the level of skill in computer vision I can demonstrate, don't you think you can teach me your programming nits in short order?)

<comment><USER17> "This has always baffled me. Is coding the only technical skill?"

What other non-soft skills are you discussing?

</comment></comment></comment></comment></comment><comment><USER22> Despite the fact that Laszlo is the head of HR, engineering is in charge of its own hiring procedures. I would take all of this with a giant grain of salt.

<comment><USER23> +1 I don't think I have ever heard of anything about leadership being asked of IC engineering candidates, and this is the first I've heard that intellectual humility is important either. I mean, it's a good thing, but nobody ever directed me to look for it.

</comment></comment><comment><USER24> WARNING: TOM FRIEDMAN.

Why is that not on the link?

<comment><USER25> Why is that important?

<comment><USER26> Because Tom Friedman is a cluster of bad ideas masquerading as a newspaper columnist.

</comment></comment></comment><comment><USER27> Candidates always optimize for what is being interviewed, never ever forget that. The moment you put down a interview 'process' its all over. Doesn't matter what that process is, doesn't matter how difficult that is. Once you do it, its only a matter of recognizing patterns. Then all you have to do is develop a elaborate process to game it. A couple of months of practice is all it takes to do that.

The Steve Yegge, 'Get that job at Google' article is nothing but a 'Game this interview' article. Read this book, practice this X times and you shall get through. For heaven's sake, Do people hire for getting job done everyday or playing some puzzles from some book not even remotely relevant to any thing you will ever do on your job.

Companies like these want the best of the best people. Yet their methods completely revolve around knowledge and fact based questions. And sorry I don't believe asking difficult questions make it any better. There are forums on the net dedicated to train you crack such questions.

How many interviews check if candidates can last tough on the job situations? How many check if the candidate is hard working, How many check the candidates appetite to work on tough challenging time pressing projects? How many check if the candidate is innovative? Or checks a candidates general abilities like gumption, persistence or general appetite for work and delivering.

These companies often speak on how difficult it is to hire good people and then purposefully invent processes to avoid hiring such people.

<comment><USER28> Much of the technical advice in Steve Yegge's article boils down to this: "Study a data-structures and algorithms book. Focus on complexity, graphs, discrete math, operating systems, and algorithms."

While you could consider this advice on how to game an interview, I would argue it's also excellent advice on how to become a better software engineer.

<comment><USER29> I think may be you missed the important point in the article. Everybody these days knows algorithms and data structures. It's much more about being fast at arbitrary problem solving on the spot and attitude.

<comment><USER30> +1000 to this. I relate you comment to Neil deGrasse Tyson's quote: "There are people who say "I'll never need this math -- these trig identities from 10th grade or 11th grade." Or maybe you never learned them. Here's the catch: whether or not you ever use the math that you learned in school, the act of having learned the math established a wiring in your brain that didn't exist before, and it's the wiring in your brain that makes you the problem solver."

<comment><USER27> I have an even better of 'wiring' your brain. Work on projects, that forces your brain to wire and you also get work done along the way.

</comment></comment></comment></comment></comment><comment><USER31> "The No. 1 thing we look for is general cognitive ability, and it's not I.Q. It's learning ability. It's the ability to process on the fly."

I do not understand.

<comment><USER32> 'Restriction of range'. When you look at the general population, from janitor to tenured professor, one of the single best predictors of job performance is IQ; but when you look at, say, 'MIT physicist grad students', well, they're _all_ very smart, so when you look at the minor differences in intelligence between them, the scores barely correlate with their future success _even though_ no one becomes a MIT grad student in physics without being terrifyingly smart. Because they have been selected on that trait already. Hence the apparent paradox.

(No surprise Friedman doesn't understand this; the NYT article on the Google findings did not explain this.)

<comment><USER33> The limits of Friedman's understanding should come as a surprise to no man.

<comment><USER32> I dislike Friedman a fair bit, but I can't hold this one against him: restriction of range is a subtlety that pretty much no one understands in interpreting correlations, and I only know about it myself because I happened to read a paper torpedoing a previous famous paper as having fallen prey to restriction of range (and then did some more reading on range restriction).

</comment></comment></comment></comment><comment><USER34> 100% BS.

The best way to get job at GOOG is to get job at MSFT, AMZN or Apple, pimp your Linkedin and Dice profile, get contacted by GOOG recruiter and then get auto-hired.

Although, as Ballmer is gone - MSFT becomes sexier and smarter bet for new hires than overbloated GOOG. AMZN seems to get obsessed with it's Silk browser project and is desperate to hire anyone who is desperate to get hired.

PS: Here's my recent reply to AMZN recruiter pimping Silk browser project:

</comment><comment></comment><comment></comment><comment>===== Hi ____,

Thank you for your email. I think I am a perfect fit for this position:

>>You should have an intimate understanding of how the web works from the underlying infrastructure of the Internet, to web servers, to browsers. [http://i.imgur.com/C6nE2.jpg](http://i.imgur.com/C6nE2.jpg)

>>You like thinking "outside the box", are not afraid of ambiguity, [http://i.imgur.com/1dQII.jpg](http://i.imgur.com/1dQII.jpg)

>>get excited about difficult distributed systems challenges, [http://i.imgur.com/Kh7bX.jpg](http://i.imgur.com/Kh7bX.jpg)

>>and are a motivated self-starter. [http://i.imgur.com/B6vRRMs.jpg](http://i.imgur.com/B6vRRMs.jpg)

>>You are a strong team player [http://i.imgur.com/k9NTgH0.jpg](http://i.imgur.com/k9NTgH0.jpg)

>> and thrive in a startup-like environment where flexibility is essential [http://i.imgur.com/3p7rF.jpg](http://i.imgur.com/3p7rF.jpg)

>> and delivering rock solid, customer focused solutions is paramount. [http://i.imgur.com/kuDJTmx.gif](http://i.imgur.com/kuDJTmx.gif)

>> "The trouble with quotes on the internet is that it's difficult to determine whether or not they are genuine" [http://i.imgur.com/xxotrDe.jpg](http://i.imgur.com/xxotrDe.jpg)

<comment><USER35> > get auto-hired

There's no such thing. Before you are offered a job at Google you have to go through the interview process and the interviewer's feedback is assessed by a hiring committee.

</comment></comment><comment><USER36> "For every job, though, the No. 1 thing we look for is general cognitive ability, and it's not I.Q. It's learning ability. It's the ability to process on the fly. It's the ability to pull together disparate bits of information."

The cognitive dissonance required to say "general cognitive ability... is not IQ" is astounding.

IQ testing bad! Hiring smart people good! Does not compute!

<comment><USER37> He specifically said that he defined general cognitive ability as the ability to learn, process on the fly, and pull together disparate information. Which is indeed distinct from IQ.

Intelligence varies, and people differ in which tasks they are good at. They very well might not think that IQ is a good predictor that they are looking for.

<comment><USER38> > ... the ability to learn, process on the fly, and pull together disparate > information. Which is indeed distinct from IQ.

Do you have cites for that?

<comment><USER39> To be fair, "ability to learn, process on the fly, and pull together disparate information" is indeed "indeed distinct from IQ", but only in the pedantic sense that it'd be more accurate to say such capacities refer to g.

But then...IQ is indeed a reliable, verifiable, and efficient indicator of g, so...indeed.

</comment></comment></comment></comment><comment><USER40> This is not ad hominem but I should note that anyone reading this article should take care more that it is written by Thomas Friedman and maybe one should take care before wasting too much time reading it.

</comment><comment><USER41> Generally, a technical interview is meant for you to beat their best technical person in the team, either for the knowledge of cleverness, so everyone adore you in order to pass the interview.

Yet it is well known that most technically talented people have a common OCD, especially of those with strong ego, is to compete to be who the smartest is, which is not a bad thing, but some went extreme and they do it at all cost. Evidently as often observed, in office the loudest person is perceived the smartest be the mass. Can google avoid that?

In a few hours interview, psychological problems like these are difficult to detect and rule out. If google is serious about their claim that they don't care about expertise but .... They should send candidates to psychiatrist and filter first.

Personally I think this NY times google HR is BS marketing. There is better way to recruit top talent, and this is not it.

</comment><comment><USER42> One potential issue is that it appears the coding is the only criterion listed that can be fairly assessed before meeting someone and having a dialogue or giving some test. If they truly don't care about your past traditional leadership, grades, and are most concerned with being good on the fly, what criteria are used to make a decision on bringing someone in for an interview? One's list of accomplishments, transcripts, and past job titles won't typically provide the answers. If these are the hiring criteria that is one thing, but it's a bit of a reach to say that those are the interview selection criteria. I feel that is a key factor here (perspective - I'm a recruiter of developers).

<comment><USER43> I suspect this article is what they say they do as opposed to what they actually do.

They may even say these things internally as part of the "official" plan, but as you point out, there's reason to be skeptical it's how they actually operate in practice.

<comment><USER42> You can hire on these things once in the interview process, and I think most companies probably like to think they do the same thing. But they aren't interviewing everyone we can assume. So how do you pick people for interviews? This is where all the GPA and good schools stuff comes in usually, or something unique about the candidate. Perhaps certain experience that serves as an indicator that the person will have the required attributes.

I'd have been more interested in hearing how they select for interviews and not how they hire. If someone from an average school with a below average GPA, he/she could impress and get hired on these attributes (which the candidate certainly can possess), but the real trick for these candidate types is getting in the door.

</comment></comment></comment><comment><USER44> My experience with interviewing is that what I think of as level two or three problems are presented, where you have to solve some recursive problem and give the O() notation for the implementation. This happens in each of the multiple interviews over the three to six hour interview session. If you're not extremely good at this you are going to be false negatived tf out of there right quick.

Top level problems are where the solution involves days, weeks or months of study and contemplation where the solution possibly involves a large amount of refactoring of existing code. Architectural level stuff. There is no way to test for this in an hour long interview.

When I mention refactoring, I often get a blank stare from the interviewer. I had a coworker recently who insisted on calling it "refuctoring", having no concept of it or any desire to investigate.

</comment><comment><USER45> Shameless plug warning. I'm preparing for my Google interview and I've collected some resources for a fairly quick but thorough preparation for the interview (well, except for CLRS, I just like it).

[https://github.com/andreis/interview](https://github.com/andreis/interview)

</comment><comment><USER46> As a current student, I don't buy what they're saying one bit. If this is the case, why do they ask for your GPA when you apply online for a programming role? More anecdotally, everyone I know who made it to the interview process had a stellar GPA. Most were members of the CS honor society on campus.

I don't have any hard proof, but it's clear to me that at least for new hires, GPA is a big requirement for Google.

<comment><USER47> Well, GPA shouldn't be tricky at all if you have what it takes to work at Google.

This means, unless you were fed up with the educational system like I was and you purposely did a mediocre performance, you probably aced them.

</comment></comment><comment><USER48> too bad their hiring practices are flawed

</comment><comment><USER49> I like Friedman's summary. If it is accurate, Google has a pretty forward- looking hiring philosophy; definitely better than the view espoused by Marissa Mayer sometime back about only hiring good students (or something along these lines). Working there sounds like it could be quite nice. It's too bad about the stacked ranking, which is a huge liability for a company in my view.

<comment><USER50> Unlike some stack ranking horror stories I've heard, at Google the engineers are stack-ranked against everyone else at the same tech level. That's thousands of people. If you're in the bottom 10% of thousands of people, then "needs improvement" is probably a pretty accurate assessment.

To be let go you have to do quite poorly, by the way. Once you get in the door it would be possible to coast, assuming you did have some productivity. You just won't get the fun stuff.

</comment></comment><comment><USER51> I had just submitted the OLD link summarizing last summer's interview to HN today (to answer a question that came up overnight about Google's hiring practices), and I see that the author here was just as struck by the statement from last summer, "G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless. ... We found that they don't predict anything." In more complete context, from last summer's interview by another New York Times writer,[1] that statement was "One of the things we've seen from all our data crunching is that G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there's a slight correlation. Google famously used to ask everyone for a transcript and G.P.A.'s and test scores, but we don't anymore, unless you're just a few years out of school. We found that they don't predict anything." So, yes, if you have little work experience and were in full-time school recently, Google will ask you about your G.P.A. But if you have substantial industry experience, Google will ask you about that, or at least such is the claim here about _current_ Google practice.

Some comments posted before this comment mentioned the confusing way that "general cognitive ability" is said not to be the same thing as IQ. No psychologist, and especially no psychologist specializing in industrial and organizational psychology, would speak that way. The universal statement in that branch of psychology is that IQ scores are a very good indicator of "general cognitive ability" indeed, the least expensive and most efficient indicator possible for most hiring situations. I think the Google executive interviewed in today's article and in last summer's article is just trying to be legally correct while eating his cake and having it. In the United States, there are tricky legal issues surrounding using IQ tests as a hiring criterion that grow out of the Griggs v. Duke Power case decision by the United States Supreme Court.[2] IQ tests are disfavored as a hiring criterion by that decisions (but the little known fact is that so are educational credentials disfavored as a hiring criterion[3] by the same case), so companies tend to try to use more expensive, less reliable proxies of IQ tests to get the results that IQ tests would give them more reliably and less expensively.

[1] "In Head-Hunting, Big Data May Not Be Such a Big Deal" 19 June 2013 by Adam Bryant

[http://www.nytimes.com/2013/06/20/business/in-head- hunting-b...](http://www.nytimes.com/2013/06/20/business/in-head-hunting-big- data-may-not-be-such-a-big-deal.html)

[2] [http://www.naacpldf.org/case/griggs-v-duke-power- co](http://www.naacpldf.org/case/griggs-v-duke-power-co)

[3] [http://econlog.econlib.org/archives/2013/04/iq_and_hiring_d....](http://econlog.econlib.org/archives/2013/04/iq_and_hiring_d.html)

<comment><USER52> Since companies increasingly look for it , is there any way to improve "general cognitive ability" ?

<comment><USER53> Hard to say, obviously. Not unlikely though the answer is no. See for education for example.

 _Recent reports suggest a causal relationship between education and IQ, which has implications for cognitive development and aging—education may improve cognitive reserve. In two longitudinal cohorts, we tested the association between education and lifetime cognitive change. We then tested whether education is linked to improved scores on processing-speed variables such as reaction time, which are associated with both IQ and longevity. Controlling for childhood IQ score, we found that education was positively associated with IQ at ages 79 (Sample 1) and 70 (Sample 2), and more strongly for participants with lower initial IQ scores. Education, however, showed no significant association with processing speed, measured at ages 83 and 70. Increased education may enhance important later life cognitive capacities, but does not appear to improve more fundamental aspects of cognitive processing._ [1]

What you can improve is your test scores. Which is just as good for getting hired. They won't know the difference.

[1] [http://psycnet.apa.org/psycinfo/2012-34906-001/](http://psycnet.apa.org/psycinfo/2012-34906-001/)

<comment><USER54> How about: Education makes you better at intelligence tests. I took the GRE (Graduate Record Exam) before going to Stanford. I got 99%, even though the test covered Engineering areas I had never studied. I did that by noticing, in the areas I did know, that the questions were all about fundamental principles. I could guess at the answers then for fluid mechanics, deformable bodies etc, and apparently got them all right. Am I intelligent? Sure, but not THAT intelligent.

<comment><USER53> _Education makes you better at intelligence tests._

Or even taking tests in general. Yes, that's what I meant.

</comment></comment></comment></comment></comment><comment><USER55> One thing Google will never do is explain how they measure employee productivity. We hear lots about their discoveries about predictors, but for all we know this is coming from a google VP with no tech background that counts lines of code as a measure of productivity.

</comment><comment><USER56> Here's the thing, I believe I have all those qualities. What I lack is the courage to apply. I am a Google fanboy, to be honest, and working there is a dream for me. I also know that i am not the only one.

<comment><USER57> Many people panic on their first interview and fail, and if you show any potential they'll let you interview again. I bombed my first one and they told me that I'd be eligible to interview again in 18 months and I should call them then, and they actually called me in 12 months and set up another interview.

Don't worry about the courage, just apply. If you fail... well, many many people (and many many Googlers) also failed their first Google interview, it's not much of a reflection on you.

</comment></comment><comment><USER58> Does google pay significantly more than, say, any old rails job? Perks don't really impress me. 20% time sounds interesting, but I hear it's gone. Why would I want to deal with the interview process?

<comment><USER14> [http://www.glassdoor.com/Salaries/google-salary- SRCH_KE0,6.h...](http://www.glassdoor.com/Salaries/google-salary- SRCH_KE0,6.htm)

<comment><USER58> Thanks. I think that speaks for itself.

</comment></comment></comment><comment><USER59> Despite all the bashing of Google's hiring process, I think Google's hiring process is very good. My experience was completely positive.

 _Google had determined that "G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless. ... We found that they don't predict anything."_

No, that's not what Google is able to determine. The blanket claim is too broad. Predict _anything_? I don't think GPAs are worthless, but that's another discussion for another time. Regardless of whether they're useful, the only thing Google has is its own performance ("perf") data which represents political factors more than anything else.

If you find a zero correlation in junk data, you know... absolutely nothing more than you did beforehand. That's the thing about junk data. You can't trust it, so you can't use it to infer anything.

</comment><comment><USER60> why not cut out the middle man and just apply straight to the NSA?

<comment><USER61> Google by itself is much worse than NSA. At least, NSA does not screw up the industry and does not try to destroy local economies. I'm surprised there's not a lot of public backslash against Google and their employees.</comment></comment>


Why PostgreSQL doesn't have query hints - <AUTHOR> http://it.toolbox.com/blogs/database-soup/why-postgresql-doesnt-have-query-hints-44121?rss=1 <comment><USER2> This is how you make your users upset.

Speaking personally I have personally encountered times when PostgreSQL (this was in the 8.2 series) simply Could Not Find The Right Plan. I looked at the tables and indexes. I saw the right query plan. It did not. I did not have DBA access so I had no ability to try to figure out why. However I _did_ have the ability to rewrite my query into two, with the first going into a temp table. This effectively forced PostgreSQL to use the query plan I knew would work, which caused my query to run in under a second as opposed to taking multiple minutes.

I had similar experiences with Oracle, and used query hints very successfully.

I don't dispute the claim that this likely happens only on 0.1% of queries. Based on what I've seen in Oracle, I also suspect that most people who use hints use them as voodoo, and most of the time use them incorrectly.

However you're much more likely to hit that 0.1% if you're writing very complex queries (which happened to me when my role was devoted to reporting). People can learn how databases are supposed to work (I certainly did). Even then I acknowledge that 95-99% of the time it does better than I could. But it is still really, really helpful to, in the remaining 1-5% of the time when the database goes wrong, be able to tell it the right way to do its job. And in my personal experiences, the cases where the database gets it wrong, it wasn't a temporary problem - it stayed wrong.

But they don't acknowledge the existence of people like me. They assume that I must be lazy, ignorant, or have had my bad experience decades ago. Because _their_ wonderful software certainly couldn't have given me bad experiences _in the last 3 years_. (I'm no longer in that reporting role, so I don't hit that case any more.

<comment><USER3> Here's the real reason: this dev is an jerk who thinks postgres users get too much sleep.

My experience on 8.4, within the last 3 years: a query that is run hundreds of times per second with an average run time in milliseconds and a max query time of .3 seconds suddenly starts taking 100 to 1000 times as long to run. Six hours of debugging later starting at 3 in the morning when systems started failing, we figured out that some magic in the query planner had tripped over and changed the query plan to something that is at least two orders of magnitude worse. No indices have changed. No schemas have changed. Data grows by maybe 30k rows per day which is very reasonable given table sizes and the 128GB of ram dedicated to pg.

Of course, there's no way to specify the query plan. Instead, we ended up fucking with configs until the query plan swapped back.

 _That's_ why people like locking query plans. Not necessarily to control the best case, but to control the average / worst case.

<comment><USER4> > That's why people like locking query plans. Not necessarily to control the > best case, but to control the average / worst case.

This. I've had exactly the same experience as you.

Nobody wants to put hints in the query plan. But when your web site is down because one day the freakin' query planner decided all by itself that it was time for a change and some of your worst case queries are taking 2 minutes to return I don't want to be fudging about with statistics trying to understand the internal mind of the planner to convince it to return to sanity. I just want to make it do a plan that I know won't lose me my job.

The kind of attitude in this article reminds me - sadly - of the BS that used to come from the MySQL devs. "No you don't need transactions! Your system is broken if it uses transactions!". Of course it was BS and the minute they supported transactions they were all over how good it was.

<comment><USER5> For the record, that form of statement ("no, you don't need X") cuts both ways, and therefore whether or not I agree that PostgreSQL needs query hints, this argument of comparison bothers me: you could use the same argument for "No, you don't need the ability to violate referential integrity! Your system is broken if it violates declared foreign key constraints!", which is to me a very correct (and "very PostgreSQL") statement to make about how horrible it is that MySQL /added a feature/ that turns off foreign key constraint checks in a way that leaves your data in an incosistent state due to the same kind of pressure from DBAs who weren't really thinking about the long term consequences. Put differently: you stripped away all of the reasoning and are now comparing the lexical syntax of a conclusion and trying to claim that this means two situations are alike, which is actively misleading to your readers and yourself.

</comment></comment></comment></comment><comment><USER6> The bottom line here is if your site suddenly goes down because PostgreSQL changed its query plan in the middle of the night, the answer PostgreSQL has for you is "sorry, you're stupid and now you are fucked." Until you can decipher what exactly is happening and make an appropriate fix, which can take days or weeks, your site is effectively dead.

Nobody disagrees with the noble goal of building the best query planner, or thinks that index hints are anything less than a hack for dire siutations. Where we disagree is if it's OK for the sites we're responsible for to go down at the whim of the query planner. PostgreSQL's decision to not provide hints tells me they care more about the purity of their query execution engine than the applications that rely upon it.

<comment><USER7> A couple people have mentioned the query plan changing in the middle of the night. Is the scenario a single query getting executed two different ways by the same running system? Or is it from upgrading to a new version of PostgreSQL and hitting different behavior? If it's the first scenario, would using prepared statements help somewhat with avoiding unexpected changes?

// Edit: Sorry for the naive question. I see that other people are saying that is the first one.

</comment></comment><comment><USER8> tl;dr They trust the cost-based query optimizer more than the DBA; for the rare cases where the optimizer gets it wrong, they're more likely to add cost adjustments than the explicit "do it like this" directives that, say, Oracle DBAs are used to. There's already one cost adjustment in Postgresql 9.0.

(They also claim that query optimizers all over are good enough that they don't need the hints anymore, and the only reason that anyone asks for them is because they've gotten so used to messing with bad query planners that they don't know how to work with a good one. Which, regrettably, has not been my company's experience with Oracle 11g --- we haven't resorted to explicit hints, but we've restructured queries in other ways for order-of-magnitude improvements in performance.)

<comment><USER9> >Which, regrettably, has not been my company's experience with Oracle 11g --- we haven't resorted to explicit hints, but we've restructured queries in other ways for order-of-magnitude improvements in performance.

He's not claiming that query planners are so good that you'll never have to restructure your queries. I've written many queries that seem perfectly reasonable, but once I start optimizing I can see I'm doing things completely backwards. It's just like almost everything else programmers do. If I dash off quick program, I don't expect the VM figure out how to make it perform perfectly. Why would I expect database software to be different?

<comment><USER10> > It's just like almost everything else programmers do. If I dash off quick > program, I don't expect the VM figure out how to make it perform perfectly. > Why would I expect database software to be different?

Except that SQL wasn't designed as a programming language. It was designed to be a declarative language that normals could use.

In a normal language, you have the freedom to express an algorithm in a billion different ways. You're _expected_ to find the right way to write it. That SQL falls down here is a fundamental flaw in what was supposed to be it's greatest strength.

<comment><USER11> In what was supposed to be its greatest strength _a long, long time ago_. I don't know that anyone considers SQL to be anything other than a technical detail any more; certainly not to the extent of expecting normals to be able to get what they need out of a database by writing raw queries.

<comment><USER10> Yes, but that feature still hinders SQL today. If "usability by normals" was dropped as a design requirement, SQL would look a lot more like a normal programming language, and suck a lot less.

</comment></comment></comment></comment></comment><comment><USER12> I think you could probably get hints into Postgres if you did them right (and volunteered to do the work!). Where "right" means trying to make the hints at least somewhat declarative: not "use index X for query Y", but "the values of these two attributes are not independent" or "this predicate is likely to be highly selective." Basically, look at the most frequent planner failures, figure out why the planner makes a mistake, and let the user specify the information the planner needs to choose the right plan. Postgres has added features in this direction: for example, you can now specify the cost associated with evaluating a function as part of CREATE FUNCTION.

The work required to implement this would be substantial, and most of the people with the necessary skills would rather improve the planner itself (e.g., collect statistics on cross-column correlations to avoid making the attribute independence assumption in the first place). So it isn't too surprising this hasn't got done.

</comment><comment><USER13> > Many older DBAs, and their proteges distrust query planners because of bad > experiences back in the 80's and early 90's. They are sure they can do a > better job, even though they can't.

I remember having cases in the past year (on Oracle 10g) where adding hints helped substantially. Since it's impossible for me to ever do better than the optimizer, I guess this must mean I've gone nuts?

> Other DBAs are lazy or labor under unrealistic deadlines. Applying a query > hint is often faster and easier than diagnosing the real reason the query > has a bad plan.

Hints are evil and you should never use them, even though they make your job easier.

> Ignorance and weak software play a role too: good diagnostic tools and > techniques for troubleshooting bad queries did not become available until > relatively recently, and most DBAs still don't know how to use them.

Instead of just telling the optimizer what to do, you should have an extensive discussion with it and attempt to persuade it with concessions so that you don't hurt its feelings.

> The developers who work on the PostgreSQL not-for-profit database project, > though, have the privilege of not implementing a bad idea just because a lot > of people seem to want it.

Hints offend us, and we don't care if people find them useful.

> All that aside, there are those 0.1% of pathological cases where the query > planner does The Wrong Thing even when it's patently obvious what the right > thing is. In our community, that usually leads to patches and improvements > in the query planner and the statistics system

Not supporting hints helps us extract useful code from our users. (hey, this is an _aweseome_ idea, I need to think how to apply it to my projects...)

> Well, one thought is a system which would allow DBAs to selectively adjust > the cost calculations for queries. For example, one which allowed you to put > "SELECTIVITY 0.1" after a WHERE clause to tell the planner that despite what > it thinks from its statistics, that set of criteria will give you 10% of the > table. Even better would be a system of fudging the statistics on database > objects, to allow DBAs to indicate (for example) that using a particular > index is more costly than it appears because of the poor clustering of the > data or because of the complex calculated expression it uses.

Hints are normally done the wrong way, and when we say we won't implement hints we really mean we won't do them that wrong way. We think we know the _right_ way to do hints, and we're working on implementing it.

<comment><USER14> What arrogance. Perfect optimizers are an illusion. They are guaranteed to fail and to fail at the most inconvenient times. It is not only possible to do better than the optimizer on complex queries with hints its a given. Now that I know hints are not possible in PostgreSQL I will strictly curtail the kind of projects I will apply this RDBMS to. If you haven't found the need to use hints you have not explored the full range of what SQL is capable of (or worse yet use an ORM).

</comment></comment><comment><AUTHOR> Personally I have witnessed a pretty drastic level of hoop jumping to get pgsql to force a particular index, and forgo destroying the site for another few days. And this hasn't been by schmuck DBA's either. Rather, by actual postgres committers that we have employed as contractors.

Granted, I don't know if reversing the orders of things in WHERE clauses, doing a rain dance, and hoping for the best would have been necessary if the original author of our schema hadn't made some critical mistakes. Like using text fields for enums--that boner seems to have confused the bejeezus out of the planner on numerous occasions.

Then again most big sites start out as some dude learning databases for the first time and making major mistakes. So anyway, I question the 0.1% figure Berkus throws out there.

<comment><USER15> _to get pgsql to force a particular index, and forgo destroying the site for another few days._

It's just a belly feeling but I smell a deeper problem in there. If your live queries are so complex as to require advanced massaging then perhaps you missed to collect some low-hanging denormalization or caching fruit earlier in the game.

The PostgreSQL planner has always done a flawless job for me in terms of choosing between indexes and tablescans - on reasonable queries. I can count the occasions where it made obviously bad decisions on seemingly simple queries (due to thrown off stats after some slony confusion) on two fingers.

However, if your site depends on cascades of sub-queries to build interactive views then I'd first take a step back and re-evaluate your persistence strategy before putting blame on the database.

 _if the original author of our schema hadn't made some critical mistakes_

Okay, perhaps my belly feeling isn't too far off?

<comment><USER16> _The PostgreSQL planner has always done a flawless job for me in terms of choosing between indexes and tablescans - on reasonable queries._

While that has been my general experience as well, things can break down at the edges sometimes. I sometimes have to join 200m-row tables that don't fit into RAM (though individual partitions of the table do), against 1-10m-row tables of new data. This task is probably something more suited to Hadoop, but we're using PostgreSQL for it as that's where the data is.

With a plain vanilla join using indexed columns for the join, the smaller table is scanned while an index is used for larger table on the joined column. This is reasonable except for the fact that this results in a lot of random IO, so if you don't have an SSD or something, this will be slow. Of course, you can just increase the random_io_cost or whatever it's called, to adjust costs estimated by the query planner. I believe that results in a sorting operation for the tables before the join. Sorting can be better compared to a thrashing disk, but still slow.

The core issue is random IO here, so how about just warming up the cache for that particular partition's index/table with a big sequential read of the entire index/table, and letting it use the index-based query plan as usual? The sequential read takes a minute in the worst case for a single partition, but then when everything's cached.. BAM. The join query runs in 2-20 seconds (for a single partition, one of 26). If Postgres were psychic, it could have done this itself instead of making me force caching with an otherwise pointless extra query. But I suppose that behavior would usually not be desired, as it would destroy the existing cache. In this case, destroying the older cache was the right thing to do, but Postgres can't know that.

The whole point of this comment being: DBAs matter and you can't rely fully on the query planner to do what you may expect. Even a scrappy one like me (I'm mostly a programmer, the DBA stuff just comes with the territory for me). This is partly in response to a comment elsewhere on this page, mentioning how the query-planner is better trusted than a DBA - I wish it were always so.

<comment><USER17> That doesn't sound right. In default conditions, PostgreSQL will shy away from doing index scans unless you're operating on <0.1% of the table. The planner builds a selectivity figure for the table based on table dimensions, index statistics, the operator used (= is more selective than <), and of course it's estimation of cost for the various operations needed to run the query. The decision to perform an index or sequential scan usually has little to do with the table size ratio during a JOIN. It's likely your index statistics were poor and/or the server configuration was messed up.

<comment><USER16> You're probably right, I'm an amateur at this. My current solution works pretty well, but I should really drill into what may be wrong here.

If you don't mind me asking, what in your estimation would be an ideal query plan for a join of the type I've described? A hash join? Also, I've left statistics to their defaults and run ANALYZE after bulk data uploads (the only time data is written), but I'll try bumping statistics collection up and running VACUUM ANALYZE again. I don't know what server configuration could be messed up to cause something like this; I have my memory settings (shared_buffers, effective_cache_size) set up fine, and cost parameters have been left alone. Other configuration settings I've changed shouldn't be affecting read queries.

</comment></comment></comment></comment></comment><comment><USER18> Ouch, I'm not a DBA nor advocating for hint support in PostgreSQL but were I either, I'd take Josh's broad brush of "old", "lazy" and "ignorant" unkindly.

PostgreSQL is indeed an awesome technology, I'm using it again after several years of working with folks who were stalwart MySQL adherents. I'm glad to be back. The query planner, among many other things, is a huge improvement.

</comment><comment><USER19> SQL is a declaration of "what should be done". Hints - procedural directives, ie. "it should be done this way". Modern optimizers do pretty nice job so hints aren't ususally necessary, yet taking away control which is extremely important in 10% cases just because it isn't necessary in 90% is just plain stupid. That one more reason explaining why Oracle or MS can charge so much - because no viable alternative.

And i fail to see how supporting hints would prevent improving the optimizer. Oracle does it extremely well. MS seems too.

<comment><USER12> MS and Oracle also have large teams of people that work full-time on improving the optimizer. With, the handful of people with the skills and inclination to work on the Postgres optimizer don't view hints as a high priority.

<comment><USER19> there is a difference between statements : (typical marketing style BS coverup of missing feature) "you don't need hints" and (honest) "we think that our limited resources should be all working on optimizer. In our view the priority of a few percent of cases where hints are necessary is much less than the rest that is covered by optimizer".

The latter shows that the people are rational and open to reason. The former shows that ideology and resulting BS byproduct have overtaken the project.

<comment><USER12> I agree that saying that hints have zero value is silly, but the blog post is just Josh's opinion. If you read the discussions on this topic on the pgsql- hackers list, you'll find a more balanced view.

</comment></comment></comment></comment><comment><USER20> I've had to use hints from time to time on stored procedures in SQL Server. I had to use WITH RECOMPILE more than once because of the server generating query plans that would take 60 seconds for one input and .002 seconds on another. The plan seemed to be optimizing for a weird case that made no sense for the general case. Adding that option made it work in .002 seconds for every call. I never noticed plan generation overhead doing this.

In MySQL I've had to use FORCE INDEX usually when sorting on a large set where the primary key was used on the table or was available for sorting but when that wasn't what I needed it for. I used force index on the column I was sorting by and that improved performance by around 50% maybe but then I ended up caching the results because the worst case was still too slow for my taste and they didn't need to change as often as I was executing this query (it was a random sample of the latest actions on a site).

These are edge cases I suppose but I'm glad they're there for me when I need them. I respect their reasons for not including them though as I normally see them used incorrectly or just strangely. (e.g. insert into table (nolock) (id, blah)...)

<comment><USER21> You can take NOLOCK out of my cold, dead, hands! That one is a lifesaver for reading from terabyte-sized OLTP tables with 100/sec inserts...

I really like what the guy is saying in the article. However, it's the perennial idealism vs pragmatism argument. And, i'll put my hand up for pragmatism: I've needed hints in the past, because like everything else in this world, query planners simply aren't perfect.

<comment><USER22> In SQLServer in general it's better to write one time SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED, than write hint NOLOCK agains each table.

Result is the same.

This is not a hint for planner. You can write it for PostgreSQL also (I mean SET TRANSACTION).

</comment></comment></comment><comment><USER23> I don't know that it's as rare to need query hints as the author points out. Perhaps Oracle's query optimizer is bad in some ways, but I have seen plenty of queries that greatly benefitted from query hints. Sure, you might not need them for _most_ queries. But once you start dealing with more complex or bigger queries, they become more important.

</comment><comment><USER24> Based on the excellent responses here, the conclusion is that this guy is completely wrong and that query plans are definitely useful and even required. Now I think it is interesting to figure out how it can be that he is so wrong. It seems to me the simplest explanation is that the pg devs aren't eating enough of their own dogfood and are just unaware of a certain uses their product is put to? However, I would expect them to have been supplying consultancy services to exactly the users that come across these kinds of problems. In that case, the 'no query plans' must also have become stuck in their mindset, to remain unmodified after coming across practical uses cases that actually require them.

</comment><comment><USER25> Developers need control of their tools in the 0.1% case. When you are dealing with complex systems, you will start running into that 0.1%. Either due to a poorly designed scheme, legacy cruft, or unexpected growth.

When things get hairy in other languages, sometimes it make sense to bypass the built in controls. Go around the system libraries, implement your own, or even drop into assembly. The same should be true of a query plan. Yes, your query planner (compiler) is going to get things right 99.9% of the time, but man when it gets it wrong, it's worth it to have tools to deal with the other 0.1%.

<comment><USER26> In my experience the best way to tackle those problems isn't to wrestle with execution plan hints, instead break the query into 2 and use a temp table or table variable(s) for the inbetween data. This has the additional benefit of helping you debug the cause of the slowness. And there's no way for the planner to stab you in the back.

</comment></comment><comment><USER27> It doesn't have hints but you can turn off the ability for the query planner to use certain types of operations. And let me tell you that the Postgresql query planner does screw up. And the way to fix it is to turn off its ability to use certain operations and then run your query, yes it's not a hint, but it's necessary to make postgresql work.

They should change the FAQ to 'we don't have hints because we're so arrogant that we think our query planner is perfect'.

</comment><comment><USER28> I like this approach. The attitude here is "Let's invest our time in making the optimiser good, instead of spending time implementing ways for users to walk around it."

</comment><comment><USER29> If they trust automatic analysis so much, they should rewrite PostgreSQL in Haskell and trust GHC to generate C-like performance assembly.</comment>