Study is forcing economists to rethink high-deductible health insurance - <AUTHOR> http://www.vox.com/2015/10/14/9528441/high-deductible-insurance-kolstad <comment><USER2> They missed the real issue. There is no way you can actually price-shop. The providers and the insurance companies keep their final negotiated price secret. All they will let you compare is cash prices, or a select list of agreed-upon common services. It is absolutely impossible to get the bottom- line cost for any procedure to compare between providers. Beyond that, it also takes up too much time to find the medical code from one group, confirm with the provider that is the code they will use, talk to a third group to see what the cash price is, and then contact your insurer to make sure they actually cover that code.

<comment><USER3> This is the real heart of the issue. There's no transparency, and the health- care 'market' is set up in order to discourage shopping altogether.

It's a shame, because a free market is the best way of insuring that resources are fairly allocated.

<comment><USER4> Even where there is at least gross-level of price transparency. Like in the case where you have to choose between an Urgent Care Center and an Emergency Department.

The former is much cheaper than the later in most cases. But how do you self- triage? There are 4 possible outcomes:

1\. You go to an UCC and they treat you. You made the right choice and saved the most.

2\. You go to an ED and they treat you but an UCC would've been enough. You made a mistake and pay more than you should.

3\. You go to an ED and it was the right choice. Congratulations you saved money and got prompt care.

4\. You go to an UCC, they can't treat you and refer you to an ED, you need need to pay _both_ and quite possibly an ambulance ride for the transfer.

Now suppose you have sharp abdominal pain... Medically, the right thing to do is go to an ED (especially if you still have an appendix.) But will you gamble that the cheaper UCC is the right place? You have some chances of being right.

<comment><USER5> _2\. You go to an ED and they treat you but an UCC would 've been enough. You made a mistake and pay more than you should._

In this case ED will often send the patient to their affiliated UCC located nearby.

</comment></comment></comment></comment><comment><USER6> It's a bit baffling, for someone living outside the US, why any reasonable person would have expected that sick people, of all people, would spend a lot of time comparison shopping.

The appendectomy example really underscores the insanity of the idea -- who in their right mind, when facing a burst appendix, _would risk their life by stopping to do some in-depth reading of price lists to find the best deal on continuing to live_?

Your only concern at that point should be getting yourself to the nearest emergency ward. It's crazy to think that you're supposed to stop and figure out if Big Al's Discount Appendectomies three towns over is having double- coupon Thursdays.

<comment><USER7> Emergency care is only about 2-8% of healthcare costs. Most of what we spend on healthcare is for chronic conditions that you have for years, decades, or your lifetime.

@waynemr's comment gets it right -- the real issue is that prices are secret in healthcare, and quality metrics are nonstandardized, so there's no information on which to base your decision.

The Affordable Care Act is laying some of the groundwork for healthcare markets. For example, look at the experiments in bundled pricing that CMS is starting: [http://innovation.cms.gov/initiatives/ccjr/](http://innovation.cms.gov/initiatives/ccjr/)

<comment><USER8> No dude. The real issue is that you don't consider health care to be a human right or even something that should be a high priority for wellbeing.

I have _never_ , _ever_ had to even think two seconds about going to a doctor for any ailment I've ever had. I would shit my pants if I had to take monetary decisions with regards to that.

</comment></comment></comment><comment><USER4> The pricing of healthcare is so opaque and full of subjectivity (Is this doctor good? How much do I need this?) I don't see how a patient can make an educated decision.

It's exactly like getting your car fixed if you know nothing about cars and engines. You go to a garage when you can't avoid it, you close your eyes and you pray you won't get fleeced. And if you find a garage you trust, you always go there and you don't shop around to 5 places to get competing quotes.

<comment><USER9> In addition in healthcare none of the 5 places will actually give you a meaningful quote.

</comment></comment><comment><USER10> I think the opacity argument is beginning to shift. I was listening to NPR on my drive home the other day and some people have really started working on this very interesting problem.

There are now 3rd party companies that work with insurance companies that will show you the costs that they incur from specific providers. Not only that, but there's an economic incentive for you to skip on going to the highest priced location for drugs or a physician. An example is MRI costs. They differ wildly in costs for the exact same thing. Anywhere from $1K-$10K. There really isn't much difference in either company in terms of quality of service. So what happens is that the insurance company hedges their bet. They'd rather pay you a certain amount of money (say $100) to go to maybe a $2K place instead of the place in Beverly Hills due to their pricing.

So it's changing, but it's taking some time... Either way, it is a brilliant economic mechanism to solve the problem of rising costs in healthcare.

<comment><USER11> I heard it too, and also found it interesting. For the curious, its Planet Money[0], Episode #655: Pay Patients, Save Money

[0] [http://www.npr.org/podcasts/510289/planet- money](http://www.npr.org/podcasts/510289/planet-money)

<comment><USER12> and the name of the company in the episode is Smart Shopper, and a quick google search appears to the website to be [https://www.vitalssmartshopper.com](https://www.vitalssmartshopper.com). Unfortunately their list of U.S. states doesn't include where I live.

</comment></comment></comment><comment><USER13> >Instead, both healthy and sick patients simply used way less health care.

>This raises a scary possibility: Perhaps higher deductibles don't lead to smarter shoppers but rather, in the long run, sicker patients.

Spot the leap. The author implicitly assumes health care usage/health care spending is correlated to health care outcomes, but there is _highly_ convincing evidence that that is not the case. So: has Sarah Kliff (senior editor, "Writing about health care.") never heard of the Oregon Medicaid health experiment, or is she just pretending it never happened?

Also the impossibility of price-shopping is obviously a huge problem.

<comment><USER14> The Oregon Medicaid experiment was very interesting but clearly wasn't the definitive word on emergency use and outcomes, and as to whether Kliff had heard of it, here's one of many articles she's written on the subject;

[http://www.washingtonpost.com/news/wonkblog/wp/2014/01/03/ne...](http://www.washingtonpost.com/news/wonkblog/wp/2014/01/03/new- research-shows-medicaid-increases-er-trips-oregon-has-a-plan-to-stop-that/)

</comment></comment><comment><USER15> I don't understand how health care "economists" can even imagine that Doctors are fungible commodities. Last I checked we're still using actual humans for the task, we haven't invented good, working medical droids just yet. From that perspective of course people aren't going to shop for a Doctor like they shop for a dishwasher: most people pick a general practitioner/family practitioner based upon their social culture. How many people's GP is an old friend or a family member? People develop attachments for their GPs. Their GPs are generally a part of their extended community. It's then their GPs that recommend their Specialists, typically through the GP's own social network/community... Shopping for a Doctor as if they were a dishwasher is a bit like trying to shop for a new pet or spouse on the Home Shopping Channel.

</comment><comment><USER16> The article is based on an odd premise. It sounds like the employees in the study were given an HSA with a deposit equal to their deductible, so they could spend up to that limit without incurring any personal cost. The trick here is that an HSA _should_ encourage you to eliminate unnecessary spending because you get to keep the money you save and bank it for bigger health care costs down the road. I'm 27 and healthy, had a really uninformative annual physical last year where my doctor bilked me for a totally unnecessary EKG, and as a result I skipped my annual physical this year and am banking the savings for a serious illness in the future. The study should actually look at whether people went to the doctor less over the long-run for serious issues -- like not going to the doctor for symptoms diabetes, which left untreated can cause much larger long-term costs to the system -- rather than whether they simply avoided "valuable" medical visits, which in this case as "preventative care" has unclear value.

EDIT: The answer to her question "Why wouldn't this group get the care they wanted, pay for it with the HSA, and just run through the out-of-pocket spending earlier in the year?" is that "the care they wanted" is treatment for a catastrophic illness later in my life rather than dubiously valuable preventative care right now. I'd rather skip a doctor visit for the flu and have more money banked in the future for cancer treatment or a heart attack.

<comment><USER17> Yeah, this is close to my reaction as well. Higher deductible plans lead to less spending; confusion among the economically illiterate ensues.

Nothing to see here.

</comment></comment><comment><USER18>
      > In 2006, about one in 10 employees had a health insurance
      > deductible over $1,000. Today? About half do.
      > To health economists, this sounded like good news; they've
      > long theorized that higher deductibles would force down
      > health-care costs. The idea was that higher deductibles
      > would make patients become smarter shoppers: If they had  to
      > pay more of the cost, they'd likely choose something closer
      > to the $1,529 appendectomy than the $186,955 appendectomy
      > (yes, some doctors really do charge that much). This would
      > push the really expensive doctors to lower their prices so
      > cheaper physicians didn't steal their business.

This seems so illogical that I think I must be missing something. Presumably, the patient pays the deductible out of pocket, and the remainder is covered by the insurance.

      Case A:  $1,529 appendectomy.
      Patient pays $1000, receives "cheap" appendectomy.

      Case B:  $186,955 appendectomy.
      Patient pays $1000, receives "luxury" appendectomy.

Why would the "health economists" think that a patient be incentivized to choose the less expensive option? Rightly or wrongly, the frequent presumption is that more expensive services are higher quality. Why would a patient go out of their way to choose the lower priced option if they are paying a flat rate?

      > But when the researchers looked at why spending dropped,
      > they found it had nothing to do with smarter shopping. The
      > average price of a doctor visit wasn't dropping.
      >
      > Instead, under the high-deductible plan, workers just went
      > to the doctor way less. The paper finds that "spending
      > reductions are entirely due to outright reductions in
      > quantity." Workers did use less "potentially wasteful
      > care," like imaging services, but they also cut back on
      > "potentially valuable care," like preventive visits.

Clearly the incentive is for the patient to avoid costs that are less than the deductible. Lo and behold, the patients chose to avoid costs that are less than the deductible (doctors visits and preventative care), and to ignore cost in cases that are higher than the deductible (surgeries and emergencies). How can this be construed as a surprise?

<comment><USER19> Plans with a deductible "over $1,000" usually have deductibles _well_ over $1,000, e.g. $3,000 to $6,000+, and only cover something like 20% costs _after_ the deductible.

So, neglecting the annual limit, the $1,529 vs. $186,955 appendectomy situation wouldn't be $1,000 vs. $1,000, it would be more like (with $3,000 of unused deductible at the time of the procedure) $1,529 vs. $39,791.

<comment><USER18> I was concentrating on the lousy logic expressed in the article rather than specific numbers in the study. I think my point stands, since the total cost for any appendectomy is almost certain to exceed any deductible.

As you point out, the real difference in costs (and the factors that actually might influence patient behavior) are the percentage of coverage after the deductible and the levels for out-of-pocket max. As you probably noticed, these weren't mentioned in the article.

I'd hope the actual paper has a better discussion of this, but it doesn't seem to be publicly available: [http://www.nber.org/papers/w21632](http://www.nber.org/papers/w21632)

One of the authors do have some summary slides available: [http://eml.berkeley.edu/~bhandel/wp/BCHK.pdf](http://eml.berkeley.edu/~bhandel/wp/BCHK.pdf)

From Slide 8, it looks like the studied group had a 10% in-network copay, with a $6,250 out-of-pocket cap.

</comment></comment></comment><comment><USER20> There was a recent Planet Money episode about a company that will shop around for medical care for you and give you a cut of the money that the insurance company saves:

[http://www.npr.org/2015/10/09/447098694/why-most-people- don-...](http://www.npr.org/2015/10/09/447098694/why-most-people-don-t-shop- around-for-medical-procedures)

</comment><comment><USER21> What a big shocker! You are telling me that a sick person would not try and spend many hours comparison shopping for doctors with poor metrics for quality and low pricing info and then make a possibly large upfront payment based on a guess? That they would just say - forget it; and get much sicker?

Universal Medicare with single payer systems is the way to go as most of Europe and Canada seems to have figured out decades ago.

</comment><comment><USER22> this is the only industry in the US that has an anti-trust exemption. So they can charge people differently for different services and they do not have to publish prices.

I think if the employees in the study were given a web interface that is required for booking any appointments, they could get better results. The booking interface could show all qualified specialists that work with their plan, and they could also display prices for specific treatment or average costs for a visit to each particular specialist. So instead of forcing people to price shop, they get to see the prices at the point of booking.

<comment><USER9> This might work. I have a high deductible plan and the only rational way to reduce costs is to not go to the doctor.

Otherwise it's all guesswork since you don't have any information. You can't get any prices and no quality numbers are available.

A website with binding/fixed prices, some kind of quality number or reviews and whether the doctor will take your insurance would help.

<comment><USER23> They can't do it.

They change less than cost for medicare and medicaid patients. The only way to make it up is charge more than cost for those with better health plans.

Forcing them to show prices would make that impossible.

They would have to charge cost for everyone - but that would cause the US government to pay probably double in health costs, no one would go for it.

So we are left with a sort of hidden tax on anyone with a private health plan.

<comment><USER9> Why is it impossible to show prices? Medicare and private insurances have defined price lists. They should just be forced to make them public.

A system where one group gets charged too little (is that really true in the case of Medicare? I doubt it) which then forces another group to be ripped off in a hidden way doesn't make sense and hurts the country.

<comment><USER24> It is absolutely true in the case of both medicare and medicaid for certain things (not everything). My dad ran business and R&D for pathology ta state level hospital at a major university and he often had to figure out how to continue offering services to medicare patients by making up the money somewhere else when medicare wouldn't even cover the cost of materials.

</comment></comment></comment></comment></comment><comment><USER25> We are about to switch to high-deductible only health insurance plans here at work (they had a PPO and an HMO but they're getting rid of them). I already avoid seeing the doctor more than I should. I don't need even more reason to avoid them. Ugh. This isn't good.

</comment><comment><USER26> The study isn't forcing anyone to do anything.

High deductibles save health insurance companies billions. We already know that health insurance companies own congress: that's why we don't have a sane socialized medicine system in the US. So high deductibles are here to stay. The only thing that's going to change that is consumers becoming more savvy; economists have no power here. They _should_ , but they don't.

</comment><comment><USER27> Economists might be surprised. Actuaries are not. In most actuarial health cost models, cost sharing induces changes in utilization, not cost-per- service.

</comment><comment><USER28> Health insurance in America is actually about managing cash flow rather than managing an insurance policy. After extensive research and data investigations, you'll discover that the actuaries at insurance companies are fantastic at what they do: you'll usually end up spending the same amount out- of-pocket in a year no matter the plan--the question is just would you rather spend more at the beginning of the year on lower premium plans, or spread out costs evenly throughout the year on higher premium plans.

A few associates and I are working on a (very very alpha) tool for plotting out actual expenditures for all plans for which you qualify, giving you a report of where you'll spend the least based on your expected health events. The tool saved me over $4,000 this year by guiding me towards a better health plan than what I had originally picked out when I switched to self employment.

If interested, check out [http://lucentary.com/](http://lucentary.com/) (but please keep in mind we're still fleshing out an MVP!).

<comment><USER28> And as a side note, for my family I actually love high deductible plans when I can get the deductible around the same dollar figure as my max HSA contribution. Once I can hit the deductible for an individual, free healthcare!

I never hit it, because I have very few health needs. But add a pregnancy and delivery into the mix and you're set. It's the opposite effect of what the theory in the article supposed, but just one data point.

</comment></comment><comment><USER29> This was obvious. The whole point of high deductible plans and PPOs is to push costs onto patients who then don't choose care unless it is absolutely necessary. Preventative care suddenly becomes $400 a visit instead of $40.

</comment><comment><USER30> Sounds like the "economists" didn't think too much the first time around.

</comment><comment><USER31> I understand that the economists who made this study are not the ones calling the shots on the (bad) way the health insurance business is conducted, but man, they're pretty dense if you take the article at its word.

"To health economists, this sounded like good news; they've long theorized that higher deductibles would force down health-care costs. The idea was that higher deductibles would make patients become smarter shoppers: If they had to pay more of the cost, they'd likely choose something closer to the $1,529 appendectomy than the $186,955 appendectomy (yes, some doctors really do charge that much). This would push the really expensive doctors to lower their prices so cheaper physicians didn't steal their business."

Even the sloppiest and most cursory examination of human behavior refutes this line of thinking-- patients don't shop around for life-saving medical procedures because they need them urgently or semi-urgently, meaning locations close-ish to home are the only choices, if there's even more than one choice that accepts their insurance. For plastic surgery, sure, whatever. For a bypass? No. I guess this gaping hole in their analysis went unnoticed or somehow explained away by their insufficient economic model.

"The new paper shows that when faced with a higher deductible, patients did not price shop for a better deal. Instead, both healthy and sick patients simply used way less health care."

I would expect to see more plans converted to high-deductible plans after a finding like this percolates through the insurance companies. This behavior is health insurance working as designed: unused, making a fat profit for the insurers while the patients scrape by until they're in critical condition and can't refuse care for the sake of cost-cutting on their end. This system is evil, and it is intentionally evil for the sake of producing more money.

""I am a little bit surprised at just how poorly patients were able to do when looking at very similar products, like MRI scans, and with a shopping tool," says Kolstad, an economist at University of California Berkeley and one of the study's co-author. "Two years in, and there's still no evidence they're price shopping.""

This man is a fool of the highest order, truly an overflowing septic tank of stupidity unparalleled in obliviousness even when compared to a slime mold. How are patients supposed to understand comparisons between medical procedures? "A shopping tool" would be too simplistic, anyway. You need a real doctor to answer questions about care choices.

"Workers' health spending dropped, and did so quickly. Average per-patient spending fell from $5,222.60 in 2012 to $4,446.08 in 2013. That's about a 15 percent decline in a single year — and it held true across all types of health services. Between 2012 and 2014, there was a 25 percent drop in emergency room spending, an 18 percent decline in physician office visits, and a 6 percent decrease in mental health services."

Great job, the patients have been made sicker as a result of cost-cutting measures. Looking forward to seeing more high-deductible plans.

Finally, there is no "shopping around" in the sense these economists mean. Try getting a straight itemized quote for a given medical procedure before you get it. Try comparing that with someone else's quote for the same thing. At this point, you've probably wasted a few weeks on the phone/email trying to finagle these things out of the horrendous systems in place. Then, you can cross reference the results with your insurance benefits, and see how badly you're going to get fucked. Yeah, right.

<comment><USER23> I wouldn't be quite that harsh, but I agree with you overall. I don't understand why in the world they thought this would work.

Are they confusing deductibles with co-insurance? I know the field is confusing, and that sort of confusion is the only way this study makes any sense.

With a deductible ANY non-routine spending means you go over it. So why bother price shopping? It makes no difference.

With co-insurance you pay a portion of every bill, so in that case price matters.</comment></comment>


I don't care about your code challenge - <AUTHOR> http://blog.mattbasta.com/i_dont_care_about_your_code_challenge.html <comment> at-fates-hands I actually had one of these and I won't do another one either.

Here's the gist of how it went down:

1) first part was the "personality" interview. Two "HR" people blew smoke up my ass for about 30 minutes about how awesome my site and my work was, then asked me to do a "code challenge". I agreed - I mean, how bad could it be?

2) The code challenge. I was given a PSD with a straight forward design. Asked to code it, make sure it worked in legacy browsers, publish on a server and show them the code on GitHub

3) Passed and then was asked to come back and explain some of my coding decisions. I was told this was more for show and I was already in effect hired, I just had to pass this last hurdle. It turned out to be a very confrontational, inquisition type of an interview. Not at all the type of laid back code review I'm used it.

I got pretty frustrated by the two "Senior" devs who seemed to take great pride in tearing my code and my decisions apart. After my first heated exchange with one of the devs, I actually asked them how many of the sites listed in the gallery on their site they actually tested and validated the code on. They said "All of them, why do you ask?" I told them I actually ran all of their sites through the W3C validation site and none of them passed. Every single one had over 60 errors on it. I basically said, "If standards are so important to you, then why are you holding people who don't even work for your company to a higher standard than those who actually do?"

There was a long silence and then I finally spoke up and said, "Clearly, this company has its priorities in the wrong order. Thank you for time, but I'm not interested." and got up and walked out.

I got several worried calls from the company recruiter, but by then I was so pissed at how much of my time they had wasted, and vowed never to do something like it again.

<comment><USER3> I'm amused that you take so personally the concept of a professional who's not familiar with your competencies trying to evaluate your ability. Which by the way is more than just pumping out code, it also involves defending difficult decisions to your peers. If you can't handle a little pressure from a code review, find a new career.

<comment><USER4> There's a difference between evaluating abilities and trying to engage him in a pissing contest. Interviews should determine if the candidate will deliver what he's being paid for, nitpicking and flamewars can be saved for future coffee break entertainment. "It's not my solution therefore it's shit" is not what code review should be about.

</comment></comment></comment><comment><USER5> Many years ago (wow...almost 10 now...I'm so old...) I was given a phone screen and at the end of the screen I was given a question to code a solution to and email it in. Since this was for a job across the country, I figure what the hell.

The problem was - A man has to get a fox, a chicken, and a sack of corn across a river. He has a rowboat, and it can only carry him and one other thing. If the fox and the chicken are left together, the fox will eat the chicken. If the chicken and the corn is left together, the chicken will eat the corn. Write a program that will help the man make the correct decision and explain the operational complexity.

I was given an hour to come up with a solution. I worked on it about half-an- hour coming up with the algorithm. And then for some reason my brain keyed in on the "write a program" part of the question. So I wrote the following in C++ (from memory so may not be exact) -

      int main(int argc, char** argv) {
          std::cout
              << "Man takes chicken, leaves fox and corn, and returns" << std::endl
              << "Man takes fox, leaves corn, and returns with chicken" << std::endl
              << "Man leaves chicken and takes corn, leaves corn with fox, and returns" << std::endl
              << "Man takes chicken back across" << std::endl;
          return 0;
      }

It was a program that would tell the man the correct way to get everything across the river.

Must have been an OK answer because I got flown up and ended up working for them for 4 years (before a series of bad jobs that have finally lead back to something good).

</comment><comment><USER6> _You have no idea how the "challenge" was completed. A candidate could have rent-a-coder-ed the project, refactored it a bit, and slapped their name on it._

Sure, if you ask no follow-up questions after the challenge is completed. However, with a few well-placed questions about the implementation and some inspection of the code, it'll be quite clear whether you're talking to the author or not.

We once gave a (non-web-dev) programming "challenge" (I thought it was a joke/insult given the candidate's claimed experience) to a senior dev candidate. Mind you, this was on-site. I even tested it by putting our newly hired grad to the test and he aced it within 20 minutes. We gave the candidate an hour to complete it (and added a 30 minute extension when he was struggling).

The challenge involved developing something against an SDK (interacting with a camera) and we gave him internet access and the SDK examples to work off of. In the end, he produced something not quite what we were asking, but he was sure to claim it as his work. Although it wasn't ideal, I was prepared to say that he "passed" the assignment, because he was _close_ and I was sympathetic to factors of stress and nervousness.

As he was toured around the facility, I inspected the code and saw that he was completely passing off one of the SDK examples as his own work (diffs showed only two variable names had been changed). The reason his code didn't quite do what we wanted, is that he wasn't even able to tweak the SDK example to do what we asked. He was a complete fraud and never mind technical ability, we couldn't have dishonesty like that on the team.

Before that interview I thought those kind of challenges were a joke, but after this I saw their value. Of course, their value depends highly on how their are implemented and targeted, but dismiss their usefulness at your own peril.

<comment><USER7> This is a lopsided view. You don't indicate how long your "newly hired grad" was working there, whether the "SDK" was an in-house SDK or a third-party one, what the candidate's resume skills were in comparison to the kind of programming around the SDK your test was designed for, etc.

These are all relevant details, because except for the most trivial of SDKs and their appropriate domains, a thoughtful, skilled developer might spend an entire hour simply reading through the SDK and/or its documentation to get a handle on its capabilities.

So I don't get that your story is an anecdote in support of the potential usefulness of code challenges. Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet _my_ requirements.

<comment><USER6> >You don't indicate how long your "newly hired grad" was working there, whether the "SDK" was an in-house SDK or a third-party one, what the candidate's resume skills were in comparison to the kind of programming around the SDK your test was designed for, etc.

I wanted to avoid writing a novel, so I left out some of those details you're interested in. Trust me, when this "test" was first suggested, I was wary of it myself and did everything in my power to make it very achievable. That's why I considered it a joke, it was not nearly difficult enough to assess the level of seniority this candidate was claiming. This was someone with 10+ years of development experience, in the same field, experience with the language and environment we used. On paper and even on the phone, he made himself sound like the perfect candidate. In fact, before the on-site interview, I thought he had it in the bag and was excited to get some help on the team.

The SDKs in question were for a camera (to get images from the camera) and OpenCV, which is well known in imaging circles. The test was to receive a video stream from the camera (there was an example showing how to do this and I made it clear I didn't mind if he made that example his starting point) and to subtract consecutive images to create an image representing the "difference" between the two images. Not a meaningful result and one could barely call it image processing, but it would show us the candidates abilities to interact with multiple well-documented SDKs to accomplish a very simple task. If someone who claims to have years of experience with image processing can't accomplish subtracting two consecutive images and drawing the result to the screen, given all the tools to do this, there's a big problem.

Our new grad was mostly a tester and an occasional developer. He had less than a year of experience. He completed in 20 minutes without supervision.

>Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet _my_ requirements.

While I am fully aware that there are severe limitations to the usefulness of a test like this, in this situation, it saved us from a terrible hire (this test wasn't the only indication that he wasn't who he claimed to be).

In all honesty, you sound like a fairly thorny/difficult person to work with and in your hypothetical scenario where the test caused you spurn the position, it would have inadvertently done its job.

<comment><USER8> I'm often surprised by what people will take umbrage at. It's one thing to decide that it's a seller's market, and that therefore you'll interview somewhere where they'll buy you lunch and offer you a six figure salary and generous share allocation after nothing more than a cozy chat. But it's quite another to become actively offended by the effrontery of a company demanding that you demonstrate the very skills they're hoping to hire you to apply on a daily basis!

What are people worried about? I have to say I can't help but assume the worst. Because what have they been doing for their career as a professional programmer, if when asked to actually write a program, they can't actually do it? Even with an example right in front of them?

Don't forget that in this particular example we have a supposed senior programmer! Somebody applying for this sort of job should, I feel, be able to actually program. Even if they've never seen the library before. In fact good candidates should be barely fazed by an unknown language! What is the "senior" bit for, if not this? At the very least, perhaps the candidate could admit that they seem to have found themselves interviewing for the wrong sort of role, and one that makes better use of their other skills would be more suitable? That might be a better tactic than storming off in a huff.

But I've never stormed out of an interview in a huff - so what do I know.

<comment><USER7> There's a reasonable expectation to have to demonstrate one's skills in an interview. Whether any given test or set of questions actually is valid for doing that is another matter entirely.

Put another way: it's just as unreasonable for a company to expect a developer's skills to be measured by contrived tests or to demand they spend any unpaid time at all doing "work" (e.g. projects), as it is for a candidate to expect a company to effectively kiss his butt and lay out red carpet just because he decided to apply.

</comment></comment></comment></comment></comment><comment><USER9> Even "small" programming assignments take a lot of time. Several years ago, I applied for a job as a developer for a financial trading system. The take-home assignment was to build a small trading system that received buy and sell orders from the command line, and executed trades when the price matched. The input was just: buy/sell, price and number of shares.

This functionality is pretty simple, but it still took many hours to code. My feeling was that they thought it would perhaps take an hour to do. It _was_ fun and interesting to do, but it took much longer than I thought. I think the companies giving the challenges should have their own engineers do the task, so they get a good handle on how much they are asking from the applicant. And of course it also depends on if it is a buyer's or seller's market (jobwise that is, not in terms of my "trading" system). If there is a lot of competition for developers, maybe they should give too big projects.

</comment><comment><USER10> Sigh - yet another hiring process complaint post. As an employer hopefully I can shed some light here.

a) If you are trying to hire 5 engineers, say, you need to talk to ~25 candidates at least. It's a competitive market. b) If you aim to talk to 25 candidates in a short period of time, you need to have SOME standard way to evaluate candidates if you don't want to just throw darts on the board. Being sloppy with process is bad for companies and the engineers and a waste of everyone's time. c) In order to create a standard process, you have a very limited # of options to work from. I've seen comments like "do a co-working day to get a real sense" or "do a contract and see them work with you daily" positioned as realistic options. They are not. If an engineer has an existing full-time job they don't have the time to do co-working days or contracts. d) The other options are 1) whiteboard interviews 2) coding challenges 3) some combination thereof. e) Lots of candidates hate whiteboard interviews. Lots of candidates hate coding challenges too. Neither is a perfect system but it's better than doing nothing or asking for co-working time. f) We use coding challenges at the top end but we time-box it to 3 hours. If someone asked us to pay for the time, we would happily do so. g) Our coding challenges have 3 different problems so candidates can pick whichever aligns best to skills. One problem is JavaScript-heavy and 2 others are Ruby/Rails-heavy. h) Several engineers on the team solved the coding challenges on a timed basis before we put them out to candidates. Doing otherwise would be stupid/unfair. i) In the few cases where we've been able to do 1-2 co-working days, we have offered to pay for the candidate's time. Not full freight consulting dollars but something that indicates our seriousness. j) In this case, the OP got caught by a dumb coding challenge to build a full website. But the headline paints with a broad brush because coding challenges by themselves are not evil. They are a tool that can be used well or poorly. That's all.

<comment><USER11> I think the "25 to get 5" is aggressive. My experience is close to "25 to get 1".

Standard processes are important, but they need to respect everyone's time.

<comment><USER10> I was talking about 25 onsite conversations leading to 10 offers and maybe 3-5 acceptances. In order to have 25 onsite conversations companies likely have to talk to 125-150+ total individuals via phone/email/networking events. That's at the low end once all the processes are dialed in.

I agree that companies have to be respectful of candidates' time. People do talk to their friends before they engage with companies or accept offers and word can get around fast if companies mistreat candidates (as it should).

<comment><USER11> Ahhh - gotcha. I think our experiences are similar.

</comment></comment></comment></comment><comment><USER12> While I agree that the request to build, deploy and open source the code of a website is a bit too much, I think you are exaggerating a little bit on the why's you wouldn't do it.

"I'm not equipped to do your challenge" \- really?

"Engineers aren't different" \- again, really?

Rhetorical question: you don't want the job, do you?

I am not sure if the issue is with the hiring process, or with the candidates' ego here. Candidates. Plural. As the article's author doesn't seem to be the only one.

Apparently there is a lot of people that believe they are rockstars and they don't need the job, the job needs them. Challenges like this help to figure out non technical skills as well, like: how committed a candidate is; how much he wants that job.

I'd rather hire a junior that is committed than a senior that is not.

<comment><USER13> There are a lot of jobs going around these days. Employers no longer have the luxury to treat prospective hires poorly. This is the employer's problem, not the candidates.

Personally, I've asked potential hires for work product demonstrations. However, two crucial differences: 1) I paid them for their time, and 2) the task was on a real part of our infrastructure, the code they produced eventually being used in production. Everyone was happy with this situation.

<comment><USER14> I'd say that's a much more fair way to go about it. If you're going to ask someone to do work for you, that should be a paying gig, and it should be a task worth doing. If you're looking for a free sample a public repo or a task that fits within the parameters of a normal interview time would be better alternatives.

</comment></comment></comment><comment><USER15> The worst thing about code-challenges, is they never give feed back.

We're sorry, but what you produced is not good enough. We're not going to tell you what you missed.

At least technology tells you what you missed. If you have a standard, have it documented so that I'm not firing a bullet into the dark. Chances are I'm just as good as you; but you won't know because I haven't seen your coding style. You're on my project, you code like me. I'm on yours, you better believe I'll code like you.

And then there is the times they like to dismiss you for it but have some other reason and use it as a cover. So you worry about your code not being perfect and you develop a ritual to validate yourself before it goes through a compiler. Yep, I write my code three(avg) times now. 10 lines, over and over until it's a effing poem. It's called ocd.

Coding challenges, pfft, worthless. There is only one company that I did a coding challenge for in the past year and it's because they were really awesome people. - I'm here to make a product and work with a team. I'm not here to explain shit to you just so you can tell me to look up the documentation. If I take the time to explain my code to you, you better well explain why you think something is better rather then telling me to google it.

fufff, I needed that.

<comment><USER16> _> The worst thing about code-challenges, is they never give feed back. We're sorry, but what you produced is not good enough. We're not going to tell you what you missed._

This is most interviews in general, and I hate that. When interviewing candidates I try to give actionable feedback to each of them, regardless of whether it results in whiny emails back (and it often does). There was only one case in recent memory where I couldn't because he'd gotten literally nothing right while bragging about how fun it was to write thousand-line functions. Eek.

</comment></comment><comment><USER17> Building a full production grade website seems extreme, but giving a prebuilt website to a front end candidate and asking them to add a few simple pages or features is basically fizzbuzz type vetting. I also like it when coding "challenges" are timeboxed to say 1-2 hours. This keeps the time expectations reasonable for both the interviewer who has to read a bunch of candidates' challenges and the interviewee who has to do them for (potentially) a bunch of gigs.

The type of coding challenge a company asks (interview questions also) can be one of the most important data points to use to judge a company's culture...

<comment><USER18> Yes exactly, this is overboard. We have "take home test" (I hate that term but haven't come up with anything better) for both our front-end and back-end candidates. They're

(a) Short - 45 minutes to an hour

(b) Only as complex as necessary (for the front end we provide and end-point serving JSON, for example, so they don't have to worry about that)

(c) Real problems we've run into, distilled down into a form appropriate for an interview

(d) Made up of problems that we were having candidates do on the whiteboard before we developed the at-home test, meaning they're not super complex

I think that at-home coding is the perfect second step to an interview (after a phone screen). It's more realistic than whiteboard coding. You have the internet at your fingertips. You don't have the pressure of an interviewer sitting there. You have syntax checking. From my side, I don't have to waste 30 - 60 minutes on a bad candidate. And, assuming the code is good, reviewing it provides an awesome first hour of an in person interview. And the problems I give you to take home would have been given to you in person anyway, so you're not spending any more time than you would have if you'd just come in for an interview.

The issue here is that the tests he's being given are bad examples of a take home test, not that take home tests are bad ipso facto.

<comment><USER9> Have you (or people on your team) done the task yourselves? It's easy to underestimate the effort required.

<comment><USER18> Yes absolutely. They're all problems that we've solved in the process of building out our product. In addition to that, as I mentioned, we originally gave them as white board questions, so we're well aware of the issues that come up when solving the problems and the effort required to resolve the issues.

Our front-end test is a bit longer than the back-end test with a tradeoff of a shorter in person interview, but the front-end test shouldn't take more than 90 minutes.

One of our problems is a small matching algorithm. Another is a thread safety problem. The front-end test consists of pulling JSON data from a server, formatting it, and posting a single action back to the server. It sounds like we're in a minority though. Reading these other posts it seems like a lot of the tests are very onerous (build a small trading system? that is a ton of work).

<comment><USER19> > They're all problems that we've solved in the process of building out our > product

Have you gotten feedback on the questions? It's easy to say that it's a valid question, but when you're close to the problem, it's easy to make a mistake since you might encounter a specific sort of problem that others might not be nearly as familiar with.

The difference to me I think is that yours are very specific, and "build a small trading system" can be very general. If it's timeboxed, there might be only so much that you can do--i.e. you might not be expected to complete the task. You want to test someone's knowledge, without being too 'in depth'. A candidate that can solve a problem on his own in 30 minutes isn't necessarily better than a candidate who can solve the problem in 10 minutes with the assistance of Google.

0.02

<comment><USER18> Thanks for the feedback, I'll definitely take a look at getting an outside opinion on our takehome test

</comment></comment></comment></comment></comment></comment><comment><USER20> Pro tip: these tests are filtering mechanism on only one candidate funnel. Go figure out a way to get in through another funnel. Impress someone at a meetup, send a targetted email. If you can't distinguish yourself from the masses, you are the masses.

<comment><USER21> Personally, I wouldn't bother. Asinine requirements are usually the product of asinine leadership and toxic environment. Just find another company to apply to.

</comment></comment><comment><USER22> Totally agree. I am so busy creating and making real products, that I just don't have time for silly quizzes or challenges. If you like my work, hire me. If you don't, then don't waste my time. I was seeking employment in the Bay area briefly and did three code tests. Absolute waste of time that didn't reflect my skill, diligence or anything for that matter. After I realized that these techniques were the norm, I quit looking. I have way better things to do with my time.

</comment><comment><USER23> His complaints seem to be that this method of take-home-project is not as good as the method of hiring the candidate to work in the office temporarily for week's trial. That is true, but this is still superior to a typical interview. (Also not all candidates and not all employers can spare a week to do a trial.)

Nevertheless you should never agree to complete a full production project without being paid.

<comment><USER24> Is it? I "interviewed" at one YC funded startup in particular and they asked me to build a trivially small and simple js application (even though I applied for a DevOps position, but I guess devops is different for everyone...). The only communication up to this point was over e-mail and I sent a resume.

I built it. It worked. Then they said my application was not aesthetic enough and said thank you for your time.

The whole thing seemed backwards...

In my opinion (as someone who has been going through the process for a few years now throughout undergrad with internships and now looking for a full time job at the end of my masters), very few companies do the "coding" problem right.

I don't want to work for you if you ask me to write a nested for loop and regurgitate the overhead involved in searching a b-tree. I also don't want to work for you if you ask me to build a application with no pre-defined criteria.

In my opinion...the best coding question you can ask me is fizzbuzz types of questions or pseudo-code for something complex (to see if the candidate can think through the steps of solving the complex problem without having to waste the time solving it).

So far not a single company has done the below.

<comment><USER16> Pretty sure I interviewed at the same place, with the same reaction. I wasn't too impressed.

</comment></comment></comment><comment><USER25> If a company asked me that, I'd tell them thanks but no thanks and end the interview. Ask me a few questions, have me solve your riddles, see if I'd work well with your team/company. But building an entire production ready site? Ain't nobody got time for that.

<comment><USER26> Pretty sure no interviewer ever asked for a production ready site, this is just hyperbole from you and the author of this article.

If I were a front end developer, I would much prefer to be asked to write a quick, tailored to fit within the time allotment (e.g. one hour) web site rather than answering obscure coding puzzles that probably have little relevance with what I'll actually be doing at the company.

As the interviewer, I would phrase it as: "Build as much as you can in one hour, with the following functionalities in order of decreased priorities: login, cart, etc...".

Then I would judge you on how much of it you built, how solid each of the features are, etc. In my book, you can choose to go broad (build a lot of features, albeit superficially) or broad (only a few features, but well designed, tested, clean code).

<comment><USER27> > Pretty sure no interviewer ever asked for a production ready site this is > just hyperbole from you and the author of this article.

How can you be so sure? Were you there? You're essentially calling the author a liar without even pretending to have access to more data than that author. Thats just rude.

If you know something that the rest of us don't that would lead you to make such a forceful assertion, please share.

</comment></comment></comment><comment><USER28> I happen to really like coding challenges. I much prefer them to a "technical interview" situation, which I find very stressful--trying to answer difficult questions in real time.

However, the coding challenges I've done for applications have always been in the vein of "solve this interesting puzzle and show us how you did it", not "do some boring day-to-day work, but do it for free".

<comment><USER29> This exactly.

With the exception of my current employer, who asked me to make a small static site with some JS oddities (about 6 hours total), they've all been fairly simple challenges just to get a grip on my thinking process, not really have me do the routine stuff.

</comment></comment><comment><USER30> I wonder whether the same companies that insist that job candidates spend a day of their lives on a "code challenge" are the same ones complaining about good developers being impossible to find.

</comment><comment><USER31> On the other hand, as someone currently looking for my first dev job, I'd /much/ rather have a take home code challenge, where I can work at my usual pace, with my usual tools, without stress than have to do a code challenge while someone hovers over my shoulder. The take homes have been time intensive - I would say the average one has taken about 15 hours to do, but I feel they much better reflect my ability to code than real time nervous coding.

<comment><USER32> Your perspective changes when you need to find that extra 15 hours of time while also working full time. Apply to 2-3 companies at once, and all of a sudden 30-45 hours are added on to your normal work week.

</comment></comment><comment><USER33> Every time this comes up, I find it quite depressing how everyone becomes so self-centered. Half the comments here are along the lines of: I'm good at X, but bad at Y, so X is a better way to judge programmers than Y and if you do Y, your process is broken because I'm awesome despite being bad at Y.

Another small fraction of the comments are: I don't care if you can't do X despite being good or won't do Y because it's a big hassle - the bug is a feature and it filters out people who are not into our instasnapsocialmobileresponsiveapp. Get real guys, no one's that into you, some people are just desperate. Pretending that you're some prestigious company where everyone wants to work doesn't make it so - you have to earn it and if you get there, you won't have to pretend because you'd get so many resumes that you'll be forced come up with a quick way to be selective.

The secret to hiring is just like what pg said about the secret to VC investing - get better at distinguishing good programmers from bad ones so that you don't need to waste so much of everyone's time to make a decision. Being indecisive, after being clearly presented with enough information is a sign of incompetence.

</comment><comment><USER34> We used to ask job candidates to do something similar.

Since then, publishing code samples became the normal thing to do, so we simply require code samples instead, and give out the coding task only when they have nothing at all to share with us.

If you're in the job market, publishing some sample code, even if it's just a throwaway toy instead of, say, a full blown OSS project, can be a great tool for people to evaluate you as a candidate.

<comment><USER35> I would never go to an interview which wanted to present me with a code challenge, and I don't have publicly available code repositories for you to browse.

What I do have is years of experience building mission critical applications that move millions of dollars per day in transactions, and I have a solid track record of being easy to work with. I am good at what I do and I work hard to be an asset to any company that employs me, and I work hard to be enjoyable to work with.

If you can't be good enough to figure that out from a conversation and looking at my track record, why would I work for you?

BTW, I haven't had to look for a job in over 13 years - although I've worked for several companies during that time.

<comment><USER34> > What I do have is years of experience building mission critical applications > that move millions of dollars per day in transactions, and I have a solid > track record of being easy to work with.

> If you can't be good enough to figure that out from a conversation and > looking at my track record, why would I work for you?

Just because you can get something done, that doesn't mean you did it by writing well-thought-out, sanely structured, maintainable, readable, useful code. I mean, you probably aren't bad if you're bragging about yourself like this, but who really knows until the code is visible?

The language and environment we work in isn't known for high-quality coders. Quite the opposite. Filtering out the wheat from the chaff is basically a requirement, even for the people that seem good after a phone screen and an in-person interview.

<comment><USER35> "Just because you can get something done, that doesn't mean you did it by writing well-thought-out, sanely structured, maintainable, readable, useful code."

At no point did I claim to 'just get things done'. YOU said that. What I wrote was that I have years of experience building mission critical applications (which, of course, I would not claim if I could not prove).

It's not hard to prove or disprove if someone knows what they are talking about. I view 'coding challenges' as a sign that the company is not confident in their ability to assess people. It's a red flag to me. It's an amateur move. It tells me a LOT about the company from the inside, and it's not good.

'Coding challenges' are for companies who don't understand the skill set they are hiring for. I've hired plenty of devs over the years as well - we've had some bad ass programmers who worked for us, and I never needed a coding challenge to figure out who to hire.

</comment></comment></comment></comment><comment><USER36> I understand the point the author's trying to make.

For me, personally, this would be the better choice to judge my skills - <USER2> probably would lead to better chances for me to get the job. I suck at these braindead trick questions that are considered 'clever'. I'm not comfortable around a whiteboard only, being grilled by one or more other people while thinking.

I'm not slow, not really anyway. But I'm faster and better the more I know about the stuff at hand. I'll be good on the job, I'll suck solving math puzzles that some guy just read about in a book that ranks high on Amazon's list of code puzzle collections..

So - I'm glad that there's both. The author obviously would've preferred to show his skills live, in a discussion. This guy here would be happy to hand you a (small, mind you - I agree with the time-sink argument) finished project and discuss _that_ afterwards.

</comment><comment><USER37> I agree and disagree. I am a candidate and prefer coding challenges provided they are reasonable. Meaning, if you are wanting to check the person's ability in x, check the person ability in x, not y. So for example give a starter code with part that would showcase person talent in x is left out. Yes, this involves prior effort on the part of the company to come up with this exercise and on the person. I don't quite vouch for code samples, because it is very difficult to understand where they came up with the ideas and who wrote the code, and if it was a multi-person project. In the former case, it is a very controlled experiment and one you designed, so one can judge much better.

</comment><comment><USER11> The single most important point is that _the process is candidate unfriendly_. You can get away with being candidate unfriendly during college hiring. Maybe. If you're not hiring at a top CS school. You can't get away with it when you are trying to reel in a 10x superstar. Those types of candidates need to be wooed. If you make someone do too much work in the process, you'll only wind up with candidates who hate their jobs, not love their jobs.

That's not to say don't make sure people are smart. And the idea of proving coding skills is good too. But the manner has to be consistent with respecting the time and skills of an expert.

</comment><comment><USER38> The code challenge is usually just one part of the interview process to see if you can produce a quality product within a certain amount of time. It's not designed to tell every single thing you outlined in your post. It's meant to provide some signal before the actual on-site interview occurs. The on-site interview is expensive on the company's part, it is easier to filter out people who can't solve a simple full-stack solution. Companies who ask things like these are looking for people who understand both front and back ends. In short, just knowing either front or backend but not both, is not enough for the company.

<comment><USER39> We all know _why_ they are asking candidates to do this. The question is whether this is a reasonable way to go about getting this information.

<comment><USER40> My work speaks for itself and I am more than happy to share working codes samples.

I had someone offer me a paid test. I refused because I am too busy.

By requiring coding tests, employers and managers may be accidentally selecting only desperate candidates.

</comment></comment></comment><comment><USER41> To my mind the ideal challenge is no more than 4 hours. You make an appointment. At the agreed time, the hiring company emails you the requirements. If you have clarification question, they remain available throughout the challenge time to answer them. From a hiring point of view the challenge should have some vague areas in the requirements in order to test whether the candidate is a coder or a developer who asks decent clarification questions. The challenge requires no special tools to do, either JS in a browser or Python or something like that. The exact requirments of the dev environment are specified at the time the appointment is made, i.e. if you want the person to develop in Python 2.6.3 or to use JS with backbone.js, then the candidate can get set up in a few minutes and be ready.

I consider 4 hours to be a max. 45 minutes is a bit short IMHO and does not allow for including unit tests. Also, I believe that a good challenge should ask for a complete set of unit tests as well.

But a full working MVP is way overboard and is too complex for the hirer to make meaningful decisions based on the outcome.

Any challenge should be followed up by an interview in which questions focus on the solution as developed by the candidate. The why of choices and the intent are important factors, probably more important than the actual code.

<comment><USER42> This is essentially what we do.

Write a program to fetch some data from S3 using these credentials we made you, perform some analysis on them (algorithm question, generally with recursion) and then display the final output. And write a test for it.

We provide a stubbed-out Rails project on Github so they don't have to know all the details about S3.

With this we test three things:

1) Can they quickly find missing info on Google/Stack Overflow (S3 stubs)

2) Can they handle recursion? (Algorithm question)

3) Can they write a test?

Then we go over their submission on Skype. It's worked pretty well for us.

</comment></comment><comment><USER43> I love the idea of these challenges, but as a recent SF transplant\new job seeker, I'm absolutely appalled at how awful the execution usually is:

1\. Company A sends me an exercise with slapped together and near nonsensical instructions, no browser compatibility requirements, and a link to a data API that doesn't resolve. Keep in mind this is an established company offering big bucks. After getting them to send me a link to a working API, submit perfectly functioning implementation with squeaky-clean code (~2 hour exercise). Receive absolutely no fucking feedback whatsoever - I only know that I was explicitly rejected because the 3rd party recruiter eventually told me. Still would love to know what they took issue with (maybe they were offended when I asked for clarification on their washy challenge?).

2\. Company B sends me a ~5 hour Ruby coding exercise for a JavaScript position.

In both cases, a complete lack of respect for my time has been demonstrated. How naive of me to think that the _quality_ of companies out here would be better. I'm thinking of calling quits on getting a new and exciting job out here already. Don't get me started with the other pieces of interview "process" of encountered. Yeesh.

</comment><comment><USER44> For the most part, I'm happy to do most of the coding challenges I've been asked to do. I've been lucky that in most of the challenges I've been able to learn something about how to make a certain algorithm that I hadn't done before, of I've enjoyed the feedback about using method x vs. method y.

However, I've had two coding challenges which I did not like. In the first was just too time consuming (more than 3 hours) and I stopped working on it because I felt it was taking too much time, and the second was on-site without an internet connection where when I drew a blank on using setInterval, and couldn't remember document.getElementsByClass (I was trying getElementByClass) I was unable to look-up the answer, which basically meant I had to sit there wracking my brain trying to remember and completely failed a test which would have been a walk in the park otherwise.

At the same time, the two companies that I've chosen to work for have not asked for coding challenges. My current employer did look at my GitHub profile, I don't think my previous employer did.

I think that supplying sample code should be enough for most employers, hopefully there is a way to confirm the applicant wrote the code they are taking credit for (I've had somebody claim to build one of my sites before, so it's happened. I got a bunch of e-mails by people asking me for a reference).

Another good way for businesses to decide on a coding question (if they decide to do one at all) is to take questions from larger successful technology businesses. I had one company ask me to complete one of the questions from Spotify [https://www.spotify.com/au/jobs/](https://www.spotify.com/au/jobs/), it didn't take very long at all (may 40 minutes) and if it is good enough for Spotify, it is probably good enough for your company.

</comment><comment><USER45> If you use a coding challenge as part of your hiring process, please take the time to give feedback to candidates.

I recently completed a code challenge, and I was told by the recruiter that there were "problems with it," but he didn't get into any specifics because he was just passing on the message, and no one else has called either. I look at it as a dodged bullet, since it's a sign that they don't respect my time. I expect that had I gotten the job, they wouldn't have respected my free time or created a good work/life balance. Regardless, it has left a bad taste in my mouth. After I've spent about 12 hours of my free time (in addition to my 40 hr/week job), the least they could do is give me 15 minutes of their time to call or write out some comments about it.

</comment><comment><USER46> Many companies butcher their interview processes, so it's not surprising to hear this story. That said, OP's refusal of code challenges all together seems every bit as silly as being asked to jump through the hoops described.

If we were to take a moment to think about the purpose of interviews at a high level:

Initial Interviews (Phone/HR/etc)

\- Determine mutual interest \- Estimate likelihood of match \- Check for red flags

Follow-up Interviews (With team/Code Challenges/etc)

\- Confirm initial impressions \- Evaluate finer points of fit (skills, etc) \- Determine final interest

Each of the interview goals above apply to both the interviewer and interviewee. Failures happen when either side tries to monkey with this formula too much.

If your job is to write code and be an engineer, it's not outlandish to think that you'll be asked to write code as part of your evaluation for a new position. It's also not too much to expect that as an interviewee, you should be properly sold on the job by the interviewer before being asked to invest time to prove your skills. (Both sides should estimate high likelihood of match before confirming impressions)

If the company's preference is to hire full-stack engineers into front-end jobs, fine — more power to them. But those expectations should be included way up front in the process, so candidates like OP can mark a "red flag" and opt out of additional interviews.

TL;DR: Properly setting expectations in the interview process is ultra- important, but rarely practiced.

Note: I'm the founder of Mighty Spring ([https://www.mightyspring.com](https://www.mightyspring.com)), an anonymous, passive job discovery site. Part of our service is to help our clients (companies) hire more efficiently by coaching them up on interview process improvements like this. We're in private beta now, but send out new candidate invites regularly.

</comment><comment><USER47> Your td;dr: should go at the top. Having started working at a company that asked me to do one of these, it was different specifically in that it was paid. This means it's not a waste of anyone's time. A time-constrained, feature-rich project shows a number of things about a programmer including how they handle pressure, how they prioritize their tasks, whether their estimates are reasonable, and whether they know about what you're hiring them for.

</comment><comment><USER48> I get approached all the time by recruiters who supposedly love my work and want me to interview with their clients. It's getting harder to tell them that I'm not interested in a job. My consulting is doing okay and I'm actually starting to hire out some work. Why would I give that up, for a dice roll with a good chance of losing on whether or not my new boss turns out to be an asshole?

<comment><USER49> Good point. Getting to get back into consulting for that reason.

</comment></comment><comment><USER50> If you are going to do a code challenge, and take time out of your life to work on something without pay, the very least a company should do in return is provide feedback, good or bad. too many times I've done a code challenge and it's never even mentioned in follow up interviews. And if I don't pass the challenge, I should learn something from my time spent, right?

</comment><comment><USER49> Lately I have been solving coding challenges. For the sake of solving them only. I submit them, wait for "good job!" or "nice work!" and move on to the next challenge--at the next company. After a few months or years of doing this, I may decide to advance to technical phone screens on shared documents. I'll try that for a while. After that who knows...

<comment><USER51> I understand the desire for a challenge, but you might consider not submitting them. Effort goes into advancing a candidate's application at each stage, so someone out there is wasting their time so you can get your "good job!" boost.

<comment><USER49> It's a tough market. It can take years to hone your problem-solving skills. And years after that to hone your technical-phone-screen-on-shared-document skills. This takes deliberate practice. When I'm ready to advance to the next stage, I will.

There is a serious point: companies often aren't forthcoming about day-to-day working conditions. But the response you get to your puzzle solving activities can be telling.

</comment></comment></comment><comment><USER52> i don't think every candidate should be interviewed the same way, and i don't think these processes should be set in stone.

if you look great on paper (e.g. have some nice name brands on your resume, are a minor celebrity in some relevant community, or come in through a strong internal referral), there's no need for stuff like this.

however, if you don't fall into one of these categories, until there's a reliable way to tell if you're good from your resume, having an OPTIONAL coding challenge is a great way to get your foot in the door.

coding challenges shouldn't be about making people jump through arbitrary procedural hoops. they should be about making the (very broken) eng hiring process more meritocratic for people who'd never get their foot in the door otherwise.

</comment><comment><USER53> If you're being hired to code, expect to be asked to code in an interview process.

<comment><USER54> not many other fields of work where that analog exists, so I call BS. What's the personalized test given to CPAs? Sales? Marketers? You ask about past experience, accomplishments, etc..but rarely put them under the gun to produce on the spot.

<comment><USER21> This problem is typically solved through certification. As none exists in our world, we have to improvise.

</comment></comment></comment><comment><USER55> If someone asked me to write an iOS app for them for free as a test I'd offer them to download some of my work from the App Store. Otherwise, no.

</comment><comment><USER56> I don't care about code challenges either, I just always assumed that people don't care that I don't care.

</comment><comment><USER57> I have to say, that headline font is awful and the link highlighting on the blog's front page is worse. There is a bit of oddness in declaring yourself a front-end person by talking about not needing to be tested as a back-end person when you're applying for a front-end job yet having a blog with a such a jarring front-end.

<comment><USER58> I hope one day we can get to the point where we discuss the content of a post on HN, and not cast subjective opinion about its presentation.

I don't think there is anything wrong with the headline font. I'm not too fond of the link highlighting used on the front page, but it's fine. Really. And you can be a "front end person" and have very, very little to do with design these days. Entire webapps are made on the front-end.

<comment><USER57> Presentation is always important. It doesn't matter if it's a scientific paper or blog, presenting in a clear and intelligible way is critical to communication in all forms.

As for the content:

I've dealt with numerous developers who only want to do one thing, only backend, only front-end, only design. I've constantly run into shenanigans when working with such people in a fast paced dynamic environment. "I can't work on the frontend till the backend api is finished", "I can't work on the backend till the frontend data requirements are defined", "I can't work on the frontend till the UI/UX are defined". It's a never ending requirements dependency argument.

More capable full stack people are able to mock or fake the bits that are as of yet undefined allowing them to continue working on their core piece while gaining a better understand of the system as a whole. This allows them to contribute to the product definition at all levels instead of just living in their own little piece of it, waiting for definition from outside.

</comment></comment></comment><comment><USER59> I told FB the same thing, too!

</comment><comment><USER60> Though it really based on what lever you are at. This kinda tests are always for entry-mid level recruitment I guess. I agree with author 100 percent.</comment>


Python is only slow if you use it wrong - <AUTHOR> http://apenwarr.ca/log/?m=201110#15

<comment><USER2> ...and java is slow if you use it?</comment>


International Design in Government Day at the Code for America Summit 2019 - <AUTHOR> https://www.codeforamerica.org/events/summit/international-design-in-government-day <comment><AUTHOR> I'm co-chairing the Code for America Summit this year, and we're hosting an international design in government day at the Summit on 29th May in Oakland.

The International Design in Government Day is a one-day event: $150, in Oakland, and it's not just for designers working in or on government!

If you're a designer and interested in working with people -- tech _and_ non- tech, many of whom are government lifers -- passionate about improving how government works, then please come along!</comment>


Headphone Jacks Are the New Floppy Drives - <AUTHOR> http://daringfireball.net/2016/06/headphone_jacks_are_the_new_floppy_drives <comment><USER2> Perhaps it's just my reading, but this seems like Gruber is really struggling with the cognitive dissonance of "The headphone jack replacement is obviously not a great idea" and "I must shill and worship Apple at all costs".</comment>


New fees cast shadow on HEVC - TD-Linux http://www.cnet.com/news/new-fees-cast-shadow-on-next-gen-high-quality-streaming-video/

<comment><AUTHOR> Speaking of which - any recent updates on Daala? How is it coming along? Is it already better than HEVC? Close enough?

Also it should've been obvious that this would happen with HEVC. Typical bait and switch strategy. Make it so they depend on you, then raise prices. That's why everyone in the industry should've been more bullish on open source standards.

<comment><USER2> > everyone in the industry should've been more bullish on open source > standards

This, 100x this. Look at the pedigree of Xiph[1] - non-profit corporation dedicated to protecting the foundations of Internet multimedia from control by private interests.

1\. Ogg/Vorbis

2\. FLAC

3\. Opus/Speex - low latency voice codecs

4\. Theoria video codec

5\. Icecast

etc.

These people know how to get stuff done.

There is also Google's VP9. It has comparable quality to HVEC, and recently released libvpx 1.4.0 which adds support for 10-bit and 12-bit bit depth, 4:2:2 and 4:4:4 chroma subsampling, and VP9 multithreaded decoding/encoding

[https://en.wikipedia.org/wiki/VP9](https://en.wikipedia.org/wiki/VP9)

[1] [https://wiki.xiph.org/index.php/Main_Page](https://wiki.xiph.org/index.php/Main_Page)</comment></comment>


United States Transitions To A 'First-Inventor-To-File' Patent System - <AUTHOR> http://www.forbes.com/sites/johnvillasenor/2013/03/11/march-16-2013-america-transitions-to-a-first-inventor-to-file-patent-system/ <comment><USER2> I remember an article about SpaceX where Musk said that they didn't file many patents, because then other countries essentially had free access to their inventions if they didn't honour patents (China was mentioned).

With this change in law, they may no longer have protection against a patent filed by someone else that they (SpaceX) invented first...simply because they were not the first to file.

This may bring about more innovation because it forces "sharing" of ideas. Even if that "sharing" is done to stop real sharing.

<comment><USER3> SpaceX is in a very distinct position that its products are launched into space, making it very hard for a third-party to dissect and reproduce them.

<comment><USER4> Industrial espionage is still very powerful, and recovery of boosters is also possible.

<comment><USER3> SpaceX is probably protected against domestic copiers by trade secret law.

</comment></comment></comment></comment><comment><USER5> Even if this were a better approach, all I see it doing is forcing an even bigger torrent of patents into a system already so overloaded that it hardly functions as it is.

How is the patent office going to manage the increase? It's not, the system is going to slow down even further.

This also increases the cost of innovation. If I have a business, invent something, and don't patent it - I'm screwed regardless of if I prove I created it three years prior to the filing. In other words, now you have to file patents for everything to protect your ass (because it's not good enough to be able to prove you invented it first X years ago), and that's a very expensive proposition, particularly for small companies. This encourages the patenting of every little detail possible, even more so than today. Because if you don't patent every little detail, now you will have no right to self defense on the basis that you were the original inventor - they can come after you for every rounded corner you failed to patent, so to speak.

This forces everybody into the patent game.

The rationalizations being offered up, such as that this will be good because it forces openness on innovation are not only wrong (all it actually does it make it easier to steal technology for countries that already don't respect foreign patents), but the negatives are going to far outweigh any supposed benefits.

In five years everybody here is going to be complaining about how they really botched this piece of legislation and how it created more problems than the old system. This will turn out to be an extraordinarily expensive and destructive change; it is not the right approach.

<comment><USER6> There are a lot of concerns about first to file, but I think they're overblown because they ignore the context in which "first to file" versus "first to invent" comes into play: when there are two simultaneous patent applications.

What the AIA does is two-fold:

1) It changes the rule from "first to invent" to "first to file" when two inventors simultaneously seek a patent for the same invention;

2) It expands the "prior user rights" defense so that if you've been using a technology commercially for a year before the patent filing, you're not liable for infringement.

So if you weren't going to patent something, and keeping it as a trade secret instead, it's not clear that under AIA you have a new incentive to go patent it. If you weren't going to patent it anyway, there wasn't going to be a "dueling applications" situation, and "first to file" wasn't going to come into play anyway. And if you disclosed the technology instead of keeping it secret, then prior art comes into play and AIA expands the cases in which prior art can be used to invalidate a patent.

I'd personally like to see a merging of the "prior user rights" and "prior art" defenses. It should be possible to invalidate a patent by showing that _someone else_ was using the technology before the patent was filed, even if that other person didn't publish.

Of course, one way to achieve that effect under current rules is to publish instead of keeping technology secret. If the tech community is really opposed to patents, especially software patents, then it should take additional measures to publish exactly how all their technologies work. This is something Google, etc, can take the lead on without waiting for legislation. Establish a rich database of prior art that can be used to invalidate future patents.

</comment></comment><comment><USER7> I'd rather transition to a system where if two people try to file for the same thing within a few months or a year of each other, neither one gets it because it's probably pretty obvious.

<comment><USER8> What if both of them put real effort (money) into research and arrived at similar/same place? Should it be dismissed as well? Or is there something as joint patent?

<comment><USER9> I think in that case, the answer is "too bad, so sad". The patent system shouldn't _guarantee_ some kind of monetary reward. If it does, it sets up the wrong incentives.

Independent invention should be a sign that the idea isn't worth patenting, that it was "in the air" at the time.

<comment><USER10> > The patent system shouldn't guarantee some kind of monetary reward. If it > does, it sets up the wrong incentives.

They don't, unless people use it and the patent holder wants to charge for licensing.

<comment><USER9> The comment to which I responded seemed to take the view that spending a bunch of money on discovering or inventing something should more-or-less be automatically rewarded with at least a portion of the "intellectual property" rights. That's the concept I wanted to rebut.

I don't have facts at hand, but I imagine that like any other thing (starting a business, writing a book, creating an App) 99.9% of patents result in no monetary gain whatsoever. But if we're going to go all in on the property aspects of patents, then I think that morally and ethically, we have to give an independent inventor a share of the property. Anything else just looks arbitrary, and leads to disrespect of patents, "intellectual property" and The Law in general.

</comment></comment></comment></comment></comment><comment><USER11> I don't really understand who benefits from this change? Doesn't everyone lose?

<comment><USER12> I'm pretty sure the USPTO and Patent Attorneys win, given that it'll encourage more people to file patents, and sooner rather than later.

Not sure of the impact on innovation, but my gut instinct isn't good.

<comment><USER13> The number of applications consisting of just ideas, that haven't been implemented, tested, or tried, is going to sky rocket. And who can afford to file so many patents? The "big" established guys. Things just got worse.

</comment></comment></comment><comment><USER14> I still think that the occurrence a of situation where multiple parties file for patents on essentially the same thing in a short period of time independently should be considered unassailable evidence of obviousness and consequently unpatentable subject matter.

Whatever happened to a "person having ordinary skill in the art"?

</comment><comment><USER15> If an invention is invented twice independently, it should by definition be "obvious" and not patentable.

<comment><USER16> So if thousands of engineers independently worked for years trying to solve some important problem, and then finally a mere two of them come up with solutions around the same time that are similar, you'd say the solution is obvious?

That's a pretty weird way to define obvious. It is true that sometimes a term used in the law deviates a bit from the meaning of that term in ordinary English, but rarely as far as your definition of obvious does. In English, obvious means that it is easy to discover. It should be readily found by any reasonably competent person who looks.

</comment></comment><comment><USER17> 1\. My gripes with our patent, and trademark system comes down to fees.

2\. I draft all my own patents(only 1), and found the process difficult, but doable. I also filed my own trademark.

3\. I did have an issue with the filing fees. I contacted the offices involved and complained about the fees. I felt the fees might prevent many young inventors fron protecting their invention, or trademark.

4\. They told me their was some programs for low income individuals, but I couldn't kind anything. The fees for filing a patent should be based on what a person makes?

<comment><USER7> The fees are a deliberate barrier.

The patent office must get funding from somewhere. If the fees don't at least mostly cover their staffing and research needs then huge companies that file a lot become a huge financial burden on the taxpayer, and the taxpayer is then paying for a business advantage for these companies.

Obviously this sucks for the small inventor. I'm not sure how you get around it. The first X a year are free? But then the patents are always filed in individual names and then assigned to the company anyway, so I'm not sure that works.

<comment><USER18> While it's possible that too-low fees could hypothetically make the USPTO a drain on the federal balance sheet, it's currently a revenue _source_. [1]

And there's long been a separate (reduced) fee schedule for "small entities". And it's long been recognized those are still fairly high for situations like the lone-inventor, and such situations make up such a small slice of the USPTO's total revenue, that they certainly could be tweaked to be more accommodating without much net impact.

And because all that was well-known: The AIA included giving the USPTO some fee-setting authority. And one of their first proposals was to further reduce some small-entity fees and to create a new micro-entity status with commensurately-lower fees.

So it's not only a problem that would be fairly easy to address, without much adverse impact, it's actually already being addressed. (Though not really in- effect until later this year).

[1] Much ink has been spilt over the fact that the USPTO brings in more than it takes from the federal balance sheet, yet their budget has been left so low that they're chronically short on resources and thus backlogged.

</comment></comment></comment><comment><USER19> Patents are a necessary evil and a little bit addicting once you get your first taste. First to file simplifies a lot of the issues with prior art (ie the challenge of proving your idea was original), but I believe this system stifles small companies even further because of the costs associated with the patent system. We've made a decision to pursue key patents given how important some of these technologies are for our product, but I don't think that applies to most startups.

Here are some thoughts, cost overviews, and what to think about for a startup patent strategy: [http://kylethered.tumblr.com/post/45273614239/good-rules- of-...](http://kylethered.tumblr.com/post/45273614239/good-rules-of-thumb-for- a-software-startup-patent)

</comment><comment><USER20> I'm confused by their example. How does this change the rules of public disclosure?

Before this law, public disclosures _prevent_ the inventor from giving patents. In a conference I attended, the scientists were very careful to avoid publicly disclosing the details of their inventions for fear of removing their ability to file for a patent. But the brief blurb in the article mentions that if inventor A publically discloses their invention before inventor B files, then inventor A gets the patent.

Could someone clarify?

<comment><USER21> Publicly disclosing an invention does not prevent a patent...it simply puts a deadline for filing (1 year under the old rules, not sure under the new rules) starting with the date of disclosure. Thus, the purpose of avoiding disclosure is to prevent giving yourself a deadline.

</comment></comment><comment><USER22> Out of curiosity, what kind of protection does the system offer against companies in certain markets patenting against potential innovations in these markets by inventors?

<comment><USER23> In theory, (normal) patents should only be granted when they describe an invention to the level of detail that someone versed in the art can replicate the invention after expiration.

This property is unfortunately not generally present in patents once patent lawyers are done broadening the scope of possible 'embodiments' and rendering the language unintelligible.

</comment></comment><comment><USER24> Doesn't this put the pressure on companies to file even more patents on things quickly, in case someone else beats them to it?</comment>


OldAms.nl – photos of old Amsterdam 1850-1999 using Google Maps - <AUTHOR> http://oldams.nl/ <comment><AUTHOR> This project is inspired by OldSF.org, we used the same approach to create the website.

Also, we are thinking of scaling this idea to other cities now.</comment>


Some Thoughts On What The Obama Election Meant To Me - <AUTHOR> http://www.whatisleft.org/lookie_here/2009/02/over-the-last-few-months-i-have-felt-a-strong-urge-and-almost-responsibility-to-share-with-my-friends-why-the-obama-campai.html#more

<comment><USER2> Can anyone explain to me what relevance this post has to HN?

<comment><USER3> HN carries "Come to Jesus" links all the time.

Oh wait, it doesn't? In that case this link is off topic, and very creepy.</comment></comment>


Google Reportedly Dodged $2 Billion in Taxes With Bermuda Tax Haven - <AUTHOR> http://mashable.com/2012/12/10/google-2-billion-tax-dodge/

<comment><USER2> I'd rather give Google $2bln to spend than any government. More likely to have a positive impact on your life.</comment>