The Poisonous Employee-Ranking System That Helps Explain Microsoft's Decline - <AUTHOR> http://www.slate.com/blogs/future_tense/2013/08/23/stack_ranking_steve_ballmer_s_employee_evaluation_system_and_microsoft_s.html <comment><USER2> The only thing that matters about your coworkers––the only thing––is if, after the year's over or the project's done, you'd work with them again. If a team is doing good work and everyone on it enjoys working with each other, do not fuck with it.

Everyone in life has disagreements, petty jealousies, things they struggle with in other people. The beauty of the question, Would you work with her or him again?, is that it specifically doesn't try to disentangle those things––because to a productive team the quality its members' relationships with each other as important as the quality of their work. If one or the other is off, it's not necessarily anyone's fault, and it may not even be cause for termination. But it's as good a metric as you need.

But it's utterly foolish to winnow on a percentage basis. You'll break up good teams and prevent better ones from forming. You'll destroy morale. You'll encourage a leadership culture of cutthroat sociopaths whose success is at best tangentially related to the quality of their work. Do not do it.

<comment><USER3> This is the system Bill Gates' foundation is currently trying to peddle to the US government for school reform.

<comment><USER4> The crucial difference is that teachers don't work in groups i.e. you would rank them against all other teachers in a given school.

People say stack ranking is bad within Microsoft because you're not ranked against all 50k+ employees but against 10+ people that report to the same manager.

If 2 out of 10 people are guaranteed low ranking, then it's better to be a mediocre programmer in a team of poor programmers than to be a very good programmer in a team of even better programmers.

It also means that it makes sense to sabotage the performance of your 9 colleagues because you only care about looking better than them, not about doing best possible job for the company or your team.

A global stack ranking, if it was actually possible, would be great. You do want to get rid of low performers (and hire better replacements) and in a large enough pool the bottom 20% will really be worth replacing with better people.

<comment><USER5> I agree, I guess that I would do my best to get the easiest tasks so my performance would be up, and the hardest tasks would either never get done or in a really dirty way.

But how do you implement a global stack ranking in such a diverse company like Microsoft? There is no good criteria for comparing the kernel developers to the frontend programmers, for example. I doubt you could even find 100 employees working on the same kind of task.

<comment><USER6> I think michaelochurch has talked about "concave" vs "convex" work a lot.

<comment><USER7> Can you explain in greater detail please? Are you using the terms "concave" and "convex" in the same way that NNT does in his recent book Antifragile?

<comment><USER6> [http://news.ycombinator.com/item?id=5177851](http://news.ycombinator.com/item?id=5177851)

[http://michaelochurch.wordpress.com/2013/03/14/gervais- macle...](http://michaelochurch.wordpress.com/2013/03/14/gervais- macleod-9-convexity/)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> "Stack ranking" as it has been reported to be used at Microsoft, is multiple things conflated together. Some of these things are, by themselves, not necessarily bad. The problem is that people hear the words "stack ranking", and immediately jump to "eeeeevil" and "that's what killed Microsoft", and "Google does stack ranking, it must be doomed / I'd never want to work there", which is sloppy thinking.

Let's taking what people are calling "stack ranking" apart. The first aspect of stack ranking, is simply forcing people to make decisions about what things/object/people are more important/better/etc. than others. This can be a valuable technique. For example, there is a very useful testing instrument I took a while back that took a list of twenty values, e.g., "justice", "loyalty", "honesty", "security", "family", etc., and asked me to force rank sets of five of these values in order of which I thought were most important to least important (i.e., DNI Clapper would rank "honesty" below that of "loyalty" and "security"; others might make different choices --- there were no right answers). By taking twenty or thirty of these different subsets, ranked in the order of which I thought were most valuable, a computer program would allow me to see which values I considered most important, and which I considered least important. This was a valuable thing, and so the technique of doing forced ranking is and of itself a useful thing.

The second technique is that of trying to normalize employee performance review ratings across departments --- whether using a bell curve, or some other technique. Again, in isolation, this isn't necessarily a bad thing. Some managers might grade their employees more harshly, and some might be soft touches. Within a departmental level, it may be easier for a manager to argue that his team were all stars, but how do you deal with the question of an entire department using a harsher or more lenient grading system? One of the simplest things to do is to use a bell curve --- which may or may not be fair, especially if the company says (and all companies will claim this) that they hire the "best and brightest".

The final component is what do you actually do with this normalized ranking number? If you really believe that everyone you've hired are the "best and the brightest", then firing the lowest scoring 5% is insane. If however you don't have confidence in your hiring mechanisms, such that some duds make it past your hiring screens, then firing the lowest 5% may make a lot of sense; you want to get rid of them before they do (more) damage. (BTW, this is is why I believe a hiring system which tries extremely hard to avoid hiring duds, at the expense of sometimes failing to hire qualified individuals, is extremely important; it has its downsides, but it's very much a necessary evil, because the alternatives are far worse.)

What about using this number as an input to compensation? Here, using a normalized bell curve may make more sense. Certainly if you only have fixed pot of money to use for salary increases and equity refreshes, you want to make sure that your top performances get rewarded.

What about promotions? For junior engineers, if they are growing in their job, presumably they should get promoted as they demonstrate that they are getting better at programming and understanding the company's systems and development environment. Using the performance rating would certainly be one metric, but evaluating that person's proficiency and results should probably be just as important, if not more so.

The point is that just doing a "stack rank", whether as an input to the performance review mechanism (i.e., asking other employees to stack rank their peers, and then having some algorithm to merge those inputs together, and taking into account controversial rankings where multiple colleagues had differing opinions about whether person A was more or less effective than person B) or whether managers are forced to stack rank all of the employees in their department, is not necessarily a bad thing.

The problem comes from what do you _do_ with the stack ranking after you've generated it. It's that which can drive a company's culture, either positively or negatively. So my suggestion to Microsoft employees (other than to consider a career at Google :-), is to not rail against stack ranking per se, but to point out some of the negative effects of some of the things are done with the stack rank after it has been generated. Perhaps, if your HR people are at all competent, they might be more amenable to your suggestions. Or, maybe you'd be better off considering another company altogether, in which case my first suggestion still stands. :-)

<comment><USER9> " _If however you don 't have confidence in your hiring mechanisms, such that some duds make it past your hiring screens, then firing the lowest 5% may make a lot of sense._"

I'm not sure doing the above ever makes sense. We've heard of at least two severe problems with it at Microsoft, the worst 5% _at gaming the system_ get zapped, and a tremendous amount of effort is devoted to not being in that lowest 5%, or managing who gets that every 6 months. E.g. managers retaining "deadwood" so they've got people to sacrifice in the future, and ultimately, as dalek_cannes puts it ([https://news.ycombinator.com/item?id=6267518](https://news.ycombinator.com/item?id=6267518)), "Stack ranking causes employees to _compete with each other rather than the company 's competitors_. It's all downhill from there."

Anything you do in this direction, including not trying at all, will strongly affect your corporate culture and productivity; perhaps the best one can do is to strive to have "least worst" systems.

<comment><USER8> You can avoid this by making it a deep dark secret that you are firing the lowest 5% --- or that the reason why certain people are put on a PIP, subject to a "resource action", or "has found new opportunities outside of XXX" was due to the the fact that they fell into the lowest 5% bucket. I've seen this work at a past company at least reasonably well. A few people knew about the policy, but it wasn't something which was top of mind by all of the engineers, to the point where it started adversely affecting the company culture.

(BTW, I'm not saying it was a good thing; and I'm glad I'm no longer working there. But if you don't worry about silly things like ethics, and only worry about maximizing shareholder value, it's a strategy that might work for a while.)

</comment></comment></comment><comment><USER10> Well, one more blog post about the "ONE THING THAT KILLED MICROSOFT."

Nothing killed Microsoft. It just exhibited mean regression, as everything eventually does. It found itself on the very top of a mountain in an industry where you have to keep climbing to stay still.

The same thing happened to Apple -- it had an unprecendented string of hits with iPod/iPhone/iPad and it took competitors a few years to catch up.

Sometimes, a basketball player will make ten straight baskets. He'll seem invincible, then he'll make another and the crowd will go wild. But if he's a 60% shooter, at some point he'll miss a basket, and another, and another, and descend back into reality.

That's just how the world is.

<comment><USER11> I spent several years at MS, 3 as an FTE, and Microsoft is definitely in decline, partly due to stack ranking, partly due to general cultural shifts from being a big, lumbering company.

One of the things you have to realize is that a lot of the dynamics of the company over the last decade have caused a mind boggling amount of talent to simply evaporate out of the company. Many of the "fuck you I'm fully vested" folks who were around in the earlier days of MS either checked out or peaced out whenever they hit their personal bullshit thresholds. More recent hires have generally found that other companies are often more exciting, more rewarding, and more lucrative to work at. MS today tends to punish caring about things (not always, but more often than not). If you try to make a difference and put forth the effort to make merely sensible changes then you'll spend years struggling against the system and if you're lucky you'll only have moderate success to show for it.

And the stack ranking system tends to reward people who are good at gaming the system rather than those who are good at actually delivering anything of substantive value to end users.

Naturally, all of this tends to be very discouraging to engineers, especially those with a bit of maturity and experience who see that with their talent and skills they could go elsewhere and not get shit on all the time, work for a company they wouldn't be embarrassed about, and actually get stock options that are worth something. And, since techies tend to value working alongside other techies that they admire and look up to this sort of process tends to be a positive feedback loop.

It's simply unimaginable that this loss of talent hasn't had a huge impact on the company.

<comment><USER12> Microsoft are giving stock options instead of actual stock? That can't be true. Are they looking for stupid employees or what?

<comment><USER13> What is the problem with stock options?

<comment><USER14> Options are great for a company where there is a lot of growth (ie. a startup). Options are horrible for a company which has no growth (ie. a fortune 500).

Options give you the ability to buy a share at the last strike price set by the board (FMV for a share). If FMV for your startup share is $0.01 and you exercise it for $20, you're walking away with $19.99 in your pocket having spent $0.01. Multiply that by 50,000 and you're almost a millionaire.

If you get an option for a share of MSFT at $35, unless the stock price goes up, you're not getting a dime. _IF_ it went up by $20 a share, you'd be doing just as well. That's unlikely, however, given how infrequently the stock moves. If they gave you 5,000 shares instead, you'd be sitting on a (more or less) guaranteed $175,000 _even if the stock price didn't move_ instead of absolutely nothing.

<comment><USER15> The number of shares in an option grant is not necessarily the number of shares in a stock grant. You're assuming the choice is "5,000 options [for a single share]" or "5,000 shares." That's ridiculous. Clearly the shares are worth strictly more. But that's (typically) not the comparison.

</comment></comment></comment></comment></comment></comment><comment><USER16> This is exactly why I left Microsoft after 8 years. I worked really hard, had a great year and was expecting a great review. When I got my A-70 I told my manager I would be leaving. They offered me stock, a raise and even gave me a promotion, but I was done. I knew that I'd never be happy working under the Microsoft review system.

I saw many great developers over the years leave for the same reason. This system is poisonous. There is no way to tailor the rewards of the team to the actual performance of the team.

<comment><USER17> Sorry, what's an "A-70"?

<comment><USER18> A = "Achieved" \-- you did what you said you would do, and it's in line with your peers.

70 = You're an average Joe. You're not a bozo, and you're not a rock star, and you're not going to get a promotion soon.

<comment><USER19> _" 70 = You're an average Joe. You're not a bozo, and you're not a rock star, and you're not going to get a promotion soon"_

And then they offered you a promotion to stay when you threatened to leave ?

<comment><USER9> That tells you how bogus his review was. They didn't _really_ think he was an easily replaced average Joe cog, but they had to fill all their various slots and, I suppose in their view, he should have been happy not to have to been allocated to a worse one.

It's grading on an arbitrary curve, not on merit, " _There is no way to tailor the rewards of the team to the actual performance of the team._ "

</comment></comment></comment></comment></comment><comment><USER20> The ideal employee is both brilliant and easy to get along with. Practically no employees are ideal. Sometimes the least productive people are the peace- makers that let the rest of the team tolerate the brilliant assholes. Cogs are great, but it's idiotic to treat oil as though it were mere deadweight. Both are needed in any company.

<comment><USER21> that's fine as long as the oil is paid 10x less. oh, they want to be paid six figures to just sort of hang out a few hours a day, be everybody's friend, and leave by 3? The economics doesn't work because the productive people who are pulling 10-14 hour days will see that productivity is not necessary or valued and leave.

<comment><USER22> People pulling 14 hour days with regularity are not that productive.

<comment><USER23> There's a limit to the amount of serious, concentrated thinking that you can do in day, so it should be fairly obvious that pressurizing coders in to working long hours is counter-productive.

With other jobs it might be less of an issue. Although I've never done 14 hour days, I did once have a warehouse job assembling cardboard boxes for 12 hours a day, and I don't think the hours harmed my productivity. (My recollection is that I worked as hard as I could, just to make the day pass more quickly.)

<comment><USER21> In my opinion, this idea about no one can work more than 8 hours a day productively was created to discredit anyone who tries to get ahead through extra effort. It's similar to the idea that says we only need 30 minutes of exercise a day. In reality, pro athletes train for 4-6 hours a day.

Yes, a person is a fool if they do more than 8 hours in a workplace environment, but many people work longer days if it furthers their own personal goals. Pressuring coders who don't have an interest in working longer is pointless.

<comment><USER24> > It's similar to the idea that says we only need 30 minutes of exercise a > day. In reality, pro athletes train for 4-6 hours a day.

This isn't a contradiction; professional athletes do far more exercise than they _need_. That's why their fitness constantly _increases_ (far past what are the normal requirements for being considered "healthy") rather than just plateauing at "healthy."

</comment></comment></comment></comment></comment></comment><comment><USER25> This bit stuck out for me: "For that reason, executives said, a lot of Microsoft superstars did everything they could to avoid working alongside other top-notch developers, out of fear that they would be hurt in the rankings. And the reviews had real-world consequences: those at the top received bonuses and promotions; those at the bottom usually received no cash or were shown the door."

Individualized bonus/compensation decisions have no place in environments where the work is heavily or totally team-oriented. It forces people into a mode where they focus on getting the most individual credit instead of making the product as successful as possible. That general incentive exists in any team-setting, of course, but when you grade everyone on a curve like that and tie compensation/job retention to it, you really force everyone into playing that game.

By way of contrast, look at how investment banks do bonuses. If you're in IBD, your bonus is usually based on the success of your deal teams. If you're in S&T, your bonus is usually based on the success of your desk. Even further in that direction is how many law firms handle compensation: everyone is paid in strict lock-step. The end-result of these models is that nobody has an incentive, at least a purely monetary incentive, to jockey for credit. The way to make more money is to have the project be successful, and that means doing your best to cooperate with your teammates.

Sometimes you're forced to pay individualized bonuses. It might be the only way of attracting the superstar that would otherwise be able to make a lot more money elsewhere. But Microsoft has very high hiring standards and is rolling in cash. It should not have this problem.

<comment><USER26> That simply isn't true in the world of banking. Why do you think the financial crisis occurred? Bonuses are DISCRETIONARY and therefore even if trader A and B are in the same division, trader A may get a significantly higher bonus than B because he made more money or was simply more astute at the politics game.

</comment></comment><comment><USER27> Uhhh...decline? Gates stepped down in 2000, when Google was Goofy and the iPod was just an idea. Ballmer could've done worse for himself, to say the least. The very least. XP, Xbox, yes, Vista, but also the extremely impressive _full swing, timely as a motherfuck recovery_ from Vista to launch Win7. (And while we're here, Win8 gets way more trash than deserved. The Start Menu was overloaded. People had like 200 goddamn folders in there. Don't shit on MS for trying something new at a time when something new was needed.) --and I'm not even an MS fanboy _at all_. This is just bullshit writing.

<comment><USER28> I don't understand why hardcore techies wanted the start menu bad so badly.

I used Windows for a long time before migrating to OSX and the usage of a 3rd party app called Alfred. Now I just type what program I want to open, which is significantly faster and easier than having to navigate through an OS. I'm sure Windows probably has an equivalent.

<comment><USER29> The new system has two major issues. First, it forces a full screen context shift - everything disappears, which is just annoying. The icons are small, so a lot of results have a goofy looking scaled icon with only the first word or so if their title. On a server, it's even more ridiculous, and even MS's own TechNet admits as much (try to shut down).

Second, typing at the start screen is not the same as the start menu. It only searches some items, and you have to click or arrow down several times to select the kind of search you want. It's the opposite of unified search. It's annoying.

So, MS forced an annoying thing on everyone (even on servers!), with no way around, except to install 3rd party hacks. Of course they're going to get pushback on it.

</comment></comment></comment><comment><USER30> _I think everybody wants to work in a high-performance culture where we reward people who are doing fantastic work..._

This is typical superficial business talk about "culture." Dig what the humanists have to say about it: [http://www.fastcolabs.com/3016238/why-your- startups-culture-...](http://www.fastcolabs.com/3016238/why-your-startups- culture-is-secretly-awful)

</comment><comment><USER31> I only worked there for about 3.5 years starting in 2006, but from my perspective, dead wood was a bigger problem than the stack rank, and the stack rank was the main force keeping dead wood from being an even bigger problem.

Are there better mechanisms to systematically force managers to make the tough calls to let people go or counsel them out?

<comment><USER32> There is the Cravath Sytem's "Up or Out" policy:

[http://en.wikipedia.org/wiki/Up_or_out](http://en.wikipedia.org/wiki/Up_or_out)

      In a hierarchical organization, "up or out" is the
      requirement that each member of the organization must
      achieve a certain rank within a certain period of time.
      If they fail to do so, they must leave the organization.

<comment><USER33> That would be very difficult in a software development role. You'd either end up out of dev and in management or you'd have to leave.

Everywhere I've worked there's only a few levels where you actually code - right now we have graduate, dev, and senior dev. You'd end up moving your best people to a level where you can't use their skills any more.

<comment><USER31> Microsoft does have an up-or-out policy, but they take pains to try to avoid the problem you point out. Management is not considered "up" from dev, but a parallel track (whose bottom rung is a little higher than the bottom rung of dev). The top rung of the dev ladder is "Distinguished Engineer" and these people are spoken of in much more hushed and awed tones than any VP.

Also the up-or-out policy has exceptions: there are "stable" levels where you are allowed to stay for the rest of your career. IIRC the lowest stable level is Senior Dev (level 63) so if you are hired at level 59, the expectation is that you have the potential to make it to 63; if you earn promotions too slowly or not at all, you'll eventually be let go. This is true even if you do a perfectly adequate job at levels 59-62.

It's not a model that would work for every company, obviously...

<comment><USER34> The top dev ladder rung is Technical Fellow (promotion from distinguished engineer.) Except for Dave Cutler, who is a 'senior technical fellow' ... which is the exact same level with a different name.

</comment></comment></comment></comment></comment><comment><USER35> For the sake of balance let me provide some "defense" of the system. I don't actually like it, but there needs to be some perspective in the discussion.

A senior manager once confided to me that he does not have a shortage of good engineers to tackle even the most complex engineering problems, but he has acute shortage of people who are able to coordinate work of those good engineers. I have then concluded the review system makes sense in the light of this revelation. Hiring and Performance Review policies, however bad they seem, successfully maintain a pool of talented engineers; it's a solved problem. The problem that is not solved is making sure those engineers make coordinated effort towards a worthy goal. It is Microsoft's core belief that such coordination does not emerge naturally and so it must come from a leader or a group of "leaders", and the goal of the review system is to identify and promote such leaders, even if they have to sacrifice some of the good engineers in the process (finding more is a solved problem). The way the find more "leaders" is measure people on "scope of influence", with idea being that top engineer and manager talents will be able to influence more people compared to their peers. A sad byproduct of this approach is that a lot of people expand their visibility just for the sake of expanding their visibility. This produces a lot of sideffects, to the point where they sometimes become major output of the system rather than mere byproducts.

So if you wanted to know why the system works the way it does, above is a possible explanation that makes some sense.

</comment><comment><USER36> I can't imagine working in an environment like this. If the core idea is that all employees fall on a bell curve in their overall value, what about small sample sizes? In what dimension do you measure them--what is "performance" even? And to do this every six months?

Is there something I'm not seeing here? How is this still considered a viable management strategy?

<comment><USER37> Nobody ever got fired for aping Jack Welch...

<comment><USER9> To the extent this was responsible for his early departure from Microsoft, it didn't work out in the end. The article also says GE dropped this after Welch left.

</comment></comment></comment><comment><USER38> This sounds like one of those ideas that works well in a one off context but has side effects in a relatively big one.

Using a bell curve might be more technically correct, but the overall idea is too force the person making a judgement to make a judgment. The reality is that some employees are better than others. Getting a manager to rank them will give you better results than getting a manager to rate them. Neither are perfect. One is sensitive to the manager's biases, the other breaks when one team is better than another.

If you went to a one hundred person company with 10 managers and asked them to do this one day, you would probably end up with a fairly accurate view of who is worth what. Most one hundred person companies would be improved by replacing the bottom 5-20 people and damaged by replacing the top 5-20.

If this wasn't about employees, I don't think it would be controversial. Imagine a company with 100 clients ranked them and then decided to give special attention the 10 best while ignoring the 10 worst.

The problem with this (like most compensation schemes) is how they affect employees when they find out. People have strong reactions to being ranked and measured.

<comment><USER39> > Using a bell curve might be more technically correct

But it's not. That's what's so maddeningly stupid about it. Who gives a fuck if your employees are better or worse than one another? The only question that matters is whether they're doing the job you need done to the standard you need it done to. Arbitarily punishing some percentage of your team _even though they 're doing a great job_ is madness.

<comment><USER38> The bell curve suggests that in a group of 20 employees, #8 & #12 perform similarly. #1 probably performs a lot better than #4. Thats what I meant by more technically correct.

 _" the standard you need it done to"_ doesn't really fit the model of what MS do. _" great job"_ is relative. Anyway, the whole thing probably carries some assumptions. Some employees are great. Some stink. It's unlikely that a team of 20 will have a couple of stinkers. Employees #19 & #20 are adding little (or negative) benefit.

It's dehumanising. I agree. So are all management practices at companies big enough. It may also be wrong considering how it affects morale and how it affects productivity when people try to game it (seeking out weak teams).

I just meant that I can see where it comes from. It would make just as much sense if you asked a restaurant owner to rank his mains. Out of 15 mains, 2-3 are probably stars that people come specifically to eat. 10 are OK. 2-3 suck, they make people not come back.

<comment><USER40> > _" Out of 15 mains, 2-3 are probably stars that people come specifically to > eat. 10 are OK. 2-3 suck, they make people not come back."_

Then fire them. It's also important to note that the 2-3 who suck suck in relation to a consistent standard, not in relation to the other employees.

There is a fundamental disconnect between the justification of stack ranking and the actuality of stack ranking. In a sufficiently large population of employees you get some duds - that's a statistical certainty. The giant leap and non sequitor is that _relatively ranking_ will expose said duds better than measuring against a consistent, non-relative bar.

Where I work right now we've had the displeasure of having hired some duds. Very few mind you, but in all cases they were let go soon after it became apparent they were duds and could not be reasonably improved. All of this was done without the need for stack ranking, and (thankfully) outside the scope of some annual remove. If you've hired someone who's actively detracting from your company, why in the frell are you waiting a whole year to remove them?!

</comment></comment></comment></comment><comment><USER41> I just can't believe how or why they are continuing with the system when it is clear this is counter-productive and is hurting the company in many ways. What possible reason can there be? There is no way I would ever want to work in that kind of an environment.

Is there another side of the story? Can someone from Microsoft shed more light on this?

<comment><USER42> Before I agreed to work at Microsoft I asked the HR people about it. They felt this was a great system because a company with ~100K employees was bound to fit a bell curve regardless of its ability to hire and retain "the best". They told me the five buckets only had to meet their expected quotas at the level of Steve Ballmer, so it absolutely was not a zero-sum game as some had alleged.

I haven't seen a full year cycle applied to myself, but the one period I did participate in didn't quite work that way in practice. In reality, the zero- sum gets enforced much lower in the company than Ballmer himself. Exactly where probably depends a lot on what division and team it is.

Neither have I seen evidence of the claimed worst effects of this type of system. Instead I saw great people take a hit for the team, rather than retaining underperformers for this purpose. Who knows, maybe in some ways this is just as bad.

Every big organization needs some kind of system and there are certainly worse ones. This system is effective in allowing the company to promote some people, keep some where they are, and move some out.

But it would be nice if the current shuffling at the top became an opportunity to bring in something better.

<comment><USER43> > They felt this was a great system because a company with ~100K employees was > bound to fit a bell curve regardless of its ability to hire and retain "the > best".

This isn't true at all. If you have the ability to hire and retain the best, then the company curve will look like the rightmost extremity of a bell curve, which has rather different properties compared to the entire curve. As I understand it, this is why baseball statistics value players in terms of "wins above replacement" instead of "relative to the average player" \-- the average player is so much better than the majority of professional baseball players that it's not useful to assume in your tooling that you could hire one. (Whereas, if you have the entire bell curve, then the average person is both median and modal, and is therefore easier to find than a person at any other level.)

Disclaimers -- I don't know statistics, corrections very welcome.

</comment></comment></comment><comment><USER44> Note that at Microsoft, a good manager (i.e. somebody who takes care of his/her people) will spend far more on the stack rank process than on, say, product strategy.

<comment><USER18> Well, you have managers who are good at both. You want that kind.

But in the absence of any kind of ability to produce something, yeah, you want a manager who is politically savvy, and who can protect you, and can promote you well in the stack rankings. (I had a couple managers who were /terrible/ at this, and my "performance" suffered. In reality it was all presentation and horse-trading).

<comment><USER45> Great for the manager, and the employees, but what about the company?

What sane business do you want managers being better at "stacked ranking" than product strategy?

This to me, is indicative of why Microsoft has failed - when your daily job is to prove that the team next to you - not your team is the one that should take all the bad reviews, that's pitting employee vs. employee, when it should be employee vs. external competitors

</comment></comment></comment><comment><USER46> The most obvious question: Does Microsoft annually fire the X% lowest "performing" as some other shops do?

But more importantly, there are much bigger problems at Microsoft than gaming performance reviews: product strategy, how people are treated and relations between divisions.

MSFT may need to let go of some legacy in order to move forward faster. (Risk enterprise customer hate, but whatevs.)

<comment><USER47> Nope, it's very difficult to get fired from Microsoft unless you try really hard. Answering emails and doing minimum amount of coding (1 hour/day) is enough in most groups to get average reviews.

<comment><USER48> Knew an SDE II with 10 years of experience at MS who didn't know how to grep through a file to find out which log lines started with a certain word. Guy seemed have been holding pretty well onto his job, probably senior by now. Plenty of deadwood.

<comment><USER49> I guarantee said developer knows how to use "findstr" in the DOS shell to accomplish the same task though. Remember, this is Microsoft.

<comment><USER48> I used "grep" in a generic way. My comment referred to whatever grepping tool available on that platform, which at the time was indeed findstr, as you suggested, and Powershell.

Of course it'd have been silly to expect a Windows-only-ever person to be familiar with Unix tools, that was not the point at all.

</comment></comment></comment></comment></comment><comment><USER50> Interesting that Google also has stack ranking. If stack ranking were such a major factor of MS's decline why isn't it affecting Google the same way?

<comment><USER40> What cromwellian said, but I'd also add that, as an outsider at least, the zeitgeist amongst people I've spoken to is that stack ranking _is_ hurting Google.

The poisonous effects of stack ranking may take time to manifest in a notable way at Google. Maybe if Google's products stopped being as cool as they are now, or their compensation is no longer as generous as it is now (see: becoming Microsoft), this would be accelerated.

<comment><USER51> Hurting how? Google doesn't fire the bottom 10% or whatever. They also don't completely screw the bottom 50%.

The only curve that is enforced is really a calibration to ensure equal pay for equal performance across orgs

</comment></comment></comment><comment><USER52> This is not a problem inherent in stack ranking. _Any_ performance review system that (1) allow unbalanced pay and (2) makes this fact visible to employee, will cause exact same issues. Here unbalanced pay means you want to divide available reward such that higher performing employee gets lions share while those at the tail gets peanuts. Whether you do this through peer reviews or through manager reviews or stack ranking or team level performance or by throwing dice, you will always find some portion of employees feel the system is unjust and they deserve better ratings. Only way to avoid this is by equal bonus pays OR not assigning any ratings at all. For most technology companies at least the option of paying every one equal bonus won't be acceptable. It is known fact that highest performing employees perform 5X-20X than lowest performing employees and it would be hard to retain them if they were paid same bonuses as everyone else.

However not disclosing ratings could certainly be option. Or you can only disclose binary ratings "Stay/Fire" or trinary rating "Green/Yellow/Red" where they don't disclose amount/range of bonus but whether employee performance was sufficient for continuity of employment. People who thought that disclosing bonus indicating ratings was a good idea probably bought on an argument that this will motivate employees to improve themselves to get higher ratings. In reality many aggressive employees actually do work even harder to get back to better ratings however people who have feel lack of opportunity or are rusted eventually settle down with being labeled "average" and pull back on their effort level just to keep employment going.

<comment><USER53> The problem is that if you have an A team where all the members are performing well and deserve good compensation compared to the rest of the company, stack ranking forces you to penalise some of them anyway, even if its for minor petty reasons that in other teams would not be a factor.

Nobody is arguing that good people shouldn't be rewarded compared to poor performers. The problem is that stack ranking actively works against the formation of stable teams of high performers anywhere in your company.

</comment></comment><comment><USER54> The most frustrating thing about these kinds of systems is how they never seem to apply at the top. Was Balmer a top CEO? The market doesn't seem to think so.

<comment><USER55> Apparently the board didn't either. I guess they just get a much longer review cycle.

</comment></comment><comment><USER56> This isn't a Microsoft thing. If I remember correctly its used at IBM as well as a bunch of other big name companies.

[http://en.wikipedia.org/wiki/Vitality_curve](http://en.wikipedia.org/wiki/Vitality_curve)

<comment><USER57> From experience at IBM, it was just as terrible there.

<comment><USER56> I would never claim it was a good program, but this is actually much more common at your big companies then people would like to think. I know the Government does this in both the military and civilian workforce. Most government contracts I have worked with do this ( Lockheed Martin, SAIC, and Boeing )

I think until you have been in a big corporate environment you don't really appreciate how little they care about their workforce.

</comment></comment></comment><comment><USER14> I worked at a rather largish virtualisation company which uses the "Nine Box" performance review process. The idea is that you get two scores, your _public_ score which is told to you about your performance, and then a _private_ score which is surreptitiously given by your manager about your potential to get promotions.

Managers are then given a quota of how many people should be put in which box, and get hammered if they give too many people good reviews. The "secret" score is directly tied to your compensation, so basically your manager could be completely screwing you and you would have absolutely no indication.

<comment><USER58> > _and you would have absolutely no indication._

Except for your paycheck?

<comment><USER14> My suspicion would be they use plausible deniability. "We're not giving out great raises this year". Which is technically not a lie, I suppose. They're not giving out a great raise _to you_.

</comment></comment></comment><comment><USER59> I understand Enron tried the same thing. These people are idiots.

[http://postbiota.org/pipermail/tt/2007-June/000884.html](http://postbiota.org/pipermail/tt/2007-June/000884.html)

<comment> at-fates-hands They actually talked pretty extensively about this in the documentary "The Smartest Guys in the Room". I just remember watching with my jaw on the floor wondering why an organization would pit employees against each other like that.

The fact MS continued to adhere to it after Enron's collapse is stunning.

</comment></comment><comment><USER60> The end sentence is absolutely damning of the man:

"It will be interesting to see whether Microsoft's next CEO takes more personal responsibility for the company's corporate culture—or leaves it for Lisa Brummel to take up."

</comment><comment><USER61> Most companies use a variant of stack ranking. If you agree with the core philosophy that more productivity => better pay, then you need to implement some sort of a differential pay algorithm. Having said that the implementation really sucks. It is not natural and continuous, rather people get pigeonholed into discontinuous buckets.

Let me take a concrete example closely mirroring my experience. There are 4 fixed performance buckets. Top 5%, Next 20%, Next 65%, Bottom 10%. They get hikes of 20%, 8%, 4%, 0% respectively. Again, these are fixed numbers. Let us assume there are 4 people A,B,C,D and out of a hypothetical score of 100, score 95, 90, 87, 85 respectively based on various parameters. You would assume that since D differs in ability with A by 10%, he would get 90% of A's hike. But sorry, due to the stack implementation he gets 0%, while A gets 20% ! Let's say if the scores of A, B, C, D were instead 100, 50, 25, 5, the hikes would have make much more sense.

Summary: Discrete curve of benefits works well only when it closely matches the curve of people productivity. This is rare. So it just ends up being unfair and creates an unhealthy rat race.

<comment><USER62> I disagree with the whole premise of 'productivity => better pay'. I think it is much better and easier just to pay everyone in the same role the same (maybe with some company wide profit share) and then fire all the bad people as quickly as possible. You avoid all the politics of performance reviews, and the problems of how to rank the relative value of different skillsets. Incidentally, I believe this is the system that netflix uses?

Identifying the bad people usually pretty easy. The problem I find is that most people in software a pretty non-confrontational, so it is hard to get them to fire the bad people, even when they are obviously bad for productivity.

</comment></comment><comment><USER63> (Apologies for posting this as I know it doesn't add to the discussion but I have no idea how else to contact an MS engineer.)

Help!!!

On the off chance anyone at Microsoft reads this, my Microsoft account got deleted by a bug in Live Domains and I haven't been able to successfully contact anyone to report it or fix it. The forums have been useless and now I can't do anything MS related without my account including my phone!

My e-mail is in my profile.

<comment><USER64> Sorry, I don't know anything about the process of recovering your account info. But if you can describe the bug in enough detail that someone might be able to repro it I can forward _that_ to the right person. (Which certainly couldn't hurt your chances I figure.) Email me ['m', 'a', 'r', 'a', 'y', '@', 'm', 'i', 'c', 'r'...dotcom]

</comment></comment><comment><USER65> This reminds me of mandatory minimum sentencing that takes the jail time given away from the discretion of the judge. It seems that Microsoft doesn't trust it's managers to make objective reviews, especially to call out poor performers who are otherwise liked by their coworkers. There must be a middle way.

<comment><USER66> Exactly. And we have a tried and true way of identifying great managers. Can you deliver a great result, on time and on budget.

Why MS never focused on output is beyond me.

</comment></comment><comment><USER67> Does Google do this as well?

<comment><USER68> It does put people in buckets, but places no restrictions on the quantities that must reside in each bucket.

<comment><USER67> Sensible.

</comment></comment></comment><comment><USER69> this is just the dumbest article i have ever read. there are very few top performers, a types with an appetite for challenges, who would work in a meaningless project just to compete with mediocre talent, that's absurd! number one motivation issue is a bad manager, closely followed are lack of career development where in i mean interesting technical challenges andncompeting against the best and project interest. a types or top performers who take initiative and want to work on the biggest challenges do not shy from challenges, in fact, they want them! the types that shynfrom challenges and curl into meaningless projects are those that seek work life balance and are really your low performers.

<comment><USER70> You've missed the point. In a company where everybody performs equally, the ratings would still have to be distributed between the following rankings, regardless of actual performance. Amazing Good Average Poor

Judging by comments, it appears to be _the_ #1 choice for HR departments. Which is just plain depressing, but more depressingly, not surprising.

<comment><USER69> bell curve, nothing new

</comment></comment></comment><comment><USER71> Ranking and selection systems -- fitness functions -- are _very very hard_ to get right.

I did a lot of work with genetic algorithms and evolutionary computation / alife systems a while back, and thus did a lot of work with writing fitness or objective functions. It turns out that it's extremely difficult to write a fitness function that the evolving system will not "game."

To give a specific example: I once wrote an objective function to train an evolving system to classify images, a simple machine learning test. After running it for only an hour or so, the system's performance seemed _spectacular_ , like way up in the 90'th percentile. This made me suspicious. The programs that had evolved did not seem complex enough, and past experiments had shown that it should take a lot longer to get something that showed reasonable performance.

After a lot of analysis I figured out what it was.

I was pulling test images from two different databases. One database had _higher latency_ than the other. The bugs had evolved a timing loop to measure how long it took them to get their data (they were multi-threaded) and were basically executing a side-channel attack against the training supervisor.

In another very similar case, I found that the bugs were cooperating by communicating by way of the operating system's thread/task scheduler. They were using timing loops to kibitz.

Humans are smarter than little evolving computer programs. Subject them to any kind of fixed straightforward fitness function and they are going to game it, plain and simple.

It turns out that in writing machine learning objective functions, one must think very carefully about what the objective function is _actually rewarding_. If the objective function rewards more than one thing, the ML/EC/whatever system will find the _minimum effort_ or _minimum complexity_ solution and converge there.

In the human case under discussion here, apply this kind of reasoning and it becomes apparent that stack ranking as implemented in MS is rewarding _high relative performance vs. your peers in a group_ , not actual performance and not performance as tied in any way to the company's performance.

There's all kinds of ways to game that: keep inferior people around on purpose to make yourself look good, sabotage your peers, avoid working with good people, intentionally produce inferior work up front in order to skew the curve in later iterations, etc. All those are much easier (less effort, less complexity) than actual performance. A lot of these things are also rather sociopathic in nature. It seems like most ranking systems in the real world end up selecting for sociopathy.

This is the central problem with the whole concept of meritocracy, and also with related ideas like eugenics. It turns out that defining merit and achieving it are of roughly equivalent difficulty. They might actually be the same problem.

</comment><comment><USER72> I used to work with Andersen Consulting/Accenture. We had this ranking thing as well. Had lots of peers who were of the kiss ass type. Fortunately was very technical and stood out while still at the programmer/analyst level. Would have fared poorly at the managerial level though.

Many other companies probably have this too.

</comment><comment><USER70> I was a programmer a UK banks for 5 years where they used (and still do use) the bell curve performance rating system. High performing teams match the bell curve distribution, therefore match the employees to the curve and you have high performing teams. It, and a load of shit directly related to it, is why I left.

</comment><comment><USER73> Adobe used to have the Stack Ranking System until a couple of years, which they eventually ditched: [http://www.hreonline.com/HRE/view/story.jhtml?id=534355695](http://www.hreonline.com/HRE/view/story.jhtml?id=534355695)

</comment><comment><USER74> What I love about the last decade of the Microsoft saga is how Bill left Stevie holding the bag, just when he knew it was going sour.

</comment><comment><USER71> Hasn't Google implemented stack ranking?

</comment><comment><USER75> Haha, Yahoo does it every 3 months since Mayer came on board! Soon, even Yahoo will crumble like this.

</comment><comment><USER76> It would be a lot more convincing if they could say Microsoft Stack Ranks And No One Else Does. Google, for example is a strictly stack ranked company and hasn't folded (at least in my opinion) into a caustic environment as described in this article.

<comment><USER77> Google uses stack-ranking in a very different way. At Google the stack ranks of all your _peers_ are merged and then used as a sanity-check on the promotion process, to catch cases where a manager may be artificially inflating the reviews of his favorite reports or where he may be overlooking a star employee who everyone else in the organization agrees is worthy of promotion.

<comment><USER78> So Google has a different algorithm for computing the stank rank, but still stack ranks? Microsoft also incorporates peer feedback into the review process, but treats it as just another input into the manager's decision process.

</comment></comment></comment><comment><USER6> Personally on stack ranking at MS I'd probably settle for ending some of the worst practices. Even Google's stack ranking don't have some of the problems of the one used by MS.

</comment><comment><USER79> Can somone please explain to me the attraction of working at a company with such a ranking system? Are [tech] jobs _that_ hard to come by nowadays that one is willing to accept anything to pay the bills?</comment>