California Can Expect Blackouts for a Decade, Says PG&E CEO - <AUTHOR> https://www.npr.org/2019/10/18/771486828/california-can-expect-blackouts-for-a-decade-says-pg-e-ceo <comment><USER2> As a South African, the initial reaction is a knee-jerk, snide comment "get some", since we have frequent power outages due to a combination of institutional incompetence and malicious actors. (Apparently, at one point, we had outages that were "staged" in order for the Gupta family to make money from coal sales; I am not sure how it worked.)

But as a more sober and conscientious observer, what are the reasons for this? Is it that fires have been causing too much damage? Is it that the utility companies are slacking? From the article it is not clear whether putting off the power is a successful mitigation strategy. It's a bit unimaginable for California to have the same problems that we have. In our case we have politics that inadequately qualified people are promoted to management and hence my question about whether there is logic to this.

But the main difference is we don't have the same fire issues that California has, or at least, situations where the power lines are the cause of fires.

Edit: We _do_ have fires, but I think we have other (less straightforward/tangible) issues that we debate in our spare time.

<comment><USER3> PG&E has been paying out dividends and executive bonuses instead of reinvesting in their infrastructure for years. And bought off a number of state politicians.

Now that climate change is creating conditions for the already wildfire-prone landscape of California to suffer more frequent and severe events, the power companies are washing their hands of it. Short of some major state or federal investment in upgrading energy infrastructure (like the green new deal), I wouldn't expect that to change.

<comment><USER4> > PG&E has been paying out dividends and executive bonuses instead of > reinvesting in their infrastructure for years. And bought off a number of > state politicians.

Re the dividends part, that's an issue on which our present society will have to work more and more, because afaik most of those dividends go into the coffers of pension funds/insurance companies/big institutional investors which by their very nature "trickle down" that money mostly to people aged over 45 or 50. Most of those reinvestments you mention will get their financial return in 20-30 years' time, by which point the age-group I mentioned earlier (the over 45-50) will either be dead or to old too care that much about it.

<comment><USER5> Shouldn't PGE forgo payouts to shareholders in order to keep infrastructure to a better standard? Surely the long term value of PGE is diminished (and thus their ability to pay dividends) if they are found liable for fires or effect rolling blackouts as a "solution". The pension funds could invest in something else instead.

<comment><USER6> They suspended dividends two years ago [https://www.sfchronicle.com/business/article/PG-E- suspends-d...](https://www.sfchronicle.com/business/article/PG-E-suspends- dividends-over-possible-wildfire-12445592.php)

</comment></comment></comment></comment></comment><comment><USER7> I am not from US but this smells as if PG&E tried to force California and general population to demand PG&E to be deemed not responsible for fires.

<comment><USER8> That's definitely a dimension of what's happening, none of the utilities have ever been cool with the nebulous liability presented by inverse condemnation. I think if anyone were in their shoes, they wouldn't be either.

PG&E knows that they're way less liable for damage caused by service interruptions than they see they are for being responsible for wildfires. They're doing what California wants them to even if it's not the intent of the law.

That's mainly why I think seizing PG&E doesn't solve your problems in the short/medium term. If the state owned their lines tomorrow, they haven't seen the last wildfire. I just don't see a scenario in which the state is as eager to pick as high a number when determining liabilities.

<comment><USER9> Unless I'm missing something this completely ignores the decades of mismanagement and paying huge bonuses/salaries instead of reinvesting in infrastructure. PG&E put themselves in this position should be held accountable.

<comment> james-mcelwain Right, I'd be sympathetic to this liability argument if they weren't extracting profits and neglecting maintenance. Obviously the fire issue is bigger than a single utility, but their actions here leave them little public good will.

<comment><USER10> I would demonstrate my sympathy for their desire not to be a public utility anymore by confiscating their assets, firing their management, and turning their assets over to someone who does want to be a public utility.

<comment><USER7> What you describe is called communism.

PG&E is a company with owners and while you may not like it you can't just take other peoples' property.

There are appropriate venues to vent your frustration. There is judicial system where you can go and defend your own rights if you feel PG&E stepped on it. You can go and vote for somebody to step up and solve the problem. You can also make other people aware of the problem by using your right to free speech.

<comment><USER10> No. A public utility is permitted what would be anti-trust violations, and handed a monopoly, in exchange for acting in the public interest. When they abandon the public interest, they have violated their end of the bargain. The stockholders are responsible to prevent that; if they fail, they lose their investment.

The educational system has failed you if you never learned about this. Anticipating your next objection: no, public education is also not Communism.

Communism, by numerous well-attested examples, means exterminating millions of citizens for no offense but existing. I don't find anyone proposing that. Here.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> The amount of home generators is going to quickly increase, which will do wonders for air quality when the power lines are cut.

<comment><USER12> Purchased by who, though? The cost of living is already so high that only the wealthy will be buying them.

<comment><USER13> You can buy a decent 'emergency' generator (aka high air pollution and noise) for ~ $500 sufficient to run most homes. Compared to cost of living in California, that is dirt cheap (a week or less of living expenses).

One problem with these emergency portable generators of course, is they lack safety components to stop them from catching fire. So we might end up with notable fires caused by generators used because PG&E shut off power to 'stop fires'. It shifts the liability to the consumer though, so PG&E is off the hook.

Few people really believed PG&E was going to do this, so a lot of people haven't purchased equipment yet. We'll see how it shapes up long term.

<comment><USER14> As people turn to home power generation, PG&E will lose revenue. So, PG&E will suffer in the longer term as they lose customers. Not very good business planning.

<comment><USER11> Keeping a gas or diesel generator fueled and running for days at a time is not super cheap. Plus if your whole city's power is cut and everybody else is running their generators, there's a risk of fuel price increases or even fuel shortages.

Solar & batteries are a solution but people won't adopt as a solution with this sort of timeline and current solar/battery prices. There'd need to be a slow but certain rise in PG&E prices and a some sort of tax rebate to really get people to convert over.

<comment><USER14> Reliability is what sells me on electric supply. Take that away, I will find something more reliable, and I will make the investment in local, personal generation if there isn't a source.

<comment><USER13> Local, personal generation is usually incredibly inefficient due to loading factors (you need to buy a generator that can handle most of your peak load, but they are only efficient at roughly half that - so your idle load is a huge waste, and when near peak you will be horribly inefficient).

Maintenance is also a huge pain - I've got an off grid cabin, and I can assure you, you will quickly tire of dealing with it if you are using it as your day to day power source.

Backup (propane or natural gas) generators that auto kick in aren't a bad option, but are much more expensive (about an order of magnitude higher).

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> I asked this in another thread and got a healthy dose of downvotes instead of an answer. But why can't they do a controlled burn along the path of the power lines?

If it's too risky they could even do it at a different time of year when there's less risk.

<comment><USER16> PG&E maintains over 18,000 miles of transmission lines.

<comment><USER15> So the question is how many of those go through fire prone areas. Perhaps there's a 90/10 rule for where to expend the most effort.

</comment></comment></comment><comment><USER17> If this is the best that the private sector can do, it seems like it is probably time to let the public sector take over.

<comment><USER18> Why would the public sector do better?

<comment><USER19> Because they exempt themselves from liability.

</comment></comment></comment><comment><USER20> I am from the east coast, and compared to where I grew up, the power grid is way more reliable. Where I grew up, the utilities were not undergrounded, and many neighborhoods were heavily forested. Whenever there was a big front of thunderstorms in summer or, worse, an ice storm in the winter, large swaths of the area would lose all of their utilities besides natural gas. Sometimes these outages would last several days.

It was inconvenient, but nobody wanted to pay for the cost of undergrounding utilities, so it was something you just dealt with. A few people bought generators, but most didn't. The only time the outages really caused hardship was during ice storms, but fortunately those were pretty rare.

I now live in CA. PG&E made the right call here. A couple days of no electricity is better than burning an entire town to the ground in a firestorm.

</comment><comment><USER21> Really, California is considered the tech hub of the world, how is this ok ?

<comment><USER22> Well, like hundred years of propaganda starting from the Edward Bernays times convinced generations to accept this broken system of privately owned monopolies, which has this inherent perverse incentive to rent seek with no limit and neglect actual products or services they are supposed to provide.

</comment></comment><comment><USER23> We can't live in the forest. Building in the tinderbox is CA's equivalent of TX's building in the flood plain.

<comment><USER24> You can, there are some fairly straightforward principles of building in these environments to prevent the spread of fires, the technique works. Now, getting folks to care about fixing their homes, when they are insured for such a small proportion of the costs they generate, is a taller order.

</comment></comment><comment><USER25> I don't understand how the powerlines can affect to wildfires. They can put powerlines in any place or how it works? In my country (Spain) there are laws to say exactly how much meters of free space are needed depending on what capacity has the powerline. That means that you can't build anything there and that if a powerline goes in your forestal terrain you can't plant any tree there (thats why people don't like powerlines over theirs lands). That also does that all the forestal ground have a good number of firewalls because of the powerlines.

</comment><comment><USER26> It's really sad how tribalist politics have pervasively overtaken rational discussions. I dream of discourse that doesn't devolve into conversations about leftists, conservatives, and various interest groups.

</comment><comment><USER27> I was in CA last year with the major wildfires going on. Not there this time, so am curious if this approach worked this year.

If yes, I suppose that controlled, scheduled blackouts is a positive move (not a competent one given that this is CA we are talking about) as it avoids a huge loss of lives, property, forest and avoids a ton of air pollution. (The generators are gonna cause it anyways. :( )

On the flip side, is PG&E using this time to improve infrastructure or should expect it to extend the timeline by another decade in 7-8 years from now?

</comment><comment><USER28> this is what happens when you sue a company for something that was an acceptable risk. it now becomes unacceptable and this is the consequence.

<comment><USER29> I think you're right on target with that observation. A compromise of limited liability and aggressive infrastructure upgrades may be the way to go.

</comment></comment><comment><USER30> Context from the official statement is that PG&E does expect to require blackouts, but it will not be the same magnitude of black outs as seen a few weeks ago.

<comment><USER12> Where did you infer that? Literally the first sentence is...

>The CEO of Pacific Gas & Electric Corp. told California energy regulators that the state will likely see blackouts for another 10 years like the one imposed last week that left as many as 800,000 customers without power.

Not only that, but it's impossible to claim that future blackouts won't be of the same magnitude. The blackouts are determined by weather patterns, none of which can be predicted this far out. It's entirely possible we get another weather forecast in a year that calls for the same region-wide high winds and dryness.

</comment></comment><comment><USER31> if the ceo can't figure it out he needs to be replaced.

</comment><comment><USER32> I've lived in two developing countries in my life, and its interesting how CA is morphing back into that. Small enclaves of super-rich, very high price communities where everything works, surrounded by large swaths of shanty towns where nothing works.

<comment><USER3> After living in LA for a few years, it certainly does feel exactly the way you describe within the city.

But those juxtapositions are literal blocks apart from each other. The Google campus is next door to a shanty town in Venice, and their security sic the police on them for any minor infractions (or just made up ones). Very similar situation with the redevelopment downtown.

It really does give you a bleak portrait of the tolerance for inhumanity within the context liberal Capitalism. California is ostensibly as "wacky leftist" as the US gets and it's a bleak, cruel place in many regards.

<comment><USER33> American liberals are always yelling for more "humanity", but they keep quiet about the requirement that humanity must come at someone else's expense. Yes, we need cheaper housing - but not here. Yes, guvment must build better jails - but not here. Yes, please run more buses - but not down my street. Yes, we must accept more refugees - but put them somewhere far away, certainly not anywhere near my child's route to school.

Liberalism and NIMBYism are joined at the hip. Liberalism is loud and ostentatious, while NIMBYism is prosecuted quietly via the court system, lobbying, and political donations. There's a certain poetic beauty to how both seem to coexist in the same group of people.

<comment><USER34> I'm not sure where you get those ideas? Where I live in NYC, they recently shut down a major street to cars so buses could run more easily — and the furthest left want to expand that. The left leaning people I know want to _close_ jails, not build better ones elsewhere. We opposed an Amazon campus in our backyard specifically because it would drive up rents — and the furthest left want rent control.

Liberals are obviously not a monolith of only ethically consistent people, but I definitely don't see the pervasive NIMBYism.

<comment><USER33> I live on the Upper West Side of Manhattan. Here the same people who yell for affordable housing also go to court to block construction of new towers. The same people who complain about buses stuck in traffic on W 86th St also prevent the introduction of universal metered parking and congestion pricing.

But my original comment was mainly addressing the NIMBYism in California. Look at SF: it's a low-rise city of sky-high demand and prices. Who's preventing new construction? The same local NIMBYs who enjoy rising property values due to severely restricted supply of land that can be built on.

<comment><USER34> I until very recently lived downtown, where congestion pricing would actually take effect, and the people most in support were again the furthest left.

SF does seem to be more NIMBYish, but I'm not sure why you think it's a microcosm of liberalism.

</comment></comment></comment></comment></comment></comment><comment><USER35> "But Johnson said it could take 10 years before such outages are "really ratcheted down significantly." "

Strikes me as a CEO that suddenly wants very much to have his golden parachute and be out of this pickle.</comment>


Our next app (spend Bitcoin at Target) - <AUTHOR> http://blog.cardforcoin.com/post/93334368715/weve-been-really-excited-by-the-response-weve

<comment><AUTHOR> I'd love feedback on the design of the landing page, the brand, or any marketing ideas you might have. We want to appeal to long-time bitcoiners and newbies alike.</comment>


ChallengeHN: Popular Hanging Picture Puzzle - <AUTHOR> http://pratikpoddarcse.blogspot.in/2012/12/hanging-picture-puzzle.html <comment><USER2> Rest the portrait on top of the nails.</comment>


Lighting a rocket is easy; tough part is controlling it - <AUTHOR> http://www.startribune.com/lifestyle/147338135.html <comment><AUTHOR> posted for this comment:

      "In many ways, the worst enemy of NASA is `Star Trek',"
      Pace said. "Captain Picard says `engage' and the ship
      moves. And people think `How hard can this be?'"</comment>


Show HN: 2 Cars (Ketchapp) game web version using HTML5 Canvas - <AUTHOR> https://hkandala.github.io/2-Cars/ <comment><AUTHOR> Just felt like sharing the game I developed 4 years back, as a part of my assignment in the web development course I took at my college.

P.S: Mute yourself to avoid the horrible game sounds :P</comment>


Character literal is not a character in C  - <AUTHOR> http://arjunsreedharan.org/post/78628168199/character-literal-is-not-a-character-in-c

<comment><USER2> This is one of the few examples of code that is valid in both C and C++ but that produces different results (in C++ the size would be 1).

</comment><comment><USER3> No mystery. C's char type is a single byte, an integer type of (usually) 8 bits, though the actual size is platform-dependent. A char may be signed or unsigned, like the other integer types (short, int, long).

The value of a char is the ASCII code for the character, so 'a' is stored as the integer 97.

This is documented in "The C Programming Language" on pages 35-36.

C does not have a string type. By convention a string is an array of char with a null (0) byte at the end.

C++ follows the same conventions for char and arrays of char, but implements "real" strings with more string-like behavior.

</comment><comment><USER4> In fact, you can have multi-character character constants, e.g.

        printf("sizeof('abcd') %d\n", sizeof('abcd'));
        printf("'abcd' = %d\n", 'abcd');

produces

        sizeof('abcd') 4
        'abcd' = 1633837924

</comment><comment><USER5> All integer constants "start" as type int. The message isn't about what size your value is but the fact that you are comparing a pointer with an integer. Cast to a char and you still get the same message</comment>


Thoughts on monetizing a privacy focused search engine - <AUTHOR> https://coil.com/p/runnaroo/Privacy-and-Search-Engine-Monetization/5dXAW6NwF <comment><AUTHOR> I'm the author of the linked article. My biggest surprise over the last few months has been that it is a bigger challenge to try to figure out a better was to monetize a new search engine than to actually getting people to switch to it.</comment>


VIDEO: Come Onboard for the Last Power-Up of a Space Shuttle - <AUTHOR> http://www.avgeekery.com/video-come-onboard-for-the-last-power-up-of-a-space-shuttle/ <comment><AUTHOR> There were two major standout points to me in the article. The first was that the main engines (SSMEs) were replaced so that the originals could be used for the next generation system.

The other thing that's piqued my curiosity is that once drained of fluids, the shuttles could never be powered on again. Not just "can't be certified for flight" or even "engines can't be safely fired", but a complete inability to even power on.</comment>


Holochain: A Blockchain Alternative – Interview with Matthew Schutte - <AUTHOR> https://www.youtube.com/watch?v=rGpN0BNpTqc <comment><USER2> Soon as I get home I'm going to create vapourchain</comment>


Why do some developers consider Agile development to be nonsense? - <AUTHOR> http://www.agileoverflow.com/t/why-do-some-developers-at-strong-companies-like-google-consider-agile-development-to-be-nonsense/107 <comment><USER3> This matches my experience.

We recently brought on a couple of PM consultants who are deep into Agile as a methodology, and decided to give it a go -- it's been about six months now and productivity has basically ground to a halt as we spend more and more meeting time endlessly sorting tasks into different little arbitrary piles instead of actually making forward progress. It's made communication with the non- software part of the company much more difficult, because they keep getting hung up on the jargon; we have to slice up tasks arbitrarily to get them to fit into "sprints," which go out of date minutes after the morning standup when the first non-planned bug report lands. Sizing is frequently an exercise in post-facto book-cooking: 'How long will that bug take to fix?' is like asking 'How long is it going to take to answer 23 across in next Sunday's crossword puzzle?' It might be fifteen seconds, it might take all day; until I have a chance to look at the puzzle I'm just pulling numbers out of my ass. The short timeframes encourage everyone to twiddle around the edges of things instead of doing real feature development; the arbitrary milestones serve mostly to make everyone feel like they're constantly falling behind.

I've discussed this with a bunch of people. Everyone seems to agree that we're doing Agile wrong, but even the strongest proponents of the methodology seem to be in complete disagreement on what the "right" way to do it is, or how that would be an improvement over the way we were doing things before.

So yeah. Not a fan.

<comment><USER4> "'How long will that bug take to fix?' is like asking 'How long is it going to take to answer 23 across in next Sunday's crossword puzzle?' It might be fifteen seconds, it might take all day; until I have a chance to look at the puzzle I'm just pulling numbers out of my ass." .

I find this attitude to be a little disingenuous. Obviously every bug cannot be estimated perfectly, but the majority of the time there is some initial data to make a reasonable estimate with a margin of error.

For instance to name a few:

\- Past experience with the module associated.

\- Whether or not it has been reproduced locally.

\- Similarity to other bugs that have come up in the past.

<comment><USER5> Depends on the bug, sure.

It's my job [or used to be :-)] to give estimates of bug fix times to management.

It usually broke down to:

\- trivial (about an hour)

\- nearly trivial (maybe half a day or a day)

\- "I have no earthly idea". The last one of these that I fixed involved about twenty people, deep dives into our hypervisor and interprocessor communication mechanisms, research into some hardware details and three 100+ hour weeks about four weeks before we shipped. And we started out not having _any_ idea where in the system the problem was (that took two of those weeks).

</comment></comment></comment><comment><USER6> This article cements in my mind the issue: devs tend to take away from agile what they expect to find, rather than taking the time to understand the motivations and whole scope of activities.

Historically, most software development methodologies are excessively top- down, so people somehow still expect that from agile. Most devs are the sort that couldn't stand doing group projects in school, so the idea of organizing with teammates to produce something bigger and better than I could on my own is totally foreign.

If you don't explicitly name your development methodology, you still have one: you're practicing something called ad-hoc development. Unless you and your entire team are superheroes, there's an awful lot of knowledge being leaked away.

Here's the secret: software development isn't really about making computers work, it's about organizing knowledge. If your process focuses more on making the computers work than your institutional knowledge you're making a critical mistake.

Don't confuse some company's crappy engineering practices for agile just because they call them that. Stand ups are short. Period. If it feels like a "permanent PIP" your team could use some improvement. If it feels like a surveillance state it's probably not a very good implementation of agile.

Agile development is about making connections to your team members: _individuals and interactions over process and tools_. It sounds like the author's experience (as well as many other devs') is unfortunately in an organization that doesn't pay heed to that very first, most important agile principle.

<comment><USER3> >Agile development is about making connections to your team members: individuals and interactions over process and tools.

See, this kind of vague, fuzzy feel-good description is exactly why it seems to me that the one thing everyone seems to agree on about Agile is that everyone else is doing it incorrectly.

<comment><USER7> Agreed. I'd like to see some solid experiences that counter the author's claims being used in the wild that fit where he says it doesn't work well (anything other than small, short projects). Nearly everyone is taking the "it's not us, it's you!" approach.

A lot of things work well in theory. That doesn't really make them good ideas if they can almost-never be implemented properly.

<comment><USER8> Yeah. Communism is great in theory as well. It doesn't work though.

</comment></comment></comment></comment><comment><USER9> The free e-meter at every desk is a valuable job perk! The Kool-Aid tasted a bit funny, though.

Scrum-English Dictionary:

* "sprint": artificial crisis

* "end of sprint": abandonware

* "Scrum master": resume credit for my planned escape to a new company

* "stakeholder": someone you can't get away with externalising your costs onto

* "simplified": doesn't implement any of the actual business requirements

* "lightweight": doesn't implement any of the actual business requirements, but does so much more elegantly than the version that works

* "easy": project was born circling the drain (and doesn't implement any of the actual business requirements)

* "legacy": the version that works and implements the actual business requirements, even if no sane human wants to touch it

* "velocity": a speed with a direction, so skittering about following marketing's random hairpin turns counts

* "retrospective": blamestorm incoming!

* "stand-up": getting the blamestorming in early

* "We'll put that on the backlog": hahaha fuck you

I do enjoy some of the jargon. "empowered": do your own fucking job. "Could you just copy down these log files from these fifteen servers for me and put them on the shared drive? Thanks." "I'm sorry, I'm afraid you're empowered to do that yourself."

Seriously: every good idea is turned into a bad one by the relentless management quest to Taylorise clue.

<comment><USER10> The first and last definitions I find are very accurate. Do more! (Retrospective, velocity, stand-up.)

<comment><USER9> :-D Done, also "stakeholder". What's the proper Scrum jargon for "maybe bong hits will fix my makefile"?

</comment></comment></comment><comment><USER11> _Every_ methodology fails from pretty much the same reasoning.

The saving point of Agile is breaking tasks into reasonable sizes, long enough to actually get something done and short enough to rearrange as needed sooner rather than later, and _to delay interruptions_ until they can be evaluated and other tasks completed. I can't get to what I need to do if the relentless onslaught of self-important "do _this_ _now_!" interruptions are handled as they occur; Agile provides a sane way to say "get in line" such that important issues are addressed in a timely manner relative to other issues and without compromising other stuff that also needs doing (those that never reach the top of the interrupt stack, but without which long-term progress won't happen).

</comment><comment><USER12> The term has been hijacked. It is useless. There is merit in the idea that it once encapsulated, but it has long since been co-opted to mean something almost completely the opposite of its original intention.

<comment><USER5> Yeah. Scrum doesn't make management disappear, it amplifies it.

I saw meetings with middle managers (people who hadn't shipped anything significant in a decade) in daily meetings, going over burn-down rates of individual engineers and "expressing concern" through channels that someone was off their schedule by a couple of days. The daily scrums were just half- hour status-in-a-ring and some public shaming if you couldn't say that you did anything, so a lot of people made stuff up, or at least hedged by saying they'd completed 50% of the work or that tests were "nearly passing" (code for: my stuff is broked).

Oh, and in the mean time the build system we devs were using had been on fire and broken for three weeks and it was nearly impossible to get a working build. WTF?

I said, "We don't need Agile. We need an earthquake to collapse that wing of the building" (where the managers were; they liked to cluster).

We had Ken. Ken was there at the beginning for maybe a week, and for the next few weeks after that it was great. But we forgot that middle managers are still gonna manage, and that you have to fire them or they're still going to be there, you know, _managing_ stuff, because that's all they know how to do.

And with the right tools, Scrum can be turned into the perfect surveillance state. (I stole that from another response in this thread, thanks).

<comment><USER13> Intriguingly you've not mentioned 2/3rds of the people in a Scrum team - <USER2> Product Owner and the Scrum Master - both of whom are supposed to be the ones who handle many of the problems you list. Build servers being on fire feels like a perfect example of an impediment. People who aren't actually developing anything feel like the perfect example of people who shouldn't be at standups, and certainly shouldn't be speaking at them.

Again Scrum relies on empowerment, with the Product Owner being the "single wringable neck" whose career is tied to the project and who has the buyin from management to get stuff done. The point being that it is in their interest to have a happy and productive development team doing what they say.

Finally there's also a really good point about internal and external metrics. I've always been a fan of management getting the stats they ask for, not being provided with access to the team's internal metrics which should really be short term to address perceived issues. If we count defects for a few sprints to reduce our defect count that is good, providing them for evermore to management is bad. Separating the two out lets your Scrum Master / line managers have a good conversation about what management are concerned about and how to effectively measure it.

As you said - bad managers won't do any of this, but then flagging up that you have bad managers is A Good Thing (TM) overall.

<comment><USER5> Well, 2/3rds of the people in a Scrum team were essentially dead weight, in my experience. They made work.

In one org, the devs were excluded from the planning meetings, which is pretty much all you need to know about how badly Scrum went off the rails.

"Okay, you're signed up for X, Y and Z, it'll take you two weeks, and tell us every morning about your progress."

"This is day 15 of the build system being totally broken, and I haven't been able to run my code in a week."

"So, you're late. Is that what you're saying?"

(sound of door slamming shut)

</comment></comment></comment></comment><comment> UK-AL IF teams are using Agile like that there doing it wrong. It's meant to be recipe for collaborative, relaxed, friendly environment. Not a pressured hot house. If your goal is to use Agile to push developers to their limits, you've already failed the Agile Test.

You can twist Agile, based on your vision of good team is. So you have to get the right people to implement it. You have to approach it from the idea of it being a collaborative, friendly, no blame etc for it to work.

I've only ever used estimates in story points to prioritize things(That's why its so abstract), not to monitor and push developers. If I wanted that, i'd use actual time.

The idea of everyone is junior is also bad implementation. It means even the most junior person is capable of contributing using their solution over a senior persons solution. The right idea wins, not the right person. I can understand why this pisses of senior people though.

<comment><USER14>
      IF teams are using Agile like that there doing it
      wrong. It's meant to be recipe for collaborative,
      relaxed, friendly environment.

I'm at a place where (IMHO) scrum works reasonably well. But some of the scrum language I hear people using kind of invites misunderstandings.

For example, "sprint" evokes runners running 100m at a pace they can't sustain for minutes, let alone weeks or months. And "committing to stories for this sprint" implies it's some sort of failure or emergency if the target is missed - when in truth nobody should be skimping on testing or staying late to get things done.

<comment><USER15> Yeah, I feel like the language is one of the worst things. A "scrum" is a bunch of sweaty men moving down a field in a pile. The idea of that makes me want to get a different job. I'm attracted to programming because I love sitting in chairs, not engaging in violent pile ups.

I agree with what you said about "sprint," as that is the second worst word in the canon.

And user stories is trying to change the term "feature" so that developers will be more user oriented in their thinking, but the term "story" is disturbing.

<comment><USER15> I forgot the other one: "epic." I can't put my finger on exactly what it is, but it really disturbs me. It's just too serious sounding, like it's presumptuous to declare any component of your program as "epic" before it is done. I can think of very few software systems or components that I would refer to as epic.

Are you writing the story of the flood? Is it the oddysey, the iliad, the history of your people? No, it's just some crappy new feature that's going to take your team a few months to complete. That is totally not epic.

</comment></comment></comment></comment><comment><USER16> One thing about Agile that I don't see mentioned much is that it provides a way for bad managers to marginalize the effects of low performers among their groups, though at the expense of high performers.

Every company has "cowboy" programmers who break shit and ruin their coworkers' lives with massive 1am check-ins, and a lot Agile seems designed specifically to put up road blocks to prevent this from happening. Managers love it because it provides an escape from dealing with problem employees directly. Most of the vehemently anti-Agile people I've worked with were grossly overconfident in their abilities and difficult to work with, which I think supports this idea.

Unfortunately, the only way for this non-management technique to be effective is to do it dogmatically with no deviations (otherwise the cowboys will always have some excuse for why it shouldn't apply to them), and this is where things start pissing off the actual talent.

</comment><comment><USER17> The micromanagement - if management doesn't trust you to get on with work without having to justify what you've done every single day, why did they bother hiring you in the first place?

<comment><USER18> People aren't perfect. It's easy to get distracted, get off track, think that you'd be able to get back on track if only you could get your head down for a couple of days, and suddenly two weeks have passed, you're miles behind, and the people waiting for your work are all thrown off track because it's not there yet.

Spending 15 minutes per day making sure that everyone knows how the team is doing strikes me as a very easy way to avoid that.

Heck, it can be significantly less than that, if you use a tool to keep your tasks up to date and can just say "I got the frobulator finished, and moved on to the doohickey. Dave's giving me a hand with a small holdup there, and it should be finished on time." at the standup then the whole thing should take less than five minutes total, except when there _are_ issues that need to be dealt with.

<comment><USER19> I think that standups are one of the best parts of Agile, if they are time boxed and moderated well ('let's take this [larger issue] offline' should be heard fairly often). Super easy to implement, and very helpful. I've only worked with small teams (less than 8 folks), but I don't think it'd scale well.

</comment></comment></comment><comment><USER20> Agile is like CrossFit: the techniques are sound, but the techniques really aren't the point. The point is to create a revolution by wrapping sound techniques in an us-vs.-the-world ideology that has strong emotional appeal. This makes it easier to sell trainership certifications, which are where the money is.

</comment><comment><USER21> Agile done wrong destroys more than Agile done right can fix.

If you feel too good, or too senior for a programming methodology like Agile, then you are bound to not like it. Is that a fault with Agile, or a fault with your attitude?

A cowboy coder is not micro-managed, does not seem to need atomized tasks, does not need a direct line with the customer and decides on the complexity of the problem and the solution all on his own. They are also terrible to manage and hit-and-miss when it comes to actually shipping something with business value.

Bluntly spoken, Agile is there for the project managers, not the project developers. Agile should be implemented for as long as productivity increases, customer feedback loops create desirable features, and iteration cycles become tighter and shorter. If you know a way to increase those stats without Agile, then write your own methodology ([http://programming- motherfucker.com/](http://programming-motherfucker.com/) is taken) and join management. If you don't care about those stats, and feel too good/senior for Agile, then start your own company and proof it to yourself. You don't start to measure a project's success by first counting the number of developer complaints. You could probably keep the complaints at zero, and never ship anything of impact.

</comment><comment><USER22> If you get stuck in an agile environment, here's a survival guide:

1) Attack the lowest-priority and easiest stories to shoot for quantity, not for quality. This way you'll be closing stories in no time, pleasing the scrum master and product owner during the daily standup. Leave the largest and most complex problem for last - if nothing else, you can kick the can down the road by claiming it's tougher than it seemed, requires more points and should be moved into the next scrum.

2) Do not worry about the quality of your work. In fact, the lower it is, the more stories you'll have to fill in the next scrum - "widget for feature X" with some effort can become "fix bug Y with feature X", "improve widget performance when more than 1 user uses the product", "improve logging and monitoring of widget X" and tons of other, lower-priority, but higher-quantity stories (see point 1 of this). You'll likely be closing a story a day, earning points way ahead of the other suckers who choose to focus on long-term critical design/architecture issues.

</comment><comment><USER23> Reading over his points make me think the OP is a PM who doesn't understand Agile nor his role in it. The number of misconceptions he presents reads like someone who has never bothered to research agile, but only knows what consultants have taught him.

I think the number 1 misconception is the sprints. He seems focused on 2 weeks as being the requirement, when it's on the recommendation. Some team have gone as long as 4 weeks. The whole point to a sprint is that dedicated time to a delivery with fast feedback from the customers. It's mainly so you can fail early, ail often. It's much better to spend 2-4 weeks building the wrong product than 6 months.

Second, his claims that product development, research and design, and architecture don't belong in a sprint are also misinformed. The focus on user stories to remind people that they need to focus on users, not gold plating. In any technical project, there are a plethora to improve on the design. People can (and have) sat down and refactored and "rearchitected" for days and days. At the end, they build beautiful systems that don't offer any advantage to the customers, or have implemented a number of features the customer didn't want, or have fixed a series of concerns the customer wasn't concerned with. There is give and take of course, as the engineers needs to give feedback to help the customer understand the benefits of the feature or risks of not fixing the issue, but those need to happen first before people go off and construct digital masterpieces.

Third, micromanagement. If you have endless meetings in agile, you're definitely not doing it right. The 3 big meetings, planning, retrospective, and demo, are ones that take the most time. Standups take no more than 15 minutes. The other tasks, meaning, backlog grooming and prioritization should not involved your sprint team. Those need to be done offline by the product owner. Agile may be about teams, but you still need strong leadership.

The only point of his that has any relevance is the last one, where consultant firms have made big bucks spinning out variations of agile because people just can't understand how simple it is.

</comment><comment><USER24> Agile development is another tick in the long series of desperate attempts by management to avoid The One Thing That Actually Usually Works™: Hire great developers, pay them a ton of money and give them a lot of freedom and time.

<comment><USER25> I don't see how this makes sense. If I hired a bunch of great developers at great cost I would have even more incentive to maximize their output than I would if I hired a bunch of mediocre people at a lower cost.

</comment></comment><comment><USER26> An eager beaver at one of my client's is introducing Agile as a "Waterfall silver bullet."

I see right away one Agile defect: It prefers near-term "user story success" over long-term investments like creating debuggable architecture.

Why spend time now making things easier for the rest of the cycle if it's not a priority in this iteration?

Why write "gdb" when "gcc with printfs" will work fine, even if it simply requires more sprints?

<comment><USER27> > I see right away one Agile defect: It prefers near-term "user story success" > over long-term investments like creating debuggable architecture.

Neither the Agile Manifesto nor even the far more specific Scrum Guide even _refers to_ user stories.

A particular _organization_ using Scrum may _decide_ that architecture-related concerns are not important in the definition of "done" for items in their product backlog, and focus only on "user story success", but, to the extent that that's a problem, that's not a Scrum (and much less, Agile) problem, its an organization problem.

Agile is about what _concerns_ you favor in figuring out how to get work done, Scrum is about how a team gets work done and defines the work it is to do within the framework provided by the wider organization, but the specific definition of quality standards _for_ work is not directly addresses by either.

> Why spend time now making things easier for the rest of the cycle if it's > not a priority in this iteration?

Why wouldn't an organization have maintaining (or improving) the overall architectural standards of the work a priority for every iteration -- in Scrum terms, part of the Sprint Goal -- unless there is a conscious decision to take on technical debt to meet some other goal? Agile, or even Scrum, doesn't tell you what goals to set, and you shouldn't blame them for you choosing to set the wrong goals.

</comment></comment><comment><USER28> "Agile and Scrum, however, single out and humiliate anyone who works for 2 weeks and doesn't have something to show for it."

Holy crap, how wrong can some orgs go? The team should own the process. If there is R&D that does not match with scrum then the team should have a) a r&d framework or agreement in place b) the authority to say this is r&d.

The process does not own the team. The team owns the process.

</comment><comment><USER29> I always try to differentiate between Agile and agile. Agile (proper noun) is the overly defined practice of scrums and points and backlog grooming etc. agile is the practice of continuous iteration at all levels. Agile is a great starting point, but if you aren't challenging the prescribed format and iterating on your process itself then you are missing the point of agile.

We have found, for instance, that the 3 questions of standup are pretty useless. They make each person feel like they are justifying their existence at the company. By dropping those questions and framing it in terms of a deliverable we found the meetings to be far more productive and better for morale.

We've decided to give ourselves 1 day into the sprint to better guage progress so we can define achievable sprint deliverables, as we don't often know enough information during sprint planning.

Each of these is set up as an experiment, designed to solve an identified problem with our workflow. And the results are great.

</comment><comment><USER30> I don't know how the author is doing Agile, but it doesn't match my experience. I've implemented "Agile" processes on four different teams, and while certain aspects were more successful than others in different contexts, nothing was at all like what is described in the article.

For starters, a very significant part of the article relates to performance anxiety over story points. Over 6 years of doing this, I've had two instances where people were called out for not putting points on the board, both of which came from outside managers not familiar with Agile.

Besides that, one of the core tenets of Agile is that you adjust the processes to work for the people involved. Trying to enforce some arbitrary subset of Agile processes on a team is a recipe for failure and frustration.

This reads to me like "bad managers are bad at managing Agile processes".

</comment><comment><USER31> Pretty spot on, I think.

The most problematic part of SCRUM (and the first thing to start killing productivity) are the interminable meetings it facilitates. The SCRUM master is constantly trying to gauge scope on this or that feature, and everyone feels like they need to bring up what they've been working on during the standup in order to feel like they're not appearing to be slacking off.

I've abandoned SCRUM in favor for a Kanban approach that puts the onus on the Product Owner to manage and estimate scope and deadlines, thereby leaving us developers alone to focus on our work.

I've written more about my approach here: [https://medium.com/@glenelkins/agile-is-good-for-managers- bu...](https://medium.com/@glenelkins/agile-is-good-for-managers-but-bad-for- developers-73bcddac5fe)

<comment><USER32> Its your prerogative of course but peppering your blog posts with stale meme gif after gif is an easy way to get me to not read your content. I don't understand this trend at all for talking about a professional concept. What you did this weekend, sure, but this?

<comment><USER31> Know your audience, I guess. I enjoy gifs, and enjoy getting a sense of the author's personality when I read an article. I don't think I've ever bounced from an interesting article because of a handful of cheerful gifs, but hey, that's just me.

To each his/her own I say.

</comment></comment></comment><comment><USER33> The title is completely mangled.

The original (rather long) title is "Why do some developers at strong companies like Google consider Agile development to be nonsense?", and it seems it was shortened. There's a spurious "at" left in there now which breaks it.

<comment><USER34> This happens when people try to shorten their titles to fit 80 characters.

</comment></comment><comment><USER35> The thing that I don't understand about the way Agile development is treated is that it is often treated like a silver bullet. As the article points out agile has use cases where it is effective but many treat it as a "Waterfall Killer".

</comment><comment><USER36> So instead of Agile we should use, what?

Fully understand that process is a technology.

If Agile is so bad, what's better?

<comment><USER10> I am interested in this as well. Who's using a different process?

<comment><USER5> Why do you need a capital-P Process? Does it need to have a name?

Best teams I've been on have been self-organizing, with a few senior engineers who did heavy-lifting and some less experienced engineers learning new stuff. Management was minimal (the more we had, the more things sucked).

\- Minimize management. Minimize management. Minimize management. Management exists to set very high level directions and to remove obstacles.

\- Make sure that different teams talk. A lot.

\- Get real. Don't blow smoke up people's asses. Making a schedule with a granularity of a day on a six month project is fucking bullshit. Just stop it, okay? You might be able to do this a week out, but not much more than that. If that deadline is important, be clear on why. (I've been on projects that have been absolutely killed because someone felt they had to ship on magic date X, when shipping a few months later would have saved the product).

\- If you are working steady 60+ hour weeks, you are doing it wrong. If you are doing regular 80+ hour weeks then you will burn out and leave.

\- It's done when it's done. Ship it, and iterate.

<comment><USER36> > It's done when it's done.

Ha. Ever have a client? "Dear Enterprise Customer, We'll get it done when it's done. Cool? Can we have the gig?" Yeah no. That's not at all how the world works.

We have process because we have deadlines, commitments, responsibilities... and there needs to be an easy way for people to come and go inside of an organization without having to invent the wheel each time.

</comment></comment></comment></comment><comment><USER37> Dave Thomas (one of the original agile signer peoples), gave this amazing talk criticizing modern, commercialized agile instead of the developer-focused way of "developing with agility".

Give it a watch: [http://www.thoughtworks.com/talks/the-death-of- agile](http://www.thoughtworks.com/talks/the-death-of-agile)

Blog post version: [http://pragdave.me/blog/2014/03/04/time-to-kill- agile/](http://pragdave.me/blog/2014/03/04/time-to-kill-agile/)

</comment><comment><USER38> I've seen many criticism of agile/scrum as they (don't) apply to long-term or systems-software environments, but another disconnect I think people need to know about is with open source. Here's the "upstream first" development model typically used by Red Hat (my employer) and others.

(1) Company decides to devote resources to a feature.

(2) Developer writes code and submits patches _upstream_ (e.g. to the Linux kernel).

(3) Upstream takes some indeterminate amount of time to get patches into a release.

(4) Company defines a _downstream_ (i.e. commercial) release based on a particular upstream release.

(5) The possibly-pristine but also possibly-unrecognizable code from (2) is pulled from the upstream release into the downstream tree.

(6) More testing, debugging, tweaking, etc. on the downstream tree yields an actual release.

Just in (3) we see one part of the problem: upstream quite likely isn't agile/scrum themselves. Often they don't care about a particular vendors' release schedules; if they do, then someone would argue they're "dump and run" fake open source, and not a true independent upstream. They're almost _certainly_ not going to give a crap about sprints or story points or anything else about the local agile/scrum process. These methodologies pretty much don't work unless everyone is completely committed, and with separate upstream/downstream that's just not the case.

Even with everyone on board, the disconnect between (2) and (4) is a problem. Pushing code upstream and pulling it downstream are two separate tasks, even if they're done by the same person, and there's this big gap between them. At best, this makes scheduling more complicated. More often, it makes agile/scrum style fine-grain scheduling and tight coordination impossible. Measuring "velocity" and all that isn't possible when there's so much externally imposed variability.

For something that's long term _and_ systems-oriented _and_ open source, these methodologies are practically useless. Actually they're worse than useless, as they misdirect effort and interrupt useful work. YMMV.

<comment><USER9> And the hilarious (fsvo) thing there is that "agile" was an attempt to port open source at its best to corporate development.

</comment></comment><comment><USER39> Any methodology in the wrong hands will be a terrible experience.

Let's face it. There is no methodology that will make all software development projects around the world go smoothly and fun to work on. If you work with, or for, a bunch of dicks, you won't enjoy what you will be doing all day. No methodology is going to change that.

I would like to go as far to say that "any" methodology used by great people will automatically result in great things.

</comment><comment><USER40> Experienced engineers have seen a lot of "magic bullets" both on the process side and the product side. Generally they turn out to not be a substitute for hiring good people and getting good leaders to define a quality product and focus the team on what is important both short and long term.

<comment><USER9> +1. Attempts to Taylorise clue shall forever be with us.

</comment></comment><comment><USER28> If scrum metrics are used to identify low performers then that's so bad it's not even wrong. The whole point of agile estimates is to teach the team to estimate and pay attention to it. It's a learning tool for the team.

</comment><comment><USER27> First off, Agile isn't a software development methodology, its a set of principles for an organization to apply in selecting/developing in its own software development methodology (you could call it a software development metamethodology.) Scrum is more of a methodology (and, while it may have emerged from the people involved applying Agile principles in the context in which Scrum was developed, and has features designed to support use in an organization applying Agile principles, there is nothing _fundamentally_ Agile about Scrum -- or any other development methodology; if some decisionmaker is sold on Scrum by some consultants and adopts it as a top down practice, that organization may be doing Scrum, but they aren't doing Agile -- in fact, they are doing exactly the thing that Agile is a reaction _against_.) Almost any time someone uses the phrase "Agile/Scrum" they either mean Scrum or something that is neither Agile nor Scrum. In this case, the criticism seems to be of something that is neither Agile nor Scrum, though it may be _related_ to the latter (but not at all the former.)

As to the specific points:

1\. Here, the claim is: _There is no place for an actual senior engineer on a Scrum team. The only way to move up is to become a "Scrum Master", a bullshit management role that involves responsibility without power._

There are several errors in this description: First, "Scrum Master" isn't management role, its a facilitation role focusing on expertise with making Scrum work in the particular organization. The only quasi-management role on (or related to) a Scrum team defined in the Scrum Guide [0] is the Product Owner (which does have authority to go with that responsibility.) Either of those is a role a team member could develop into; the idea that the SM role is the only "way up" on a Scrum team is incorrect even within the narrow bounds that are defined by Scrum.

OTOH, Scrum itself _intentionally_ \-- to support its use in organizations using Agile practices -- defines only a narrow slice of practice, particularly, a basic model for operation of individual development teams. It does not prevent having different roles _outside of the defined interactions within the Scrum guide_ for members of different seniorities or specialties, it only requires that _for the purposes of the interactions it defines_ , only a narrow set of roles ("Team Member", "Product Owner", "Scrum Master") matter. So, consistent with Scrum, there can be room for moving up on a team. And real organizations using Scrum on projects greater than a single 3-9 member team use complementary practices like the Scrum of Scrums [1]; being a Scrum team's representative on the Scrum of Scrums and thus playing a larger role in the coordination of the overall work of the project is one way for a technical contributor on a Scrum team to move into a role with broader scope and exposure.

2\. Here, the claim is "It's aggressively short-term, by design" and "if you carry this way of working on for months or years, you get a lot of technical debt and you get low morale."

It is certainly the case that Scrum, specifically, largely addresses tactical- level, short-term processes. This is because Scrum _isn 't_ a full-stack methodology, its a narrow set of practices designed to adopted as part of a larger set in an organization working in an Agile manner, and to be relatively neutral to the other practices. Certainly, if you apply Scrum _alone_ with no higher-level strategic processes that set standards that constrain the definitions of "done" for work done in sprints to avoid technical debt, there is a lot of potential for an organization applying _only_ what is in the Scrum guide to lack long-term focus and development technical debt.

3\. The claim here is "It has no regard for the programmers' career needs or desires."

 _Agile_ , as a set of principles, certainly does; Scrum, as a particular set of practices, does not, for much the same reason addressed in the previous point. Largely, though, this seems to be a restatement or alleged further consequence of the first complaint about lack of a way to move up (it essentially seems to be, "because of #1, Scrum leaves me nothing interesting to put on my resume"), and so the response to the first complaint above largely addresses it.

4\. There's a lot here that deserves separate responses, so I'm going to break this one up.

> "It's micromanagement."

No, Scrum is the _opposite_ of micromanagement, since the "management" is done almost entirely by the team.

> "User stories and backlog grooming are there to control what the engineer > works on."

Sure, those things exist to define work, but they are done by the team, so can hardly be viewed as micromanagement of the team.

> "The absurdly frequent meetings (and so many different kinds of status > meetings!) are to intimidate him into not slacking."

Scrum defines four kinds of meetings: the Sprint Planning meeting, the Daily Scrum, the Sprint Retrospective, and the Sprint Review. Of them, the Sprint Review is the closest thing to a "status meeting", though even their the focus is demonstrating the concrete deliverables for the Sprint rather than status reporting. All of the rest are primarily _action /planning_ meetings. "Status" of a sort plays a role in those meetings as an input to the planning aspect, but if they become status-focused meetings rather than planning-focused, you've definitely lost the defined purpose of the meetings in the Scrum guide.

> "The story points are there to track productivity (in some superficial, > inaccurate way)."

Story points (in methodologies where they are used; they aren't part of Scrum) exist as a planning tool for the team, not a productivity measure.

5\. The claim here is "At identifying low performers, it has an unacceptably high false-positive rate."

Scrum (and, _a fortiori_ , Agile) is not a process for identifying low performers, or for performance evaluation at all -- its completely out of scope. Its not that it is good or bad at it, it doesn't even _address_ the issue.

6\. The claim here is "It punishes R&D, and it hurts the best engineers the most" because "Agile and Scrum [...] single out and humiliate anyone who works for 2 weeks and doesn't have something to show for it."

Research is a separate domain than product development, and there is no reason that the use of Scrum as a product development methodology should have any impact on how research is done. Of course, Scrum is a generic enough process that you _could_ use it to manage research efforts as well, but the backlog items, associated definition of "done" for those items, sprint length, etc., would all likely be very different for a research effort than a product development effort. You could mix research items into the backlog of a team also doing product development, but then you probably have a problem defining work items that are appropriate for the product development sprint length. (This is less of a problem for non-Scrum methods that use a flow method for backlog management rather than a timeboxed Sprint cycle, because then its easier to mix items that have a longer-than-usual time involvement into the same backlog; the sprints used in Scrum place an effective upper bound on the size of work items.)

7\. The complaint here is "I have actually seen it kill companies" backed by a single non-specific anecdote and the assertion that it generalizes. From the description, this appears to be a top-down all-at-once conversion to a new defined methodology. If one has processes in place, for the same reason one adopts incremental change to a software system where you've got an existing, functional system unless there is some reason you _can 't_ do incremenetal change, the same thing should be done to a software development organization. Failure of an organization (or many organizations) to manage a big-bang change rather than incremental change doesn't invalidate the methodology they sought to change _to_ any more than failure of an organization to manage a big-bang software system replacement invalidates the technology they sought to change to.

[0] [http://www.scrumguides.org/docs/scrumguide/v1/Scrum-Guide- US...](http://www.scrumguides.org/docs/scrumguide/v1/Scrum-Guide-US.pdf)

[1] [https://www.scrum.org/Blog/ArtMID/1765/ArticleID/12/Resurrec...](https://www.scrum.org/Blog/ArtMID/1765/ArticleID/12/Resurrecting- the-Much-Maligned-Scrum-of-Scrums)

</comment><comment><USER13> To go through some of the points I fundamentally disagree with: "There is no place for an actual senior engineer on a Scrum team" \- There absolutely is, who is peer reviewing the code, working with the BAs to design the architecture, identifying the technical dependencies in the backlog? Scrum is not a methodology for writing code, it is for getting a series of tasks done. All the things a senior dev needs to do on top of code can be broken into tasks, Scrum adds a weekly "is this still the most important set of things to be doing?" meeting into the mix.

"It's aggressively short-term, by design" \- The whole post talks about 6 week projects. Scrum is terrible for that as it imposes far too many meetings. Scrum works for the complex space where you say "12-36 months" as your gut feel estimate. It is a reaction to the waterfall world where you'd discover 2 years in you weren't building what the business wanted.

"It has no regard for the programmers' career needs or desires" \- Stories make for excellent opportunities for juniors to "own" bits of work from initiation to completion. Old school projects provided very little opportunity for juniors to work on requirements gathering and thus a big chicken and egg problem for promotion. It also offers plenty of opportunity to interact with your customers, which is almost always a good way to get promoted.

"The story points are there to track productivity" \- Oh noes the people paying you want to know how long the project is going to take to be done. The talk of many status meetings seems decidedly anti-Scrum and is an excellent example of the sort of thing that people should be screaming about in retrospectives.

"It punishes R&D" \- Ironically it can be a really useful opportunity to conduct R&D. Scrum makes you have something measurable and a time frame in which to conduct it, which makes for an easier sell than an open ended request. I think what annoys some is when they get shut down too quickly, or are forced into taking a less technically interesting approach that delivers enough business value. Scrum isn't there for pure research into the theoretical, but then no-one is claiming it is.

Killing businesses - Much like every tabloid headline everything can kill you. Value destroying mergers exist in abundance, and poor leadership/management is a real killer. Plenty of companies do very well out of Scrum, especially when they are in a cost centre environment.

I might sound rather "no true Scotsman" when it comes to Scrum but there are only a few things you need to do to be "doing Scrum". What it does give you is a powerful question to ask, and a regularly mechanism to raise it via the retrospective - "why are we doing X when it isn't in Scrum?".

<comment><USER41> > Scrum is not a methodology for writing code, it is for getting a series of > tasks done.

0_0 Y...you didn't just write that, right? I'll be unusually charitable and give you a chance to think again about that sentence up there.

> "The story points are there to track productivity" \- Oh noes the people > paying you want to know how long the project is going to take to be done.

And yet, points aren't supposed to be translated to time intervals..... (Spoilers: in practice, they are.)

<comment><USER13> What is wrong with that sentence? TFA focuses far too much on the coding aspects of Scrum by asking "what does the senior dev do?", my point is all the things a senior dev does are fit to be tasks or stories in Scrum. It's outgrown being developer focused, and even more literally writing code is a small subset of "software development".

Points don't have to track time intervals to give you a sense of progress. Averages are a powerful thing, hopefully management is focused on long term graphs not getting het up about the number of hours difference between your tasks for two similarly pointed stories.

Oh and Scrum makes no requirement re using story points - it only requires that backlog items be estimated. We estimate tasks in hours and stories in points for instance, Scrum doesn't care. All our Product Owner shows to the stakeholders is a burnup with just numbers on the x axis.

<comment><USER41> > What is wrong with that sentence?

So, writing code has nothing to do with getting a task (of ANY KIND, mind you) done?

Again, do you want to take a moment and rethink that?</comment></comment></comment></comment>


Simpson's paradox: why mistrust seemingly simple statistics - <AUTHOR> http://en.wikipedia.org/wiki/Simpson%27s_paradox

<comment><USER2> This should be part of a required reading regimen for anyone about to post yet another 'Bullshit Study Reveals Whimsical Quirk' article.

<comment><AUTHOR> More importantly, it ought to have been part of the required reading regimen for the authors of said study.

<comment><USER3> Even more importantly, it ought to have been part of the required reading regimen for every single high school student.

</comment></comment></comment><comment><USER4> This also comes up in immigration studies. Say you have two countries A and B. A has an average income of 5, and B has an average income of 20. If a resident of A earning 10 moves to B where he earns 15, the average income of both countries goes down even though total income increases.

</comment><comment><USER5> This is fascinating.

OTOH, it depresses me - we try to do a fair bit of analysis on surfing patterns, A/B testing features at <http://SmartFlix.com>, looking at average basket size, etc.

...and now I've got to worry that I've got "confounding variables". Maybe feature X decreases check out rates ... but actually increases check out rates both among male and female customers....

ARGH. My head hurts.

<comment><AUTHOR> The good news is that when you do A/B testing, you can create a properly randomized experiment, where variations of e.g. the number of men/women assigned to the A and B group are accounted for as part of the sampling error.

Where this stuff really gets you is in retrospective data analysis, where with the right choice of would-be confounding variables you can pretty much argue both directions on any question.

<comment><USER6> The problem that I have seen in these frameworks that handle A/B testing is that they ignore the basic laws of statistics. All groups are different given large enough n. That's what p is about, do you have enough data to tell the means apart. The closer the means, the more data you need. What you really care about is how large the difference is. That's called effect size, which is really just how far apart the means are in divided by a combined standard deviation for both groups.

I certainly wouldn't be making rash decisions on data without a good effect metric.

</comment></comment></comment><comment><USER7> In the example of editing wikipedia it mentions: "This imagined paradox is caused when the percentage is provided but not the ratio."

We see this so many areas. China's GDP is growing 12% every year and the US has grown just 2% type of arguments.

I haven't been able to improve on my yearly salary raise of 300% in the last 15 years when I went from 5$/hr to 15$/hr

<comment><AUTHOR> Just comparing percentages off different bases isn't quite what the paradox is getting at, but it's also a fine approach to representing data in misleading way. There's a great collection of such less technical tricks in the book _How to Lie with Statistics_ by Darrell Huff.</comment></comment>


Over-networking leads to groupthink - on digg, reddit, here? - <AUTHOR> http://cosmiclog.msnbc.msn.com/archive/2009/07/23/2007019.aspx

<comment><AUTHOR> Original article (from Science):

"Can We Reinvent the Internet?" by Viktor Mayer-Schönberger

Lee Kuan Yew School of Public Policy, National University of Singapore 259772, Singapore. E-mail: firstname.lastname@example.com

Recently, researchers who support "network neutrality" have become worried that the Internet may lose its innovative edge. They are concerned that control could be shifting from the edges of the Internet toward the service providers at the center, which would allow the providers to have "gatekeeper" capacity and would contradict the Internet's "end-to-end" principle (1–3). This core tenet states that control over information flows should take place, to the extent possible, at the end points of the network (4). President Obama supported net neutrality during his campaign (4) and in recent statements (5), and the European Parliament has added net neutrality to its recent telecom bill (6). Taking the end-to-end principle from protocols to users, Jonathan Zittrain has called for maintaining the Internet's "generativity," the ability of users at the network's end points to create, distribute, and run whatever software code they choose (7). There are good reasons to preserve network neutrality and generativity, but it is unclear whether these are sufficient to ensure continued innovation. The larger issue is what policies are optimal to foster innovation on the Internet?

Innovation and the Internet

Given that the Internet connects general-purpose processors, most of the innovation happens in software code, which, once released, is rapidly distributed through the network. A new version of Apple's Safari achieved 11 million downloads in 3 days (8); many open-source software packages are compiled nightly to be ready for (automated) download the next morning. A combination of global Internet connectivity, automatic software updates, and centralized code management makes this rapid distribution possible. But these updates are mostly minor changes to software already installed on user machines. Although the ability to rapidly react to hacks and bugs by pushing down a software update is no doubt beneficial, it tells us little about the Internet as a system that enables code innovation.

In contrast, new software products or radically new versions of existing software often face a difficult time getting accepted in the marketplace. Perceived costs incurred when switching from one software product to another, e.g., the cost of data conversion and training, make users keep the software they already have installed (9). Costs are higher still if a code switch necessitates hardware changes, as well (especially when, owing to its complexity and specialization, the required hardware is expensive). Moreover, positive network effects ensure a pull toward the popular and established, but not necessarily the most innovative, software products. A superior new software product may attract relatively few users for a prolonged period of time, before finally, rapidly taking off (10). Conservative user choice, in turn, influences product development, pushing most vendors to alter their software titles only incrementally to avoid the risk of losing their user base. Microsoft's woes in converting users from Windows XP to Windows Vista offers a powerful case in point, so does the painfully slow switch from the network protocol IPv4 to IPv6 (11). This tilts development of new code toward incremental innovation.

The Social Network of Users and Coders

Peer production of software (facilitated by the open-source culture, new tools for managing distributed software development on the net, and global connectivity) has emerged as a powerful new mechanism of software creation (12). It has even extended beyond code creation (e.g., Wikipedia). In contrast to commercially developed, proprietary software, open-source development regularly involves users—not necessarily in writing code, but for reporting bugs, flagging areas of improvement, or partaking in online design discussions (13). Decisions are often driven by the need to establish a rough consensus of those participating (14, 15). Because peer production of software is organized through social networks, the Internet functions as both the technical infrastructure to diffuse code and the tool to maintain a social network, enabling communication and information-sharing among code writers and users. However, reliance on social networks may make open-source development just as conservative as commercial software production.

Figure 1: Structural holes Diversity through limited connectedness

Many open-source projects consist of a small number of active coders and a larger number of bug reporters and commentators, all connected with each other through the Net (13). This thicket of connections within the community of a particular project facilitates coordination and information exchange among its members, but, as a social network topology, it is not conducive to radical innovation. Burt has suggested that an individual node's value is not proportional to the number of its links, but whether these links are within one's existing community or connect two disparate groups (16). In the latter case, the two connecting nodes are valuable information brokers, spanning what Burt terms a "structural hole" see figure, left) (17). Brokerage opportunities between different communities, and the value gained from these, require the existence of such structural holes. Because of the way in which we have organized the development of software code through the Internet—in relatively well-connected online groups—open-source communities may suffer from a lack of such structural holes. An overabundance of connections over which information can travel too cheaply can reduce diversity, foster groupthink, and keep radical ideas from taking hold. This may explain, at least in part, why much peer-produced code rarely is altered in more than an incremental manner (18).

Figure 2: Groupthink Many or all nodes linked to each other

This is less problematic in a competitive, profit-driven setup. If a software vendor falls too far behind the innovation curve because of too-conservative incremental changes, a competitor can mount a successful challenge. Such a competitive setup, in essence, creates two disconnected communities of development. It may require significant initial capital expenditure and time to build momentum. But the prospect of future (if temporary) market dominance fuels risk-takers, as well as a sufficient inflow of venture capital.

The situation becomes problematic, however, when there is insufficient economic incentive for multiple products. The most prominent example is not commercial software code, but the Internet, or more precisely the protocols underlying this dominant network infrastructure. It is too costly and risky for a commercial competitor to create and market a set of radically improved, but incompatible protocols. This is true for the peer-producing, open-source community, as well.

The National Science Foundation supports the Network Science and Engineering (NetSE) program and the Global Environment for Network Innovations (GENI), two mechanisms aimed at enabling more radical network innovations (19, 20). Their work could be improved by incorporating the social network perspective on code creation I have outlined above. Given that they already have a social science component, such integration may not be too difficult. We need to create incentives to form network structures that stimulate and enable innovation.

Overcoming Connectedness

How should policy-makers respond, especially when wanting to facilitate the creation and growth of the next-generation Internet?

If densely connected social networks, although they permit incremental innovation, are hampering radical innovation, two potential responses come to mind. The first focuses on the physical network, implying that it may be necessary to build a next-generation Internet outside of the existing infrastructure. This could free development of the legacy constraints that may hamper radical innovative changes. Any government money to build out the backbone of this future network could mandate that more radical innovations be implemented, with interoperability achieved through defined gateways rather than built right into the system.

This option, however, may be of limited value, if development of the next iteration of Internet breakthroughs, or even the next-generation network itself, is organized through the Internet. Unlike their brethren during the Manhattan Project, today's coders stay connected through their social networks and remain tethered to the context of the existing structures. Project requirements may permit them to consider radical solutions, but their connectedness to the thinking of the status quo through their social networks holds them back.

The second policy option, therefore, focuses less on the physical than the social networks among the research and coding community. When offering public funding for development, government could mandate that the mechanism for allocating funding breaks loose of existing social networks. For example, funding proposals could be encouraged to involve smaller, but competing, groups and for longer periods of time, reintroducing limited separation, somewhat akin to what in the business literature is referred to as modularity (21, 22). Proposals could be reviewed not by peers, as is traditional (who are likely from the same social network as the grant writer, which impedes heterogeneity of ideas), but through a panel of experts one step removed from the research community. Adding reviewers from more risky commercial startups could induce review panels to fund more radical and diverse proposals. The European Institute of Innovation and Technology, a funding initiative of the European Commission, has adopted a variant of this suggestion. Eight of its 18-member grant-awarding board come from the private sector (23). Announced well in advance, such measures signal to the research community that risky and radical proposals are particularly welcome.

Much of commercial development, owing to trade-secrecy concerns, is already less densely linked into a social network, with the result that less policy intervention may be required. Among the available policy tools for government are direct and indirect financial incentives for smaller startups. Government may also want to offer incentives for "skunkwork" projects—deliberately removed from the corporate structure and the connectedness this entails—within larger companies. The creation of the original IBM personal computer (PC), as well as the Apple Macintosh, are good examples of the kind of commercial behavior to incentivize.

Most of the deployment of the next-generation Internet—from the infrastructure all the way up to the application running on the network—will likely be privately funded. However, in areas where government steps in to fund network deployment—to overcome the digital divide, and/or to stimulate the economy—such funding could similarly shift from using efficient, but less innovative, mid-life-cycle mass-produced infrastructure equipment to more cutting edge technologies.

Conclusion

Connectedness of the current Internet constrains the ability of software coders to innovate and to build the code we need: messaging that is more resistant to spam, code to enable innovative controls of the personal information we share through social networking sites like Facebook, and measures to take open online collaboration to the next level, or to help us focus and tackle our society's most pressing problems. To enable innovations, especially nonincremental, discontinuous, and radical ones—which are needed, among other things, to launch successfully the next-generation Internet—may require unique policy intervention: reducing the social ties that link its coders.

References and Notes

* 1. W. Baumol et al., "Economists' statement on network neutrality policy" (AEI-Brookings Joint Center Working Paper No. RP07-08, American Enterprise Institute for Public Policy Research–Brookings Institution, Washington, DC, 2007).

* 2. B. Van Schewick, J. Telecommun. High Technol. Law 5, 329 (2007).

* 3. J. Saltzer, D. Reed, D. D. Clark, ACM Trans. Comput. Syst. 2, 277 (1984). [CrossRef]

* 4. MoveOn, Huffington Post, 29 October 2007; www.huffingtonpost.com/2007/10/29/obama-promises-toreinsta_n_70317.html.

* 5. Editor, Seattle Times, 8 June 2009; [http://seattletimes.nwsource.com/html/editorials/2009315207_...](http://seattletimes.nwsource.com/html/editorials/2009315207_edita09neutrality.html).

* 6. D. Meyer, ZDNet, 6 May 2009; <http://news.zdnet.com/2100-9595_22-299414.html>.

* 7. J. L. Zittrain, Harv. Law Rev. 119, 1974 (2006). [Web of Science]

* 8. J. Dalrymple, CNet, 12 June 2009; <http://news.cnet.com/8301-13579_3-10263494-37.html>.

* 9. H. Varian, C. Shapiro, Information Rules: A Strategic Guide to the Network Economy (Harvard Business School Press, Boston, 1998).

* 10. C. H. Loch, B. A. Huberman, Manage. Sci. 45, 160 (1999). [CrossRef]

* 11. A. Ely, Information Week, 20 December 2008; www.informationweek.com/news/infrastructure/ipv6/showArticle.jhtml?articleID=212501014.

* 12. Y. Benkler, The Wealth of Networks: How Social Production Transforms Markets and Freedom (Yale Univ. Press, New Haven, CT, 2006).

* 13. J. Lerner, J. Tirole, J. Ind. Econ. 50, 197 (2002). [Web of Science]

* 14. K. R. Lakhani, E. von Hippel, Res. Policy 32, 923 (2003). [CrossRef]

* 15. Steven Weber, The Success of Open Source (Harvard Univ. Press, Cambridge, MA, 2004).

* 16. R. Burt, Brokerage and Closure (Oxford Univ. Press, New York, 2005).

* 17. R. Burt, Structural Holes: The Social Structure of Competition (Harvard Univ. Press, Cambridge, MA, 1992).

* 18. S. E. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies (Princeton Univ. Press, Princeton, NJ, 2007).

* 19. NetSE, www.nsf.gov/funding/pgm_summ.jsp?pims_id=503325.

* 20. GENI, www.geni.net.

* 21. C. Baldwin, K. Clark, Design Rules: The Power of Modularity (MIT Press, Cambridge, 2000).

* 22. P. Evans, B. Wolf, Harv. Bus. Rev. 83 7, 96 (2005). [Web of Science] [Medline]

* 23. European Institute of Innovation and Technology, <http://eit.europa.eu/about-eit/governing-board.html>.</comment>


Snap Framework: What's new this week? - <AUTHOR> http://gregorycollins.net/posts/2010/05/30/snap-framework-update

<comment><USER2> This one is interesting for philosophical reasons: _Fixes to the way Snap handles accept-encoding headers in the GZip code — requests from Konqueror and Links are no longer incorrectly rejected._

There is a sort of epic conflict here -- Haskell encourages you to strictly validate all data structures (and reject anything that's not exactly right), while the Internet encourages you to find meaning in any message. Of course, you can do that in Haskell, but you feel bad about it. And then your web server doesn't work with Konqueror or Links :)

<comment><USER3> Guilt-Oriented Programming?</comment></comment>


California SB-1424 Internet: social media: false information: strategic plan - <AUTHOR> https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180SB1424 <comment><AUTHOR> A bill introduced by a State Senator (not yet law, perhaps not even likely to become law) to regulate internet speech. Note that despite being titled "social media' the law proposed would apply to a lot more than just social networks, even including text messages. From the text of the bill:

> As used in this section, "social media" means an electronic service or > account, or electronic content, including, but not limited to, videos, still > photographs, blogs, video blogs, podcasts, instant and text messages, email, > online services or accounts, or Internet Web site profiles or locations.

<comment><USER2> Maybe trying to get out in front of the feds. Certainly seems to be the direction things are headed based on the zuckerburg congressional spectacle.

</comment></comment><comment><USER3> Good intentions, I certainly want to see legal progress in response to FBAG's massive unregulated power, but this seems like step 1 to setting up the Ministry of Truth.

[https://en.wikipedia.org/wiki/Ministries_of_Nineteen_Eighty-...](https://en.wikipedia.org/wiki/Ministries_of_Nineteen_Eighty- Four#Ministry_of_Truth)</comment>


Couchsurfing Close to Shutting Down - <AUTHOR> https://blog.couchsurfing.com/couchsurfing-needs-your-help/ <comment><AUTHOR> Note from submitter (me): The title of the announcement is "Couchsurfing needs your help," but the lede of the story is that they're hanging on by a thread.

In my mind the original Couchsurfing community died years ago when they became a for-profit company, started charging membership fees, and proceeded to cycle through multiple CEOs in just a few years.</comment>