Wind and solar power are disrupting electricity systems - <AUTHOR> http://www.economist.com/news/leaders/21717371-thats-no-reason-governments-stop-supporting-them-wind-and-solar-power-are-disrupting <comment><USER2> I can't help but believe that we would be better off (at least in the US) if utilities had embraced renewables 10 years ago. Rather than fight solar and wind, utilities could have played the role of solar installer/integrator and could have encouraged renewable installation and made appropriate investments in infrastructure to support their variability. Instead, many US utilities chose to fight back against renewables slow their adoption. Now the utilities complain that the grid cannot handle the variability and complain that their job of balancing the grid is getting harder. It's just unfortunate that many public utilities in the US chose to fight innovation rather using the benefits of their monopoly to embrace the change and try to profit from it and provide better service.

<comment><USER3> > appropriate investments in infrastructure to support their variability

Erm, why didn't __you __make appropriate investments in infrastructure?

Because it cost money. Why should you expect someone else to make an unprofitable venture if you yourself were unwilling to do so?

In any case, the variability in solar and wind is known. To keep the same level of electrical service will require a HUGE increase in energy storage.

The thing about coal / gas / even nuclear to some degree... you can "turn it off" if there's too much power being pumped into the grid... and you can "turn it back on" if there isn't enough power. That's why natural gas "Peaker Plants" continue to be built, despite the fact that they're much more expensive than Solar or Wind.

The enemy of "Peaker Plants" is cheap energy storage. Coal and Nuclear power is more efficient if you just keep it on 100% of the time and then store the energy anyway.

And believe me, utility companies HAVE been building cheap energy storage. They're called Pumped Hydro for the most part. Other solutions for cheap energy storage simply didn't exist until recently. Lithium Batteries are expensive and don't provide much storage believe it or not... not on the Gigawatt scale needed to last more than a few minutes.

We will likely continue to use "Peaker plants" for the near term, at least until the energy storage issue is solved with something much much cheaper.

<comment><USER4> I haven't invested in the infrastructure because electricity is not my business.

If it were my business, I would still be at the mercy of giant incumbent competitors, and any dollar I spent on common infrastructure would benefit them more than me. They could then use that advantage to slowly force me out of the business.

The largest firms have the most to gain from improvements in the industry's infrastructure. So the onus is largely upon them to make such improvements. In many cases, there is only one firm with significant pricing power--the natural monopolist--and everyone else in their territory has to dance to their tune. They're the only ones capable of improving infrastructure, aside from a government.

The pricing power is the only way for a firm to afford projects that are neither profitable nor a competitive advantage. The only point for everyone else in the market to having firms with pricing power is so they can undertake those projects that benefit everyone in the long run.

If you're not going to improve the industry, there is no advantage for everyone else to allow you to remain as the natural monopolist. The logical thing for them to do is to seize your capital by force and transfer it to someone who might do more with it (but might do less).

A natural monopoly does not mean the same firm will always have pricing power, but that the modal situation is that only one firm has it at a time. When natural monopolists are protected as _de jure_ protected monopolies, yes it does prevent redundant investment, but it also puts no competitive pressure on the monopolist to _remain_ the monopolist.

I would guess that electric utilities would make wiser decisions about infrastructure if they were less protected from competition. That introduces the risk that a particular market might not be adequately served in an economic dominance war that wipes out both incumbent and challenger, but I think those crises would be very temporary, and would likely invoke a government intervention less costly than protecting the monopoly in the first place.

<comment><USER3> We're talking about energy storage, are we not?

If you think that energy storage "makes money", build a small 10MW battery plant to supply a neighborhood or two. Buy electricity when its cheap, sell it when its expensive.

There's no "common infrastructure building" going on here. You own the batteries, and maybe you have to pay the big utility for some power-poles and comply with some basic regulations.

The fact of the matter is: these sorts of infrastructure issues are being solved by the free market all the time. Frequency regulation, long-term energy storage (which really is just ~hours) and such are commonly subcontracted out to smaller companies who own small plants from the big utility companies.

\</comment></comment></comment></comment><comment>-

> I haven't invested in the infrastructure because electricity is not my > business.

Then do the next best thing. Buy stock in companies that are doing grid-scale energy storage across this country.

<comment><USER5> Sell it to whom? Your neighbors? In a lot of municipalities, there's no such thing as "selling it when it's expensive" because the only customer is the utility, and the utility is only paying wholesale rate from solar / alternative energy customers.

It may be an option in some places, but at present, I think you'd lose money under your current plan in a lot of places, and are unlikely to have neighbor- customers in many of the remainders.

</comment></comment><comment><USER6> If you all are interested in the 'deregulated' energy markets in the US, you may want to check out PJM. Via PJM (and similar regional transmission organizations), you can trade energy and even hook in your own generators and storage.

Interestingly, renewables still comprise a tiny fraction of generation. I'm not sure if this is because of regulatory requirements, lack of reliable storage, or because of cost (nuclear and fossils are pretty cheap and reliable producers).

Generally, in the US, distribution companies (i.e. electric utilities) purchase energy from the transmission companies. Many don't generate electricity themselves. Actually, most distribution companies are co-ops. There are still regulation via FERC, and they stay active because of high- impact capacity transmission failures in the past.

But, it's all cool stuff. And, for those interested in machine learning, you can download a lot (or all) of the data and make a lot of money (I used to work in a pseudo-quant like capacity in that space).

[http://www.pjm.com/markets-and-operations.aspx](http://www.pjm.com/markets- and-operations.aspx)

[https://learn.pjm.com/three-priorities/buying-and-selling- en...](https://learn.pjm.com/three-priorities/buying-and-selling- energy/capacity-markets.aspx)

[http://pjm.com/markets-and-operations/energy/real- time/histo...](http://pjm.com/markets-and-operations/energy/real- time/historical-bid-data/unit-bid.aspx)

<comment><USER7> How do you "make a lot of money" in this? Is there some publicly available market that anyone can trade in?

<comment><USER8> liquidity is not inspiring:

[http://futures.tradingcharts.com/marketquotes/JM.html#footer...](http://futures.tradingcharts.com/marketquotes/JM.html#footerclose)

</comment></comment></comment><comment><USER9> Wind energy production is remarkably volatile throughout the year, especially if wind farms are co-located in the same pathways as is the case in Denmark.

[https://apps.axibase.com/chartlab/06c5e0b2/2/#fullscreen](https://apps.axibase.com/chartlab/06c5e0b2/2/#fullscreen)

The difference between min and max production for wind is over 100x throughout the year and inter-quartile range is 14 and 52GWh per day.

So you can't really rely on renewables in this type of configuration just yet. Until storage 'problem' is addressed, possible solutions are:

1\. Maintain traditional power sources as baseline providers. The problem is figuring out what is the right baseline.

2\. Have interconnects with suppliers of non-correlated sources such as Norway's hydropower plants.

3\. Built out nuclear.

</comment><comment><USER10> This is not a US focused article, but I want to provide some clarifications specific to the US:

While the incremental operating cost of wind and solar plants (and the related staffing levels for a given plant size) are much lower than conventional thermal plants because of no fuel cost, they are far from zero or "free". This is a misconception that ignores the need for staffing and maintenance services. Over the 20 year design life of a typical wind plant, for example, operating costs are expected to roughly equal the initial investment in total.

Additionally, the subsidies most used in the US for wind and solar development are the Production and Investment Tax Credits (PTC and ITC). Tax credits for new projects will be phased out completely in the next approximately 5-7 years (the phase out plan is of fixed duration, but the tax accounting rules allow projects to claim credits ahead of project completion as long as certain conditions are met, such as start of construction of outlay of sufficient capital to show commitment to project completion). The US renewable industry does not have a general expectation of tax subsidy renewal, and developers, utilities, and suppliers are all looking to be able to build economic projects without tax incentives. The reality is that costs have come down and performance has increased to the point where these renewable resources are largely able to compete with conventional generation on price already.

Also, the need for transmission line development is apparent, but it's mostly about bringing transmission to the resources rather than needing to rebuild existing lines. That means large scale transmission projects to connect low population areas with high wind and solar resource to population centers. Existing lines and substations do often need upgrades to accommodate new generation, but those are determined based on study and the costs are often (though not always) borne by the project or group of projects requiring the upgrades.

As others have highlighted, large scale energy storage technology is going to be key to reducing the need for peaking power plants to cover load gaps related to these intermittent resources.

Editorially: We are fools if we think we can keep burning things we dig out of the ground forever. The solar energy needed to power our lives is already here, and it's on us to learn to harvest and use it effectively. And by all means continue to work on clean nuclear power but at the moment it is not cost competitive.

<comment><USER11> IMHO, we should have increase nuclear power 20 years ago to avoid the current carbon pollution. Now, it is too late for nuclear. The future is solar power and it is quickly coming. The amount of available solar energy is enormous. In a further future, we may also have solar plants in orbit that would reduce weather problems.

<comment><USER12> I hope that methane cracking will also help.

</comment></comment></comment><comment><USER13> Yes, which is why we need to (as a civilization) put more effort into grid- scale energy storage.

My leading candidate for that is some kind of flow battery. There have recently been some announcements about a better and more stable electrolyte that is also non-toxic and non-corrosive. But as always, it might be five years (or never) before that gets commercialized.

<comment><USER14> We'll need a lot more storage and overcapacity than people think.

For example yesterday germany had a renewable fraction of around 50% throughout the whole day. Yay? Nope. During january there was a whole week where it was in the 0-10% range during the night and 5-20% during daytime. That means we don't just have to smooth out daily variability but on the span of multiple weeks.

Of course that doesn't mean we need to store all of that in batteries or that we need to keep a whole shadow infrastructure of fossil/nuclear plants around. With enough overcapacity from renewables we could also produce hydrogen and store that for example and run it through fuel cells or gas plants when needed.

But that still means that a bunch of smartmeters won't fix the issue, we'll need real infrastructure to handle that variability.

<comment><USER15> > With enough overcapacity from renewables we could also produce hydrogen and > store that for example and run it through fuel cells or gas plants when > needed.

Aren't lithium-ion batteries more efficient than hydrogen fuel cells?

<comment><USER16> More efficient, but also more expensive and harder to scale.

To expand battery capacity you need to dig up lithium and half a dozen other rare elements, deal with the toxic byproducts (or dump them in a lake), manufacture batteries, control circuits and run ongoing maintenance.

To expand hydrogen capacity, you need more pressure vessels. These can be made of steel or aluminum, two of the cheapest materials in existence, and with practically zero pollution. Ongoing maintenance amounts to checking for rust and occasionally replacing valves, so it scales far better.

Hydrogen has serious downsides, but there's definitely a case to be made.

<comment><USER17> I'd love some more info on Hydrogen storage, in theory and in real projects. If you have sources I'd much appreciate them.

<comment><USER16> Nah, what I already wrote already covers about everything I feel sure about, and half of it is just common sense. Shouldn't be too hard to find better sources, though.

</comment></comment></comment></comment></comment></comment><comment><USER18> Sorry for being that guy, but I'm getting really sick of the word "disrupt".

<comment><USER19> It's a red flag. It's useful.

<comment><USER18> Good point.

</comment></comment></comment><comment><USER20> I think the more interesting story regarding disruption is the story about how dominant energy players will no longer be able to influence foreign affairs due to their net exporter status. If you are looking to de-hinge yourself from overbearing geopolitical influences, standalone energy production is incredibly appealing.

</comment><comment><USER21> So this magazine offers the idea of removing fixed pricing for electricity, but is electricity demand that elastic? Perhaps in some markets, so this might be a good option; but should not be made mandatory.

A lot of electricity demand is formed by weather which makes it inelastic in the short-run - you will still pay your bill if you are cold regardless of today's price. In the longer run (next season), you'll just make people give up using electric heating - for natural gas, wood and coal, depending on the region.

Consider another policy - a carbon tax, which is a very popular tool among economists or the similar fuel excise taxes. They will lower profits for plants that solve the intermittency issue with fossil fuels, creating an advantage for energy storage.

 _To keep power flowing, the system relies on conventional power plants, such as coal, gas or nuclear, to kick in when renewables falter._ this is also partially incorrect - nuclear power plants usually are used to generate a fixed amount of electricity, only France I think which is a primarily nuclear in energy generation does it. In fact hydro-electric plants are commonly used for power balancing, with natural gas power plants in second place.

</comment><comment><USER22> The last paragraph says:

> _The bigger task is to redesign power markets to reflect the new need for > flexible supply and demand. They should adjust prices more frequently, to > reflect the fluctuations of the weather. At times of extreme scarcity, a > high fixed price could kick in to prevent blackouts. Markets should reward > those willing to use less electricity to balance the grid, just as they > reward those who generate more of it. Bills could be structured to be higher > or lower depending how strongly a customer wanted guaranteed power all the > time—a bit like an insurance policy._

Most electricity in the US is distributed across wholesale electricity markets with nodal pricing, which have all of these mechanisms where distributors/utilities are the customers. The author appears to suggest moving pricing out to end users, which is possible even just on the utilities side, without having to restructure power markets. Restructuring power markets to take into account individual end user behaviour rather than aggregate demand would be a gargantuan effort--right now the California Independent System Operator's (CAISO) optimization process takes into account a few thousand nodal prices, and there are tens of millions of households in California. Having utilities introduce demand response and smart metering to approximate their costs would be a much simpler implementation.

It is possible (after clearing many hurdles) to participate in the electricity markets in order to identify inefficiencies around renewables and incentivize their use by providing better price signals to planning markets, but this is very challenging and most utilities and power trading groups/companies are not interested in that angle. It is, however, one of the things we're doing at Invenia ( [https://invenia.ca/](https://invenia.ca/)).

<comment><USER23> I think the author is suggesting to give the end user the option to turn on/off power in an attempt to do peak shaving. If spot prices are predicted to go up at a certain time due to grid constraints, then you can inform the user and if they want, they can turn off their air conditioning/washer/pool heater, etc. There would also be the monetary incentive, saving money since the prices are high. [https://www.bidgely.com/](https://www.bidgely.com/) offers something similar to that, but I'm not totally sure.

This also depends on the pricing models allowed within the retailing side.

</comment></comment><comment><USER24> It strikes me that this is very similar to the NBN (broadband network) problem. There is a widely deployed network of old technology that is no longer worth investing in (copper in the NBN case / coal plants) and newer technology (solar + storage) that is still a little pricey (storage) and not widely enough deployed.

Which means you need to manage a transition so it's not too painful.

A first random guess -

One way might be to allow a few coal plants to go bust, and enact "load shedding" (turning off areas of the grid during times of low demand). Within a year or so, enough places will have batteries that you're load shedding less often.

If the government purchases the coal plants when they go bust (at a steep discount), they can then stagger the shutdowns enough that load-shedding is controlled.

But it doesn't seem sensible to force it to be economic to run the coal plant full-time. Surely that just drags out the transition, and suppresses the demand for storage that we're hoping will be the solution?

</comment><comment><USER25> The easiest fix to the intermittency problem is to use electricity when renewables produce [1]. That would reduce the need for batteries in the beginning.

[1] [http://www.co2signal.com](http://www.co2signal.com)

</comment><comment><USER26> The rise of renewables at the expense of fossil is also a tale of technological unemployment across mining, power, and even _transportation_ sectors.

Coal plants, particularly the older ones most at risk of closure in the US, require significantly more labor per MWh generated than cleaner sources.

Here's a recently announced coal plant closure: [http://www.cortezjournal.com/article/20161001/News05/1610099...](http://www.cortezjournal.com/article/20161001/News05/161009993/0/FRONTPAGE/Coal- fired-plant%E2%80%99s-closing-sends-shockwaves-in-Nucla)

...

 _the local job options could be pretty limited in far-western Montrose County once two of its major employers close their doors, eliminating what are currently 55 jobs at the plant and 28 at the mine._

...

According to the EIA, the Nucla plant generated 416,150 MWh in 2015 for an average annual power of 47.5 megawatts: [http://www.eia.gov/electricity/data/browser/#/plant/527](http://www.eia.gov/electricity/data/browser/#/plant/527) That's an abysmal productivity per employee (or a fabulous job source, depending on your perspective): 0.86 real annualized megawatts per employee at the plant ; 0.57 megawatts per employee if you include the mining jobs.

A well-sited utility scale solar farm like Desert Sunlight can produce an average annualized power of 147 megawatts with just 15 full time employees, for a ratio of 9.8 megawatts per plant employee.

The construction of renewable energy systems still consumes fossil fuels as inputs to manufacturing concrete, glass, metals, plastics, and silicon. Construction machinery likewise still runs on fossils. But the lifetime- amortized dependence on extractive industries for a MWh of renewable energy is much lower than for fossil energy.

\---

In the recent past, coal made up 30% of all dry bulk shipping: [http://marketrealist.com/2014/01/key-players-involved- global...](http://marketrealist.com/2014/01/key-players-involved-global- seaborne-coal-trade/)

In the USA, coal is the single greatest railroad transport category by tonnage. From 2008-2012, when volumes were considerably higher, coal was an even greater #1 by tonnage and was also the #1 source of railroad revenue:

[https://www.aar.org/BackgroundPapers/Railroads%20and%20Coal....](https://www.aar.org/BackgroundPapers/Railroads%20and%20Coal.pdf)

There's no commodity that has enough volume to make up for the decline of coal transportation. There's going to be "stranded assets" left in railroads and shipping along with fossil extraction.

And, coming with the rise of electric vehicles, trucking jobs on the chopping block too. Hazmat certified tanker truck drivers -- the ones who drive tankers carrying flammable fluids -- are highly paid, well above trucking jobs in general:

[http://www.truckdriverssalary.com/tanker-trucking- salary/](http://www.truckdriverssalary.com/tanker-trucking-salary/)

[https://qz.com/24388/how-to-earn-170000-a-year-driving-a- tru...](https://qz.com/24388/how-to-earn-170000-a-year-driving-a-truck- warning-its-messy/)

Their ranks will dwindle if/when electric vehicles suppress demand for refueling with fluid fuels. Even if self-driving trucks never pan out.

</comment><comment><USER27> what about a gravity battery? and it can serve as a space catapult sometimes too

<comment><USER28> You mean like this? [https://en.wikipedia.org/wiki/Pumped- storage_hydroelectricit...](https://en.wikipedia.org/wiki/Pumped- storage_hydroelectricity)

<comment><USER29> Probably this: [https://en.wikipedia.org/wiki/Space_fountain](https://en.wikipedia.org/wiki/Space_fountain)

<comment><USER28> That is very cool, thanks. I don't think it could be used for energy storage though.

 _Its major disadvantages come about from the fact that it is an extremely- high-energy active structure. It requires constant power input to make up energy losses and remain erect._

<comment><USER29> Wikipedia used to have a much more elaborated explanation on them.

A superconductive structure with evacuated rails wouldn't need energy to stay in place (always in theory - one would certainly need something active). An electromagnetic one will use a lot of power, but still much less than the amount stored.

Sill, it's not something simple that one could deploy today.

</comment></comment></comment></comment></comment><comment><USER30> Better title: "Green power subsidies are disruption the power economy". Subsidies are good for wind and solar, but corrosive to existing infrastructure, which are being squeezed to stay in business. Maybe some more measured plan would be good. Instead of blind enthusiasm.

<comment><USER31> And nuclear, oil, coal were never subsidized? It's not like most of these huge power plants were built with gov money ,right? It's not like we spent billions to protect our interests in places that have abundance of oil, but lack of stability. It's not like we subsidize coal industry, farming and other part of the economy, right?

<comment><USER32> Capitalism is responsible for fossil fuels and climate change but only kept alive by greed.

Government is responsible for renewable energy and saving the planet but only kept alive by subsidies.

That's the reductionist narrative that you find everywhere but largely misses the bigger picture. You don't have to dig very deep to find counter points.

<comment><USER31> While I would prefer that our society didn't have to depend on fossil fuels, without them our advanced society would not be possible. Fossil fuels are bad but they were/are necessary. All we need to do know is to just recognize that they will not be needed in the future, and keep investing into renewables.

<comment><USER32> > All we need to do know is to just recognize that they will not be needed in > the future, and keep investing into renewables.

They may soon not be needed in western society - which can afford the capital investments into renewable tech and the subsidies to push rates to competitive markets rates.

But all of those oil wells and mines still exist. They will just find a market in developing countries such as Asia and Africa. Africa is going to be where the greatest amount of population growth will be for the next few decades. Combine that will a flood of cheap oil and it will likely partially offset the benefits of renewable energy in the short term.

Eventually renewable tech will be so cheap that it will no longer make economic sense to use oil/coal but I don't think that will happen globally as fast as people hope.

<comment><USER31> >But all of those oil wells and mines still exist. They will just find a market in developing countries such as Asia and Africa.

I have to disagree. Consider this - when countries in central europe, after the fall of Berlin Wall, began to invest into IT and Communication infrastructure they usually invested into the new technology, simply because that made more economic sense. They did not have to go through analog->digital, instead they jumped right into the newest tech.

Developing countries in Africa and Asia will be able to skip right into renewable and save money by not having to build new power lines, new power plants, transfer stations etc. Renewables will grow much faster than most people anticipate, and since the demand for fossil fuels will drop, their price will grow, which in turn will turn renewable into even better investment.

<comment><USER32> True, that might be the case as power plants and infrastructure are expensive. But there's also existing infrastructure and there is a lot of industry already in place supporting fossil fuels, such as equipment makers, manufacturers, and fuel suppliers.

These operations won't necessarily just go out of business or just become renewable energy firms, they will likely drop prices and sell to other developing markets.

Renewable energy is largely defined by high upfront capital costs but long term savings (solar panels, wind turbines). This is the primary barrier that I see in poorer countries. The cost of fossil fuels is more distributed over time as you buy cheap engines and pay more over time for the fuel.

Again this is the short term side effects. Eventually it will shift to green tech as there will also be social pressures, not just economic ones. It will also be a positive benefit as prices drop and helps develop industry - even if their are climate costs.</comment></comment></comment></comment></comment></comment></comment>


Economists Take Aim at Wealth Inequality - <AUTHOR> http://www.nytimes.com/2016/01/04/business/economy/economists-take-aim-at-wealth-inequality.html <comment><USER2> It's nice to see the opinions of actual experts, as opposed to the opinions of politicians.</comment>


Stack Overflow Fatigue - <AUTHOR> http://robertelwell.blogspot.com/2010/06/stack-overflow-fatigue.html <comment><USER2> I quit StackOverFlow once they devalued asking questions. No one on the site seemed to understand that incentivizing people to ask great questions was the most important thing. The site could have been a great starting point for learning esoteric topics like Haskell, Scheme, Lisp, Grails, Scala, Lift, Go, Emacs Lisp, etc.

Basically, in general, you need one good question with 2 or 3 good answers [yes, some types are better with 100 answers].

"How do I reverse a string in Java?" "How do I reverse a string in Haskell?" "How do I ... in Haskell?"

The questions could have been linked to build a Rosetta Stone, as well as a complete beginner's guide to topic X. Actually, it could have been a guide from beginner to guru.

<comment><USER3> Are you using the site for reputation or to spread knowledge? If its the former then you're welcome to leave, if its the latter then why do you care about reputation?

<comment><USER4> The fact that points on SO actually unlock admin and moderation features on the site gives them real value. I don't care at all about my Reddit karma, but if it meant I could eventually retitle misleading posts I would.

<comment><USER5> I think that's a place where Wikipedia presents a better model in a lot of ways. People love to complain about its cliquey admin groups, but the fact is they're extremely effective at getting quality content out there.

Bottom line, I think, is that forcing people to discuss (not just summarily up or downvote) is really important to getting meaningful content. It's fairly easy to see when someone is talking out of their ass, but it's much harder to get a read on how intelligent an anonymous (or even attributed) vote is.

</comment></comment></comment></comment><comment><USER6> Disclaimer: This is no flame, and this is no whining. This is my personal experience with stackoverflow, which may entirely depend on me doing complicated akward things.

My experience with stackoverflow boil down to these two situations:

I ask a question, write my little heart out to provide context, to provide things I have tried already and to state my problem, because I don't have trivial little coding problems I can solve with google (yes, that was arrogant, I am aware of this.). After I have posted such a question, I usually get like 3-4 1 line answers and then the question goes dead, because apparently, the answer to such a question is more complicated.

On the other hand, if I answer a question, I usually look and poke around a bit in order to provide a good, complete answer to an interesting question. Once I have posted such a question after like 10 minutes, there are like 3-4 one-line-answers posted already, upvoted already and my answer (which usually is more accurate and all) is just ignored and stays somewhere in the middle.

Joining these two experiences together, I have moved on to use groups and certain IRC-channels, as these are more helpful for nontrivial questions.

<comment><USER7> I agree entirely - I've had exactly the same experiences. I think both of them boil down to the site suffering from it's own success.

This seems to be fundamentally because once one reasonably correct answer picks up an upvote or two, it often snowballs into picking up many more because future visitors won't add other answers of their own and just upvote that. This leads to there being a significant reward for getting your answer in first - more so than for it to be complete (or, in some cases, correct) - so users aim to answer as fast as possible. As their user base has grown, that time threshold has gone down, and the quality of many of the answers follows it.

I don't know what the answer to that is though - maybe there's not one.

<comment><USER8> I also agree....almost a year ago this problem was actually discussed on the SO podcast (people doing the instantaneous one liner reply in order to get upvotes asap, the problem somewhat originated from a blog post someone did one specific ways to game the SO rep system).

I have a similar problem in that I ask questions that are generally "harder" and don't often have a perfectly black and white answer, but I'll get a few "little effort" answers and then they go dead, and I am left with a poor answer acceptance ratio. If I ever eventually figure out a question, I always go back and answer it myself, so really I'm trying to do the right thing as I am a big believer in the site, but I think it misses subtleties like this. It's not particularly easy, but I think they could improve things to stop gaming of the system.

<comment><USER9> > It's not particularly easy, but I think they could improve things to stop > gaming of the system.

Or put another way, they just change the mechanics, so that the gaming the system means doing the right thing.

</comment></comment></comment></comment><comment><USER10> I'd like to be answering questions there to provide a service, but I'd also like to receive recognition for it. But it seems like that's no longer possible, at least not at a level of tradeoff that's acceptable to me.

For one thing, I always try to provide a _tested_ answer, but doing so means that someone frequently jumps in front of me, scoring the reputation. In many of these cases I can see that the answer provided might be conceptually correct, but wasn't actually tested, because there are typos in the code. Thus, it appears that the incentives are skewed away from providing the highest-quality answers.

I pretty much quit about two weeks ago. I spent time formulating an answer [1] that was entirely correct, so by the time I posted it there were three other answers there, but they relied on special cases rather than being generally correct. One person upvoted my answer, but almost immediately, someone else downvoted it. The only thing I can imagine is that one of the "competing" answerers wanted to ensure that his mostly-correct answer was listed above my completely correct one, so he could score points from readers that hadn't yet seen my reply that explained why special care was needed.

If my diligent work is going to be devalued like that, then I don't see the point in investing more time in it.

[1] [http://stackoverflow.com/questions/2975588/selecting-a- range...](http://stackoverflow.com/questions/2975588/selecting-a-range-with- two-fields/2975740#2975740)

<comment><USER11> I agree, because this kind of thing is very common. I'd have taken it a step further: if you provide an answer to a question, you shouldn't be allowed to downvote other answers to the same question. And if you downvote an answer to a question, you're not allowed to post an answer to that question. So get rid of conflicts of interest, and always allow people to upvote other answers, whether they've provided an answer or not.

<comment><USER7> I disagree; I've done that before when I've answered a question, because I felt another answer had appeared which was incorrect or misleading.

For example, I answered one question about some kind of 2d sorting issue - another answer popped up which was both incomplete (it only dealt with the 1d case, which was trivial) and incorrect (it claimed to run in linear time, but was blatantly using a nlogn sort). I felt that a downvote was the right thing to do - but that does rely on the honesty of the users, and it sounds like that's often lacking.

<comment><USER8> I disagree also (re: I'd have taken it a step further: if you provide an answer to a question, you shouldn't be allowed to downvote other answers to the same question.)

There's definitely a problem here and something needs to be done, but that's not the solution.

I wonder if something like displaying answers on newly asked questions, but locking voting for <x> minutes (where x=???), and total_answers < <y> wouldn't stop the answer sniping in its tracks and give people willing to put the proper amount of work into writing an answer an equal footing.

SO has proven that the community for some reason is more than willing to spend vast amounts of energy helping others in exchange for "karma", but from what I'm reading lately, they are somewhat in danger of letting the golden goose die (or, a lot of the golden gooses). It'll still be a great site, but it could be a lot better if they tightened some of these problems up. I think they might be focusing on other things now though.

</comment></comment></comment></comment><comment><USER12> The problem with StackOverflow is that it doesn't allow discussions, and as long as that will be the case I only see it working for general, domain- agnostic questions. If you have questions about the dark corners of C++/C#/Java, or how to best apply a design pattern, then chances are SO has something for you. As we level up as developers we gravitate relatively quickly towards specific domains. And a community of domain experts will not last long if it explicitly forbids extended discussions. The best online programming communities (e.g. gamedev, ltu, ompf, and the gold standard, flipcode (r.i.p.) ) are home to domain experts who enjoy discussing, debating and basically shooting the breeze with each other. This is what ultimately makes people stick around.

<comment><USER8> > And a community of domain experts will not last long if it explicitly > forbids extended discussions.

Totally agree, but this kind of interaction is explicitly what the SO team seems to be against, and for just SO itself, I agree. A legitimate justification for this is that people get into discussions which takes time away from input into the Q&A aspect of the site.

However, as you said, some problems or topics _require_ a discussion, not singular statements upvoted or downvoted. I think SO can do just fine without discussions, but the stackexchange platform will be sorely lacking if they don't support a discussion "mode", but from what Jeff Atwood has said, at least as I interpret it, he seems to see no need for this mode at all, and thinks the stackexchange engine _should_ eventually replace all phpBB boards out there. To me, this is insane, but this is how I interpret what he says, unless I've missed something.

</comment></comment><comment><USER13> No matter what they had done the luster would have worn off. The badge system just makes the whoring and the impossibility of ever attaining certain badges more obvious.

Every online community must go through a maturation process where excitement wanes and moderation waxes. At some point the curation of content becomes of primary importance.

Arguably they could have built a longer lasting community with Wikipedia style participation, but it certainly wouldn't have grown as fast, and maybe never would have gotten traction at all, so it's hard to make a retroactive critique.

In any case though, Stack Overflow still has a fundamentally a great user experience. It's not a growth business, but there will always be new tech, and I don't see the community drying up.

<comment><USER14> _No matter what they had done the luster would have worn off._

I'm not sure this is necessarily the case, because, as you write:

 _At some point the curation of content becomes of primary importance._

Which is why they need to find a way to build an incentive system that properly rewards "Wikipedia-style participation".

That's a tough nut to crack, but I certainly hope that they've got some folks working on it, now that they've got the VC money in place.

</comment></comment><comment><USER15> Right on the money.

I was there in the (relatively) early days where it was exciting to ask and answer questions and contribute to the site, while scoring points(!)

Nowadays I often encounter SO links while googling things, and it has certainly become a great resource in that sense. But I rarely have a need to ask questions there anymore as everything is covered, and trying to answer questions isn't worth the effort for similar reasons.

As new technologies emerge, I can see there being waves of increased activity as those topics are explored, and if you're one of those people on the forefront of said technology I could see it bringing back some of the original charm.

</comment><comment><USER16> My experience doesn't agree with the OP (though as others have noted, the plural of anecdote isn't data).

From the start in 2008 through now, my reputation curve has been a pretty straight line:

[http://stackoverflow.com/users/8912?tab=reputationhistory#ta...](http://stackoverflow.com/users/8912?tab=reputationhistory#tab- top)

I think it depends on the type of community that you participate in. If you work only in relatively unchanging technologies that everyone else works in (like .NET or Java), the OP is probably right. All of the nooks and crannies are exposed and answered. But if you participate in one of the smaller communities (Groovy, Scala, Clojure, Haskell, Io, etc).

I think the number of "niche communities" in the long tail are much larger and more vibrant than the OP supposes. Chances are that it's just the OPs pet technologies that don't get a lot of traffic.

I follow RSS feeds on a number of tags that I find interesting and answer a few questions every week.

</comment><comment><USER17> I have become a little discouraged at how stale some of the posts have become. I have run into Google searches that give me results where the accepted answer is now wrong. Sometimes the posts have devolved into multiple new answers that are correct but have very few upvotes. In some cases the newer answers are rants on how the original answer is wrong.

I do think there is still room for SO to grow with newer technology that comes out. SO became a great resource for iPhone development and I think that can still happen.

What would probably help is a purge of some sort. I'm not sure how they would go about it but I don't think just re-tagging will work I think they need to flush some of the stuff out.

<comment><USER14> I don't think a purge would get to the root of the problem, which is an incentive system that rewards adding early answers, rather than editing existing answers.

One of Joel's stated goals in the early days was that the answers would converge, Wikipedia-like, toward some Platonic ideal-- but the system in place doesn't encourage that.

<comment><USER18> Neither does Wikipedia, but at least Wikipedia doesn't have "cold" articles.

<comment><USER5> So you think that Wikipedia content gets worse or stagnant over time? Definitely, I think it's more effective than the SO model.

<comment><USER19> The WP pages on Chrome or Git only mention major features of those pieces of software. Features don't change that much. Part of the articles talk about the historic aspects of the projects, and again the past doesn't really change.

SO tends to go into minute detail on a particular aspect of something, so the staleness effect is amplified. For example, questions on how to write a Chrome extension tend to be stale, as do ones on specific features of Git such as integration with Subversion, or submodule support.

</comment></comment></comment></comment></comment><comment><USER20> Stack Overflow was a good stab at solving the problem, and the UI execution was excellent, as well as the promotion of the site by two widely-read bloggers.

My personal frustration with the site is that their engagement model was a little too game-like. While reputation and badges might be a good metric for the quality of the contribution, the actual implementation ended up reinforcing a base of easy homework questions, and people racing for the simplest answers.

I found that when asking descriptive questions, that I would quickly get several one-line answers that pretty much guessed at the answer based on the title. Even the nuanced answers tended to have answers and explanations that either ignored the problem or suggested things already asked in the question.

Sticking with the question, I can either comment on an answer, or edit my own question to address it, or I could comment a response to the comment. Working this way, I have managed to at least get good leads a few times, but the really difficult questions ended up being things I had to solve myself.

Lately, though, I find that it pops up in my preliminary googling of a problem early on, and at least provides better references than generic forums or experts-exchange.

If they could find a way to make reputation and badges reinforce the _quality_ of the answer, instead of the _response time_ of the answer, and encourage difficult questions rather than the proliferation of basic questions, and the attitude that answering the most of them the fastest made you some kind of "elite" programmer, then I think it could be very powerful.

That being said, when I am tracking down an obscure error or specific exception, I usually find it on the libraries forum, or a forum with a narrower focus, but they definitely like the UI polish of SO.

</comment><comment><USER21> My SO pet peeve was authors who answer nearly all questions in their field with "I discuss this in Chapter 5 of Volume 4 of my Everything You Need to Know About Everything series. In stores now."

<comment><USER22> And of course the flip side of that point is a lot of people's questions ARE already answered somewhere, often in books, if somebody just cared to get off their a-- and go look for it, spend time reading and investing in their own education. I've seen thousands of questions asked on the Internet where the asker could have answered it themselves by putting in a little more common sense effort, upfront.

The RTFM phenomenon.

<comment><USER8> True enough, but with the number of technologies a developer is "required" to know these days, I have a bit of trouble buying the RTFM argument (unless it is referring to freely available, easily searchable help docs on the vendor/author website). The the amount of technologies I use on a day to day basis, it's literally impossible for me to RTFM on all of them, __especially __if by RTFM you mean read a book on the technology (not to mention, which book should I read....figuring that out can take an hour. And is reading just one book sufficient?)

</comment></comment></comment><comment><USER23> _Games of code golf, "what's your favorite programming comic strip" threads, and "What language should I start learning now" threads made it way too easy to attain points without actually demonstrating real knowledge or expertise._

Actually, such threads are marked as 'community wiki' and there the reputation points aren't counted. The rest of the article is true, I also got tired of SO and mostly only google it.

</comment><comment><USER24> The thing that drove me out was uninteresting questions. I wanted an advanced search that filtered questions by minimum asker reputation, but the only operator available filters by _maximum_ asker reputation (i.e. to find newbies).

I then asked for the feature on meta.stackoverflow.com, and folks there perceived I was being elitist and downvoted me to oblivion. So I left.

<comment><USER8> > I wanted an advanced search

I agree. SO _has_ advanced search capabilities, but you have to be an uber power user and know all the esoteric switches, which I think is by design, so fanboys can be "leet". There is this page: <http://stackoverflow.com/search> (which isn't linked from the homepage), but why don't they take the 4 hours to build an advanced search form so people who aren't frickin experts on the SO platform can do advanced searches? I think this is by design.

>I then asked for the feature on meta.stackoverflow.com, and folks there perceived I was being elitist and downvoted me to oblivion. So I left.

Meta is not a particularly friendly place, and this seems to be the only place that Jeff takes guidance from, but I think it has to a certain degree become ineffective as the people that rule meta are super-uber SO users, and it it is a very homogeneous group of users, so genuinely new ideas basically can't enter. Maybe they _could_ , but after the initial rude reception you get with a new idea, most people just can't be arsed and leave, as you did.

</comment></comment><comment><USER25> I've used SO from time to time but have only participated when I came across a question I thought I could add value to with my perspective. I've never really kept up with the point system.

The author says "There's a plateau you hit there where you either have to devote way too much time to help other people do their work, or you just fall out of it." Do your points devalue over time somehow? Is there inflation in the user permissions you get with rep? Or is it just that you have to jump the shark to get the first answers in as they are far more valuable than later answers/editing answers?

<comment><AUTHOR> The plateau doesn't have to do with the points, but rather the content. If you're going to give an answer that's worthwhile, it would be nice to recognition for it. If you want recognition, you have to go after questions that haven't been answered. A lot of those are basically doing other people's work, and not really question-answering.

</comment></comment><comment><USER26> I worked for a for-pay Q&A site a little while back. They had all the incentives of SO and more. For example if you answered half-a-dozen questions a month, you got free access. The reward system. The asker declared how much the question was worth and awarded the points to one or more answerers. Awards could be adjusted by moderators so a better answer could be awarded the points.

There were a large number of categories, added to all the time. An answerer could register as an expert in one or more categories and could become an expert, guru, grand-master, etc. in one or more categories with the badges showing up everywhere.

There was an elaborate email system that kept every one engaged.If you were an expert, you got an email when a question was in one of your areas of expertise. An asker go an email whenever there was an answer of comment to one of your questions.

It appealed to the corporate IT world, but the combination of incentives, communications, and content curation seemed to work quiite well.

<comment><USER10> So why don't you do it any more?

<comment><USER8> What was the site? Surely not experts-exchange?

</comment></comment></comment><comment><USER27> Most people's questions about programming problems are going to be somewhere in the middle of the distribution. As time goes on and all the most common questions have been asked, the site is going become increasingly long tail - the dominion of obscurists - and any social/reputational component which it may have once had is going to be reduced.

</comment><comment><USER28> I find that Google is a great way to access the information on Stack Overflow, even as a participating member of the community. When I'm looking for an answer to a question, often a Google search will return the Stack Overflow post most relevant - helpful when there are many similar posts on a given topic.

<comment><USER29> Yes, it leads to better results than using the Stack Overflow search.

<comment><USER8> If SO was smart they would write an advanced search form that basically just translates into google compatible syntax and forwards it to google (assuming there are no legal issues with this.) Are there legal issues with this, is this a violation of the google TOS??

<comment><USER9> My gut feeling is that Google would probably be pretty ok if you just generate some (very specific) links to their search.

If you scrapped their results and embedded them into your page, that might be a different story. (They wouldn't like it.)</comment></comment></comment></comment>


Uber's Bundles - <AUTHOR> https://stratechery.com/2018/ubers-bundles/ <comment><AUTHOR> During rush hour, it is very inefficient for a one-tonne hulk of metal to take one person 10 blocks…We're able to shape behaviour in a way that's a win for the user. It's a win for the city. Short-term financially, maybe it's not a win for us, but strategically long term we think that is exactly where we want to head…

We are willing to trade off short-term per-unit economics for long-term higher engagement…I've found in my career that engagement over the long term wins wars and sometimes it's worth it to lose battles in order to win wars.</comment>


Ask HN: How to get a partner for a startup at a garage project phase? - <AUTHOR>

I work on a private project, a set of tests for polyglots. I plan to commercialize it in the future, but right now it's unclear when this happens. Right now it's a website with a minimum-viable thing almost working. I need someone to work together and give a competent feedback, but this is the most tricky problem for me. I'm turning 30 soon, in the past I had good and not so good projects done ALONE, but those made together have been rare.
I show this thing to my fellow language enthusiasts, they like it, but nobody jumps in to suggest something. I'm writing an email to one of them to ask her for an opinion from the user-perspective.
But it feels quite frustrating that it's so hard to make anyone suggest anything. This has been an issue few years ago when I led a discussion club. Social skills are a hard thing, though, I've read the descriptions of various disorders, but however bad I felt and however much I wanted to have such an excuse, they do not apply to me. Asperger for instance? No. I'm agile (do sports) and see social cues well enough.
Still it seems that I don't give others some kind of value. No matter how long I learn social skills. It's just almost impossible to get people interested unless our interests and intentions are identical.
I need a less technical person, better with design skills or one that would use it heavily.
Who has ever made a non-technical person join a web project?
What do people need to join a project? Do I have to praise their insights? (Well, few do say anything rather than 'cool'.) Offer them something? Ask for an opinion beforehand or pretend to show it spontaneously? Talking about the chances of commercializing it? I'm pretty sure of only one thing: they don't care of programming languages and of what's under the hood.
I'll appreciate if you share your experiences. <comment><USER2> maybe first start by criticizing it yourself to get the ball rolling? i find people are afraid to pick something apart out of politeness.

find opinionated people with a big mouth.

P.S. I'm opinionated with a big mouth but I'm not a hardcore programmer, little bit of ruby on rails, javascript, html, css</comment>


Unit Testing: Is There Really Any Debate Any Longer?  - <AUTHOR> http://www.drdobbs.com/testing/unit-testing-is-there-really-any-debate/240007176 <comment><USER2> At work, we're definitely turning away from traditional unit tests, which is _not_ to say we're turning away from testing.

Instead, for complex or algorithmic code we're using generative testing, barraging the code with random inputs instead of a few cherry-picked test cases.

Then, we're structuring our systems and operations in such a way that we can spin up test instances of the entire system and run full, end-to-end integration tests at various levels of load.

We haven't figured it all out yet, but the results are promising compared to unit tests, especially tests which contain a lot of mocks and stubs.

<comment><USER3> This is the approach that I take with my personal projects; I firmly believe quickcheck style testing is a huge improvement over manually writing out individual test cases. Do you tend to find, as I do, that when you write you code to be testable using randomized testing that it tends to look more functional? (I'm referring to the paradigm rather than the opposite of dysfunctional)

<comment><USER4> I find that the same quickcheck style testing is the fastest way to get out quality code for a project that I am the sole devleoper of.

I usually code in a terminal running screen with many shells going, one of which is running ipython so I can interactively import and test snippets, classes, or small functions that I'm writing in a larger module. It works great.

However - those tests disappear! Once a recent project of mine was handed off to a team for maintenance and more features, I found that, in particular places, where the code was subtle, it was subject to breaking by others who were just trying to fix something unrelated. This was when I, personally, realized the value of unit testing.

Yes, its slower and takes more time up front, but if your code may someday be worked on by others, it protects the sections of your code that have test coverage against breaking - which over a longer time scale can save time.

<comment><USER2> I believe the parent post was referring to QuickCheck (<http://en.wikipedia.org/wiki/QuickCheck>) as opposed to "quickly checking" code in the REPL.

Tests definitely need to be repeatable.

</comment></comment></comment></comment><comment><USER5> One thing to consider is that most code that gets written is not built to be highly testable. This makes testing harder to the point that "it's not worth it". If you optimize your coding style for testability, writing tests is faster and easier.

The problem is, writing testable code requires more thought up front not just about individual objects and methods, but also the overall architecture of your project. If your code is highly tied to an ORM or framework, then it's going to suck to test.

Also, not every language is equally easy to test. For example, Ruby has an obsessive culture around testing, so they have a lot of fantastic testing tools and the language itself makes things like mocking pretty easy. PHP by comparison doesn't value testing as much as Ruby does. The testing tools are worse and there are fewer of them. Also, due to the lack of monkey patching, you have to write your PHP code with dependency injection and testability in mind from day one for it to be highly testable.

In short, testing is a useful thing, but most code isn't written to support it and many languages and frameworks and patterns encourage you to write unpleasant to test code. If code is hard to test, programmers won't write tests unless their job requires it.

<comment><USER6> In short: by keeping testing in mind, you make your code better.

<comment><USER7> ...for testing.

<comment><USER8> In my experience, that clause is not required. Code that's been designed for easy testing is better code, period. Not just because it's been tested. Code designed for testing tends to have cleaner interfaces and is better modularized.

<comment><USER9> Unit testing can easily lead to _arbitrary_ modularity or code fracturing, which makes code _harder_ to use and reason about. Good code should be designed for solving problems, easy use and readability. If your code is designed primarily for passing code coverage checks, then it probably is not very good.

<comment><USER6> It does require a degree of latitude and good judgement. This only comes from experience.

</comment></comment></comment></comment></comment></comment><comment><USER10> I see unit testing (and especially TDD) as a barrier to agile development.

If I'm cranking out a prototype to show a customer then I don't want to spend precious time writing tests for code that stands a good chance of being thrown away at the start of the next iteration.

Tests often do need to be written, but the best time to do this seems like an unsolved problem to me.

<comment><USER8> In my experience, a prototype is never just a prototype. You crank out a prototype in an hour to demonstrate something. Then somebody says "add this and then we'll show it to the boss.". Your boss says "that's cool, add this and this and this and then we'll demo it to a potential customer". You do so, then play test the demo a few times, uncovering scenarios that need hardening. You demo to the customer, and they ask a few questions meaning that you add a few more features to the prototype for the next demo. Rinse and repeat.

After a few successful demos you're given an tight deadline to "finish things", so you have no choice but to polish up the prototype and ship it.

The moral: code that was written to be thrown away often isn't. You can still write code quickly, and take shortcuts (such as skipping tests), but recognize that you'll probably be working on that code for a very long time and may regret your decisions later.

<comment><USER10> This is what I was alluding to when I mentioned that it's not obvious when tests should be written. It simply isn't practical to write them as part of a quick prototype (or any other rapid iteration). But writing them at the end of a project may also be difficult due to the need to move on to something else. The same argument holds for things like documentation...

</comment></comment></comment><comment><USER11> I'm sitting here significantly refactoring a HTML-generating class hierarchy that I made a couple of foundational mistakes on a couple of years ago, causing excessively slow rendering. It has about 100 classes, and is used in over 1000 screens. Most of them trivial, but the non-trivial ones are definitely non-trivial. Fortunately, it happens that there is a very clean input and output that is, by design and by network structure, isolated, making it feasible to test. I've got decent automated test coverage on the existing set, and I'm in the process of adding an automated test that will verify that both the old and the new system emit byte-for-byte identical output. Without this test, this would be a hopeless exercise and I'd have to just live with the outcome forever.

I find myself wondering how many people complaining about unit tests work on leaf-node sorts of programming projects, because for me, many of the tasks I routinely take on in my environment where about a dozen projects are building on the code I am writing mean that the question isn't whether unit tests are "better" or "worse". It's a question of whether the task is _possible_ or not.

</comment><comment><USER12> Unit testing is a very very expensive operation. Consider the following: The developers add unit testing at beginning for specific results. Later on, after product is released a new design or feature change happens. Often times unit tests will have to be rewritten else they will fail.

This can be very expensive and take developers and QA engineers extra time when they try and figure out what's wrong... "Is the test failing because the test is out of date? Or is our program causing a bug?" I know for a fact HP deals with this issue constantly. Not saying unit testing is bad (I think it's usually good) but I don't think it's always good.

<comment><USER13> > "Is the test failing because the test is out of date? Or is our program > causing a bug?"

IMO, when you change code (extensively), you'd still have to invariably answer those questions. But in the absence of tests, it becomes much harder to find the answers.

<comment><USER14> Indeed. The tests tell you that you are breaking a contract that you once thought was important. It might not be any more, but good tests reflect good contracts. Removing the test requires you to think whether any other code expects this contract.

</comment></comment></comment><comment><USER15> That debate exists because the people selling the idea of unit-tests were so goddamn gung-ho for unit testing everywhere and all the time.

I have friends in the medical industry, their tests _must_ achieve 100% code coverage on multiple coverage metrics, and code coverage is only an afterthought to even more rigorous test requirements. Even they were pointing and laughing at the TDD ideologues.

Unit testing is beneficial, if you do the right amount of it and in the right places at the right time. You might have a totally sensible test plan. But like it or not, for the time being, plenty of people are unwilling to listen to you because some other assholes already burned them on the idea of unit testing.

Its going to take years and years to shake loose all the cobwebs in our industry caused by half-cocked advocacy. Then we can maybe get to the sensible discussion about how to analyze a project and develop a sensible test strategy that fits the project's needs.

</comment><comment><USER16> 1\. Unit Testing is useful ONLY for complex API method/function, which should not change too often and too frequently;

2\. Ideally, you should only test API not API's implementation but most of the mock framework will tie your test code directly against your API implementation. Once your API implementation changed, your test code fails, you have to rewrite test code, which is waste of time.

Someone may argue that it because you write bad test code; Talking is easier than doing;

So, usually, I prefer to doing the functional integration test than the unit test, which is productive for me. (I know people will throw out thousands of reasons about the benefit of unit test and "correct" way of writing unit test.)

Anyway, I will avoid unit test as possible as I can and ONLY do my designed integration test, which works for me.

3\. And writing unit test for code coverage is stupid; Some manager even brag about the unit test coverage, which is totally a false confidence about the quality of the code underline; This is even stupid;

4\. Writing unit test for those requirement-always-change area is waste of time;

5\. TDD is a BIGGEST JOKE, which weights too much on writing auto testable code over writing working functional productive code;

</comment><comment><USER17> Purposeless unit tests are a waste of time. These are tests against your compiler or external frameworks. For example:

        @Test
        testAddUser() {assertNotNull(new User("columbo").save())}

Wahoo! the ORM works!

The biggest problem I have with unit tests is they tend to be

a) just testing frameworks (crud) and not much else

b) so heavily built on mocking that changing them is like redesigning a wool sweater. These tests tend to examine how the code was written, not necessarily what the expected output is (oh shoot, I need to mock up five more objects to make this test pass... hey look at that, this test doesn't even check what should be returned. Nice)

c) there is no context into or out of the unit test. The assertions are vague, it isn't clear how I got there, it isn't clear where I'm going (testXMLOutWithInProgressUsers ... what is the xml for, who are the in progress users... what is next in this chain of events)

d) the things that REALLY need unit tests don't have them because they are too complex to write (ah, that whole system that runs a nightly bash script, takes in a stack of csv files from the client and converts it in our database... yeah, that's not tested).

With all that said, tests are invaluable, but they need to actually, you know, TEST something.

I like nodejs in that respect. It takes me seconds to fire up the webserver and test the rest api. I don't mock objects, I pass values directly through the api and check the expected results. I like this because it shows the next user what the api actually looks like and I'm testing the full stack (In my experience controllers are notoriously undertested, mainly because you need the full stack running). Since the application is really just an api this means that I'm able to test everything equally, from batch processes to CRUD.

<comment><USER18> Agreed.

API testing has that wonderful property, its appearance is the whole deal, if it looks right then its working.

Anything but app code isn't so easily tested. When frameworks are written to support testing, they often are sub-useful - crippled versions of real programming languages etc.

Anecdote: I recall a QA guy asking me how to test our OS's file system. I said, make every filespec-based call with a good spec, a non-existant spec, an illegal spec, a null spec; log every result. After each release, diff the logs and explain every deviation.

He said "That's too hard, I'll just shotgun it." and left. Why ? Because his test-scripting framework wouldn't allow anything as sophisticated as lambdas etc.

So I spent half an hour writing a table and a loop, and sent it to him. He used it for the next 5 years to test every release.

My moral: do an exhaustive test when possible. The computer is fast, it doesn't feel any pain, and you will find more fencepost and boundary issues than you ever thought existed.

</comment></comment><comment><USER19> A rare Anti-Betteridge. <http://en.wikipedia.org/wiki/Betteridges_Law_of_Headlines>

</comment><comment><USER20> In practice, I find I write code faster and with more confidence after the tests (usually specs) have been written. So, my productivity increases. I also sleep better at night.

<comment><USER21> For me TDD is doing the exact opposite. It bores me to tears, which _really_ brings down my edge.

Granted, this is one of the reasons why I'm recently thinking about whether I've got a slight case of ADHD, but I think a lot of it just stems from the fact that I like to brainstorm code and _afterwards_ use something like tests or just good documentation to reason whether that was the most proper approach.

Never mind that things like cucumber make me cringe big time.

<comment><USER22> > _For me TDD is doing the exact opposite. It bores me to tears, which really > brings down my edge._

That.

It takes all the fun out of using a dynamic language. You might as well go back to a static typing and write LESS tests for obvious things...

<comment><USER23> Yes, this is an obvious downside to the type of TDD that you sometimes see advocated, where they write tests for every little thing.

I try to find a compromise solution. I pretty much only write one unit test for a given module to start with, the nominal case. This is important, because it forces you to think a little bit about how you actually want your code to work, which I find to be a rather healthy thing to do.

Once that is done, a little later I will no doubt find a bug. When I'm trying to debug, and I start wondering whether a block of code is doing what I think it is doing, I write a test to verify. This does one of two things - it shows up the bug, which I can then correct, _and verify that I have corrected_ , or it allows me to remove the tested code as a possible cause for the bug, allowing me to concentrate on other hypotheses for the cause of the bug, speeding up the debug process.

By the time I get to the end of the project, I generally have a pretty comprehensive test suite, without ever once feeling like it was a chore.

</comment></comment></comment></comment><comment><USER24> Unit testing, like many other good and healthy coding practises is great. In theory.

In the big mean real world frequently you're shipping things in such narrow deadlines that if you say "Hey boss, I'm going to take some time to write some wicked cool unit tests, it'll make my life easier in future because we'll know this code works!" then Boss' head will pop as it's a non-tangible improvement.

Developer focused companies are great as they'll let you get away with this, but the large portion aren't.

</comment><comment><USER25> Unit Test, sure, but at what "unit level"?

I usually prefer "automatized system testing" where fewer "high level" tests involve a lot of code, preferably in a test environment that is close the actual production environment.

Of course, when a test fails, some more analysis is required to determine what went wrong. But it's usually related to recently modified code, hence easy to spot.

That way I get the benefits of automatized tests without the burden of tons of "micro" tests at the class/method level.

</comment><comment><USER26> There's no debate about testing stateless functions. I think the debate is in whether the ledger reflects a cost benefit of spending all the time mocking up different functionalities in order to test stateful components or components with external dependencies. Since these are hard to calculate ahead of time, it really does become a question of engineering culture and time to market.

</comment><comment><USER27> why do i get the feeling that you probably haven't spent much time "debugging" unit-tests...

unit testing give you the false confidence that your stuff is working, it has little to no bearing on stuff actually working when pushed to production (unless you have 100% coverage and actually thought of and defended againt every possible use scenario before hand)

<comment><USER28> Yes, unit tests are code and all code can have bugs, but you're missing the point. Automated tests are a reflection on the code just like code reviews or documentation or formal specifications. If they are well written (ie. orthogonally structured to the code they are testing and with good readability) then they provide a strong window into the intent of the original developer, _and this intent can be continually tested for almost no marginal cost for the lifetime of the code_.

Yes there is a maintenance cost, but I find most people's objection to testing is based on poor testing skills. Once you get good at testing with a decent framework, you find that it's often about the same time to write formal tests as it is to do the manual testing which we all know is necessary to have any certainty that code is working.

</comment></comment><comment><USER29> Unit tests are good. System wide functional tests are awesome.

</comment><comment><USER30> With CI, it's become a little harder to make the excuse. :)

</comment><comment><USER31> Of course there is debate.

The reason is that automated tests costs 2-3x as much time upfront. The assumption is then you will recoup that investment in the future. The problem is that automated testing doesn't cover everything so you have to augment it with human testers. Which then begs the question are the human testers being effectively utilised.

And for those of that do develop have a think about all the time spent on buggy web UI testing, screenshot comparison testing, tests being flakey due to software choices etc.

<comment><USER32> Automated _unit_ tests cost two or three times as much time, sure. That doesn't mean don't use automated tests, it means don't use unit tests. Use integration tests instead. Not only do they provide much better coverage (you're testing all the code from top to bottom, each module in the context in which it is actually used), but they are also far cheaper.

</comment></comment><comment><USER33> Yes come on, does anyone still think they are useful and simply not a complete waste of developer time?

<comment><USER34> I can't decide if you're serious and misguided or just joking.

<comment><USER22> You know, actual luminaries in our field are also against them. I've seen UNIX programming gods among men dismiss them. It's not like it's a clear cut issue.

For some things like libraries and such, there are absolutely needed, but unit testing everything and especially TDD are stupid.

To be frank, they mostly sound like some Java EE-like monstrosity that for some bizarro reason caught on with the hip crowd too. I'd put them out there with FactoryOfFactory and XML documentation...

<comment><USER35> When I think about UNIX programming gods, I think about people who deal with tools that are never modified, deal with very small problems. They're trivial to test functionally, but are next to impossible to test internally (have you ever seen source for tools like tar?)

It's definitely some solution for testing. But when you're writing a webservice, you should think twice before assuming that anything a "UNIX programming god" says is a good advice for you.

</comment></comment></comment></comment><comment><USER36> Ah, the old 'Argument by Dismissal'. Still just as bold, and still just as unconvincing.</comment>


AWS is down (Oregon) - <AUTHOR> https://status.aws.amazon.com/?date=today <comment><USER2> It's not down for existing instances, but definitely lots of API requests are extremely slow or failing. See the AWS status page[1] for slowly updated information.

Terraforming in Oregon is painful, my plan has been refreshing resources for over 30 minutes.

[1] - [https://status.aws.amazon.com/](https://status.aws.amazon.com/)

<comment><USER3> Yeah looks like a cascading failure though.

<comment><USER4> I can't tell if that's a pun or not...

(well played!)

</comment></comment></comment><comment><USER5> Yikes. Sea of red on some dashboards here.

<comment><USER3> cant migrate kube cluster cause no ec2 instances :( q.q.

<comment><USER5> It's pretty standard that when the control plane is impacted, provisioning times are… lengthy. :-/</comment></comment></comment>


It's official: Match.com Abandons Paid Dating…  - <AUTHOR> http://plentyoffish.wordpress.com/2009/01/03/its-official-matchcom-abandons-paid-dating/

<comment><USER2> I have a soft spot for match.com. I met my GF there 6 years ago. She was certainly worth the $25.

<comment><USER3> i have paid for match (and had a few dates via it) in the past. i have also used POF and one huge difference between them - people on match care to put up a bunch of pictures and actually fill out the damn profile. POF users rarely fill out profiles in my exp. What I'm trying to say is that if users are paying for the service they are very motivated to go ahead and actually use the service.

</comment></comment><comment><USER4> Signing up now. So far the interface is clean and easy to use. I have no clue if there are any hot girls on here yet though. Will report back with my initial findings.

I'm curious to see how this will stack up against OK Cupid, where I met little but cautious hipster emo girls whose photos came from webcams and off bathroom mirrors.

Match.com, on the other hand, is all blonde young professionals dressed in black. Their photos all have meat head guys cropped out of them, with just enough arm left in there to show you that they can get a guy like that.

<comment><USER4> Update: Is that a photo of Jason Fried on their upload directions page?

<http://i18.photobucket.com/albums/b124/bflora/fried.gif>

<comment><USER5> Jason Fried: welcome

Frank Zappa: not welcome

</comment></comment></comment><comment><USER6> No doubt this is a market segmentation play (or maybe a lead generation strategy), not really a new business model. Match.com also created chemistry.com to compete with eharmony.

</comment><comment><USER7> This is a pretty tough new reality for online companies. Dating was one of the few services that consumers were willing to pay for. Does this mean that ads are the only revenue source for consumer-based sites?

<comment><USER8> Nope. Selling your data & lead gen is a pretty solid source of revenue, although some people won't like that.

<comment><USER9> If the largest site is free, who are the leads being sold to?

<comment><USER8> Venture Capital backed startups.

<comment><USER10> Believe it or not, that was a big part of Yahoo's business model in the 90s.

</comment></comment></comment></comment></comment><comment><USER11> Will this be the downfall of plenty of fish? Most of his adsense income comes simply from other (paid) dating sites.

So it's kind of ironic that he helped create the very situation which may one day lead to the site's downfall.

</comment><comment><USER12> without paid dating sites what ads will plentyoffish show?

<comment><USER13> It will never happen. If there is no paid dating site, plentyoffish will become one.

</comment></comment><comment><USER14> I think markus may be wrong. If you look at the partners page it just seems like downtoearth.com is just a lead gen site for all other dating sites...

</comment><comment><USER15> it seems like a lot of paid services are developing free alternatives.

<comment><USER8> It's the same thing with most established businesses. Look at Proctor & Gamble, they have like 10 different brands of laundry detergents all targeted to different markets.

</comment></comment><comment><USER16> The way to monetize users should somehow be PERFORMANCE BASED rather than purely subscription based. Yes, subscription allows for great filtering of people..and yet, maybe it feels too costly at first (because the whole value/match is delayed to much later?). There must be a way to get money out of the HUGE satisfaction of finding the girl/guy when this happens - and this could complement (if not remove) the need for an entry cost to accessing the pool. For example, dance clubs make most money out of drinks (fun related) not from entrance fees (access to the pool of people). Maybe pay in order to go on a 2nd date? Pay every time you click on a profile (point system)? Pay a bonus when you find a match (how to prove...)? Do you know if this kind of stuff was ever tried or what do you think?

<comment><USER17> > pay in order to go on a 2nd date?

You haven't really thought this through have you?

<comment><USER18> That pretty much is the business model of "speed dating", no? If the speed dating event itself counts as a first date.

<comment><USER17> It doesn't, meeting someone anonymous for a few minutes isn't a real date. Once you've been on a real first date, you've already exchanged info if you plan on a second; no one in their right mind would pay someone after that point.</comment></comment></comment></comment>


Ask HN: If I bootstrap, does my current company own the IP for my work ? - <AUTHOR>

I have an idea and want to develop a prototype for it on weekends/nights. However, I was wondering if my current company would own the IP and if that could cause problems later on ?<p>Any advice on bootstrapping, things I should look out for before starting this ? It would be great to hear some anecdotes about bootstrapping as well. <comment><USER2> This is a gray area. If your personal project overlaps in any way with current or potential future projects that your employer might undertake, be very, very careful.

I presented one such idea to an employer I worked for, and they actually created a "quitclaim deed" saying that they weren't interested in taking on the liability, and that provided I didn't use company time or facilities they didn't expect a cut of the profits or IP either. But most employers aren't that cool.

Many employment contracts have a clause saying that the company you work for owns any ideas you come up with related to your work while you're employed there. Make sure you take a look at your contract to see if it has such a clause.

The only situation I could think of where you could probably let it slide is if your project is totally unrelated to what your company does. For example you program DSPs for a living and your side project is a social networking app.

As for anecdotes, I haven't worked for that employer for ten years, but at the same time my side project is still in its infancy. Fortunately the technology has advanced in the intervening years to the point that it is actually practical, so hopefully I'll have a prototype done soon.

<comment><AUTHOR> Thanks for the advice. I did sign a contract but I am not sure if the scope is limited to my work related to what I do at the company.

My current company is pretty vast and it's not unimaginable that they could come up with a similar product but it's highly unlikely .. even more unlikely in the coming years.

The other thing is that I don't know if the project will take off and I don't wanna discuss it with people at my current company yet.

</comment></comment><comment><USER3> It all depends on what agreement you signed with your current employer. (If you didn't sign anything, which is unlikely, the IP is all yours.)

<comment><AUTHOR> Thanks pg. I guess it all depends on the fine print of the contract I signed.

</comment></comment><comment><USER4> hey, i was in your situation (actually i had started a lot of work on my website before i started panicking about it)

i actually made friends with the company lawyer and asked her informally about it first. i suggest you can try first. if the work contract. then it's not unreasonable to ask for a letter from the company saying its ok... as long as you're doing a good job at work i guess it's fine (people may even see you as with a lot of initiative to be starting your own idea outside of work)

good luck...

</comment><comment><USER5> Have you signed any such documents?

<comment><AUTHOR> I signed a document about any "inventions" I do being the property of the current company. Would the new company / project count as an invention ?

<comment><USER5> I am not a lawyer, but my understanding is that yes it would count as an invention. You need to address this with your employer before proceeding any further.

Now you're in touchy territory, you're telling your bosses that you have a side project which may one day take you away from them, but if you're going to do this and maintain control it's a risk you have to take.

Otherwise when you leave, if they know about it, they can take you to court for this project.

If you go in to talk to them, assure them that it is a side project and that you won't work on it during work hours or using work equipment (not even a company lent laptop is safe.) If the project is still neo-natal then they should be cool with it and have no interest in it unless the idea is that good.

If you think the company might take it from you and you don't want that. You have to work on it in secret, never tell the coworkers, etc. Then leave the company before it becomes known, and the term expires on your contract document.

<comment><AUTHOR> That was the only thing I was worried about. Even if I don't tell anybody and work on it in secret, after I leave the company and say release this product, could they come after me and say : hey, you signed a contract .. this product belongs to us since you worked on it when you were with the company ?

I guess I am getting a little paranoid about this but I just wanted to understand if anybody had any good / bad experiences and the best way to go about it. Thanks for the advice

<comment><USER5> Depending on the terms of the contract, yes.

The good: They have to be able to prove you worked on it while working for them.

The bad: You'd still get saddled with legal fees if they come after you, even if you win the case and retain the project.</comment></comment></comment></comment></comment>