Why the Clean Tech Boom Went Bust - <AUTHOR> http://www.wired.com/magazine/2012/01/ff_solyndra/ <comment><USER2> Peter Thiel had a good analogy in his talk at Singularity Summit: <http://www.youtube.com/watch?v=ROrUea0gLlY#t=34m0s>

_As with technology, you have to be able to do more with less. Alternative energy has to be cheaper. And until it is cheaper, it's going to be very hard to get it to work. If you had Amazon as a computer company- if you had said in '96, "Yeah it's gonna cost twice as much to buy a book and it'll take you 6 months to get it, but we're going to get subsidies that are really big and that will make the business work and that's why you should invest." That would be quite difficult to work._

Cleaner energy sources will take off eventually, but not until they're economically competitive with coal.

<comment><USER3> Actually, competing with coal is no longer enough. They have to be able to compete with natural gas, which is getting very cheap.

Coal plants are already failing this test: [http://www.theithacajournal.com/article/20120103/NEWS01/2010...](http://www.theithacajournal.com/article/20120103/NEWS01/201030354/Cayuga- Lake-power-plant-owner-declares- bankruptcy?odyssey=tab%7Ctopnews%7Ctext%7CFRONTPAGE)

</comment></comment><comment><USER4> We've had clean, green, safe, carbon-neutral energy for nearly 60 years. Too bad people have been misled into being afraid of it.

It's called nuclear power.

<comment><USER5> There are two calls on hold for you, one from a place called Fukushima and the other Chernobyl. ;)

<comment><USER4> Fukushima: despite a massive earthquake, tsunami, fire, and loss of power, no one died. Yes, the plant was trashed, but just how disaster-proof do you expect something to be?

Chernobyl: Horrible Soviet-era reactor design that was never approved in the West.

No member of the general public or plant worker has ever been killed by a Western-designed power reactor. That's a better safety record than _any other source of power_. We've got 60 years of experience. There have been three major accidents, two of which produced no loss of life whatsoever.

<comment><USER6> _There have been three major accidents, two of which produced no loss of life whatsoever._

This is an overstatement. A crane at Fukushima fell down killing a few people, and a couple of workers died of heatstroke (due in part to radiation suits).

As for Chernobyl, you forgot to mention that in addition to design flaws, they were performing a scientific experiment well outside the safety parameters at the time of the disaster.

<comment><USER4> I would chalk those up as industrial accidents of the type that can occur anywhere, rather than nuclear accidents per se. Other energy technologies kill people directly. Natural gas explodes. Dams break. Coal mines cave in, catch on fire, and kill miners with toxic gases.

</comment></comment></comment></comment></comment><comment><USER7> I've been a huge proponent of "green" energy for years, and after watching it for those years, it's becoming more and more obvious the technology just isn't there yet. The best we can do is pump more money into research and ignore these poorly planned start ups that are only profitable due to subsidies. Every major type of renewable energy has some aspect which makes it at least a decade and several breakthroughs away from viability.

Cellulosic/Algae Ethanol: About an order of magnitude off on cost, many orders away on capacity. They are still immature technologies but seem to have a lot of promise.

Photovoltaic Solar: Cost is almost there, but no one has a plan for what to do when the sun isn't shining. Solar systems can only produce energy for about 6 hours a day, which presents a problem when we need energy for 24 hours. It doesn't produce energy during peak demand (~5-9pm) to even help with load balancing. People can currently make a small profit selling back to the grid, but there's virtually no chance of it providing more than about 5% of demand.

Solar Thermal Collectors: Better than photovoltaic as they can generally produce power for about 8 hours, then molten salt batteries can provide energy for another couple; but still not a viable large scale solution.

Wind: Geographically limited, very high maintenance costs.

Hydrogen: Only an energy storage medium, not a means to produce energy. In most cases there are more efficient ways to use the energy directly without the additional loss of producing hydrogen.

Corn Ethanol: Energy storage medium, and a bad one. The most optimistic numbers I've found indicate it requires about .84 gallons of gasoline to produce 1 gallon of ethanol. Most numbers are significantly worse, in the 1.2-1.4 range. It sounds great if the lower numbers are accurate, but 1 gallon of ethanol contains ~2/3 the energy of 1 gallon of gasoline so it's still a net loss. (note: I haven't done any serious research on sugarcane ethanol, which at first glance appears somewhat better)

Hydroelectic: Great technology that is proven, cost effective, and viable; but is severely limited geographically and the dams can cause major environmental concerns.

After years of following the news, studying it, and taking classes in renewable energy; it always comes back to nuclear. If it wasn't for legislative bungling, lawsuits preventing new plants, and the occasional (comparatively) minor failure of 40 year old technology we wouldn't even be talking about a clean energy problem. I'd love for one of these technologies to prove me wrong but there is little indication they will in the near future.

<comment><USER3> Re solar PV, the problem of the sun not shining is just a bigger-scale variant of load balancing, which has been addressed successfully before. The English grid does it with pumped hydro storage in this mountain in Wales: <http://en.wikipedia.org/wiki/Dinorwig_Power_Station>

<comment><USER8> You mean 'the British grid', since if the hydro plant is in Wales it is by definition not English.

This is kinda like referring to a project in Texas as Mexican.

<comment><USER9> More like referring to the USSR as "Russia."

</comment></comment></comment></comment><comment><USER10> "Clean Tech" is a big scam...politically connected VC like Doerr ripping off U.S. taxpayers to fund crony capitalist boondoggles like Solyndra and Fisker. The VC get all the upside, and taxpayers have all the downside.

If the elites really cared about solving climate change, they would advance honest, technology-neutral policy solutions like a simple tax on carbon production. Instead, from battery cars to ethanol, Green Tech has been an excuse for a multi-billion dollar money grab.

<comment><USER11> WRONG! Clean Tech and Green Tech are not scams, they are investments in our future. Just like any other technology the US has invested in. Like any investment some are good and some go south.

If you want to complain about our government "wasting" money and trillion dollar money grabs look no further then the Department of Defense: [http://www.nytimes.com/imagepages/2011/03/13/opinion/13opcha...](http://www.nytimes.com/imagepages/2011/03/13/opinion/13opchartimg.html?pagewanted=all) (note the numbers in this graph are in billions, not millions)

Solyndra was a rounding error in what the US Government wastes on defense.

VCs are like the government, they only win when companies win. Just like the government VCs loose many, but win big on a few. Government wins when the companies they "invest" in create American jobs. I am not even going to go into you putting "simple" and "tax" next to each other in the same sentence.

<comment><USER10> Nearly every single American green energy program, from the 1970s to today, has failed, or is in the process of failing. Some, like massive ethanol tax subsidies and mandates, have significantly damaged the environment. Battery cars are a passing fad that are also bad for the environment-- they rely on coal-fired plants for electricity, and much of that power is wasted in transmission. And the batteries are hazardous.

The fact that DoD is a bigger waste of money is not a defense of Clean Tech.

The real solution is getting the big incentives correct-- tax pollution, and get the government out of the business of picking technology and playing venture capitalist.

<comment><USER5> > Nearly every single American green energy program, from the 1970s to today, > has failed, or is in the process of failing.

Solar is booming and doing much better than in the 1970's, and there have been many breakthroughs combined to helping getting the cost per watt way down.

> And the batteries are hazardous.

You do know gas and oil are hazardous and toxic right? You do know that the extraction, processing and consumption of oil and it's byproducts are putting very large amounts of bad stuff into the air we breathe and water we drink, right? You did hear about the various huge oil spills, including the BP Gulf disaster, Nigeria, etc?

The whole point of government, one of many, is is to act as a neutral guardian for society's shared resources and common benefits. Individuals and businesses are relatively free to pursue their own interests, but only so far as it does not harm others (ideally), or bring about Tragedy of the Commons kinds of situations. Thus, some regulation, and some punishment/reward mechanisms are in place to help maintain or bring about the kind of world we want to live in. This should be pretty clear, and has no malevolent intent.

I do find it funny that some folks never complained that Big Oil was getting subsidies, incentives, breaks, special treatment, etc. but when it comes to solar, wind, anything intended to be a cleaner or renewable energy source, suddenly there are complaints that government is distorting the market place or picking winners. The difference, clearly, is which technology or industry one would personally rather see favored. Put me down for favoring energy sources that are renewable or better for health. This should not be some kind of radical position, because the benefits are pretty clear.

We agree that taxing pollution is probably a good thing for government to be doing. But the oil/coal industries world-wide are fighting such things, and they spend lots of money on Congress. And the Republican Party, in particular, does their bidding almost every time a relevant bill comes along.

<comment><USER4> Look into the environmental impact of producing solar cells. Semiconductor fab chemicals are nasty stuff, and when you think about how many square miles would be needed for solar to make any significant impact...

</comment></comment></comment></comment></comment><comment><USER12> The guys at The Automatic Earth[1] explain that after the multi-decade credit run up enabled by cheap oil would come when peak oil is reached. Once there the financial system is not able to sustain itself without cheap credit it will collapse and a sustained credit deflation will ensue. In this deflation all assets will decrease in price but the difficulty to obtain cash will be harder making them effectively very expensive.

This, they explain, will have the effect of making oil cheap to the point that there won't be enough money to invest in maintaining the current infrastructure and even less to invest in green tech. Once the bad debts are cleared and growth returns the true effects of peak oil will be felt everywhere.

In summary, they say, don't get your hopes up with clean/green tech. It is too little too late and it simply cannot replace oil. And nuclear energy is no better once you take into account the energy required to build, operate and decommission reactors and also dispose of radioactive waste in a safe manner.

They have in my opinion one of the best analysis available in the intertubes.

[1] <http://theautomaticearth.blogspot.com></comment>


The dolphin who loved me: the Nasa-funded project that went wrong (2014) - <AUTHOR> https://www.theguardian.com/environment/2014/jun/08/the-dolphin-who-loved-me <comment><USER2> This is such an out there article, I'm not surprised at the lack of comments. I suspect very few want to comment and possibly be associated with zoophila. It did remind me of a site I stumbled upon in the 90's possibly later describing in a fairly clinical manner how to have sex with a dolphin. It is not graphic at all, but definately NSFW. Not Safe For Work! [https://web.archive.org/web/20040131140419/http://www.dolphi...](https://web.archive.org/web/20040131140419/http://www.dolphinsex.org:80/) I'm not attracted to dolphins, but my purient interests found the site interesting and amusing.

<comment><USER3> I recall when my college roommate showed that site to the rest of the apartment. The combination of hilarity and incredulity was intense.

</comment></comment><comment><USER4> I had read about these stories separately, the consensual, interspecies sex and the dolphin suicide, but for some reason I failed to realized or had forgotten it was the same dolphin.

I admit I sort of fail to understand the outrage about the sex thing and about the attempt to teach it English. I'm probably naïve but I still think some basic form of communication should be possible, like with great apes.

<comment><USER5> I think the mindset of the 60's was way different. Today nobody blinks twice when a horse needs to be manually gotten-off to ship off the semen to other parts of the word.

<comment><USER6> > _Today nobody blinks twice when a horse needs to be manually gotten-off to > ship off the semen to other parts of the word._

That's perhaps a city thing. In farms people did similar things (to cows etc) all the time as part of routine job.

<comment><USER7> Yup. This is literally where milk comes from.

<comment><USER8> I'm relatively certain that's not where any milk I've ever had came from...

<comment><USER9> Heifers don't produce milk until after calving. It's a whole lot easier managing artificial insemination than a live bull.

So yes, that is where the milk comes from.

<comment><USER8> You missed the point...

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Here are the remains of Dolphin House,

[https://zoom.earth/#18.318131,-64.859424,19z,sat](https://zoom.earth/#18.318131,-64.859424,19z,sat)

We used to sail past it in the 1970s, and the buildings were abandoned but had roofs then. Back in 2011 there were plans to develop the area but I guess they fell through. Back then the risque part of the story was well known, though the LSD experiments were not. They had it all backwards, the LSD should have generated an outcry instead. Mammals are similar enough that the scent of female human would surely provoke such a response.

<comment><USER11> _Mammals are similar enough that the scent of female human would surely provoke such a response._

This couldn't possibly be correct. Female humans don't need to worry about e.g. amorous stallions, bulls, or even male chimpanzees. The fact that sexual relations of some sort have occasionally taken place between different species doesn't make anything about it "sure". If it were, mammals would waste a lot of time and energy on behavior that does not contribute to genetic fitness. Besides dolphins don't have olfactory nerves.

<comment><USER12> Weirdly when working in the Peruvian rainforest we did have to warn women about an amorous tapir. He would become very boisterous and would also come into camp and take knickers off the washing line.

</comment></comment></comment><comment><USER13> I very highly recommend reading up on Dr. John Lilly, easily one of the most interesting scientists during that era. Along with interspecies communication, he heavily researched psychedelic drug experiences and combining them with sensory deprivation tanks. He became heavily addicted to ketamine (reports say that he would come to work every day with a spray bottle full of ketamine, and noisily snort it throughout the day), and eventually became convinced of the existence of a cosmic organization called the Earth Coincidence Control Office (E.C.C.O.), which directed the actions of individuals on Earth. His story is also the inspiration for the video game Ecco the Dolphin.

</comment><comment><USER14> Side-note, this project was one of the inspirations behind the cult-classic Ecco the Dolphin game for the Sega Genesis

This video explains it very well: [https://www.youtube.com/watch?v=0xUvhUK8Dv8](https://www.youtube.com/watch?v=0xUvhUK8Dv8)

<comment><USER15> Relevant Vice article: [https://www.vice.com/en_uk/article/exmjpz/the- ketamine-secre...](https://www.vice.com/en_uk/article/exmjpz/the-ketamine- secrets-of-segas-ecco-the-dolphin-347)

</comment></comment><comment><USER16> Here's the Radiolab podcast about it (2015)

[https://www.wnycstudios.org/story/home-where-your-dolphin- is...](https://www.wnycstudios.org/story/home-where-your-dolphin-is_podmash)

</comment><comment><USER17> The Dollop podcast did a hilarious episode on this.

[https://thedollop.net/wp/episode-8-dollop/](https://thedollop.net/wp/episode-8-dollop/)

</comment><comment><USER15> There's a really good documentary about this: [https://www.imdb.com/title/tt3822614/](https://www.imdb.com/title/tt3822614/)

The BBC showed it a few years ago, it's fascinating: [https://www.bbc.co.uk/programmes/b046w2n8](https://www.bbc.co.uk/programmes/b046w2n8)

<comment><USER18> I heard an interview with the researcher on the radio while I was driving to work a few weeks ago, and it floored me. I then thought about the movie with George C. Scott, 'Day of the Dolphin', which my Dad took me to see when I was nine. I cried at the ending (no spoiler here, see the movie!).

</comment></comment><comment><USER19> Thevet, thank you for the quality of your submissions. I really enjoy reading about these little-known pieces of history.

</comment><comment><USER20> This story pops up from time to time. There's an old Rooster Teeth video about this: [https://www.youtube.com/watch?v=prmx0lYBTko](https://www.youtube.com/watch?v=prmx0lYBTko)

</comment><comment><USER21> Context for some of what's in 'the day of the dolphin' by Robert Merle which I now understand is fictionalized from the Lilly story.

Szilards 'the voice of the dolphins' story is much more a political work post bomb physiscs career.

</comment><comment><USER22> SNL did a pretty funny skit semi-based on this: [https://www.youtube.com/watch?v=-zq5kWvp528](https://www.youtube.com/watch?v=-zq5kWvp528)

</comment><comment><USER23> Poor Peter</comment>


Russian Agents Are Not Behind Every Piece of Fake News You See - <AUTHOR> http://fortune.com/2016/11/25/russian-fake-news/ <comment><USER3> 'Fake news' is as old as humanity itself and there is no reason to prefer this new term to simply 'false information' or something similar. As another poster said, this current rash of articles is pushing the idea that there is a binary distinction between fake and real news, when in reality we already have a rich vocabulary to discuss the nuanced spectrum between truth and falsehood, e.g. misinformation, disinformation, black / white / grey propaganda, selective exposure, fallacy, rumor, hyperbole, euphemism / dysphemism, limited hangout, dog whistling, spin / framing, FUD, bias, etc.

It is hard not to feel very cynical about the sudden post-election focus that large media outlets are putting on 'fake news'. I think it is mostly an attempt to discredit alternative media and reinforce the establishment / government narrative.

<comment><USER4> To me it seems that traditional media is taking this opportunity to make their case with Google/Facebook that they are the sole/main providers of "real" news and that others are less reputable. And this is just a way to re-establish their pedigree as canonical sources for news to the detriment of alternative news sources --including of course the inflated purveyors of "fake news" who are not the real targets. The actual targets being anyone not associated with an institutional news organization [your local news, independent news, bloggers, etc].

Imagine the glee in mainstream news if FB GOOG would mainly surface the news of the NYT, WaPo, FoxNews, etc.

<comment><USER5> Well you can at least argue that traditional news media does hold itself to certain journalistic standards (I'm talking news articles, not opinion pieces).

I think an issue we have with getting news from places like FB is that (I think) it will often show news or stories which you probably align with politically or reinforce your views (leading to increased polarization). Compare that to how we used to get our news from papers or the TV were we couldn't control the programming.

<comment><USER6> WE couldn't control the content. But do we even know who was controlling what should be shown/investigated/high lined or not? Or what makes a "credible" font?

</comment></comment></comment></comment><comment><USER7> So The Washington Post article on fake news is lying to its readers. This perfectly illuminates the problem with trust, truth, and the Internet.

Two good rules of thumb - and this goes for established media outlets - <USER2> reading an article.

A. Follow the links that writers include in their articles.

Example: Well the article above. But this is how bad it is: Currently the WP's most popular article is titled 'Americans keep looking away from the election's most alarming story' which includes the line "Trump reveled in information gained from..."fake news" stories that evidence suggests (link) were generated by Russian intelligence operation". Well, if you click that link there is zero evidence, just an article from 'geektime.com' (who?) asking 'Could Russia be behind US election-based fake news on Facebook?'.

This is incredibly disingenuous.

B. Read the comments.

Example: Just go to the comments on any article on The Economist or The Guardian and you'll almost always find high quality addenda that either refute or add context to the article being commented on. Perhaps it's the limited number of media outlets in Europe, or just the nature of our media, but there's a high level of healthy dissent below-the-line that I don't see on The NYT, WP etc.

This assumes, of course, that people care about the truth. Yet critical thinking may be futile against a firehose of information and this trait of human psychologically: many people value how they feel over the truth and so will be biased towards 'news' that fulfills that role.

It's not fearless critical reporting they're after, but rather a bubble of moral affirmation.

<comment><USER8> Fake news illustrates what people _actually_ want when they share "news" on Facebook/Twitter (and to a lesser degree, sites like Reddit): a headline, not a story. It's really just the ultimate in clickbait.

<comment><USER7> True. I miss the days when people made a quick buck from sharing funny pictures of cats instead of appealing to tribal instincts.

As vacuous as fake news but a lot less divisive.*

* _Well, unless you 're not a cat person that is._

</comment></comment></comment><comment><USER5> I'm not sure how relevant this is to this story but for news in general something which I found extremely helpful in my life has been actually learning how to digest the news (media) through a course that was taught at university (news literacy).

The news media is to a certain extent peer reviewed. Although perhaps not the the extent that scientific papers are but actually non-opinion news media does hold itself to certain standards. These may including correcting factual errors, attempting to contact both/all sides for comments, or if they use anonymous sources if they state a reason why they are anonymous. Furthermore the actual news media should always have the opinion section marked as opinion (which you should generally steer clear of).

Another example which is often brought up is "bias". However actual bias is extremely difficult to prove. Opinion pieces don't count towards bias since they should be under a (labeled) opinion section.

There are courses that deal with news literacy[1] that are probably taught around the country but this is just one example that I'm familiar with.

[1] [http://www.centerfornewsliteracy.org/](http://www.centerfornewsliteracy.org/) (SUNY @ Stony Brook)

<comment><USER9> > However actual bias is extremely difficult to prove.

Actually it is extremely easy. It is easy to tell if an article has an angle or as not, by the choices of word and expressions. For instance, let's take the immigration crisis in Europe. Whether a media chose to call immigrants "illegal immigrants" or "asylum seekers" denoted bias in one direction or another. So left-wing media called them "asylum seekers" because they wanted to paint them in a good way, while right-wing media called them "illegal immigrants" to paint them in a negative way.

Now calling something "fake news" is biased too. All news are fabricated to an extent. "fake news" basically means , "news that don't come from an established source", it's new speak for unsanctioned news, to make it look like trusted news can only come from the NYT, Wapo and co ... it is extremely dishonest, since these media are propaganda organs.

> Opinion pieces don't count towards bias since they should be under a > (labeled) opinion section.

Journalists are activists. Whether they write opinion pieces or not. The easiest way to tell facts from opinion is to take multiple sources opposed politically, and compare what they have in common and in which way they differ. Facts may be inferred that way. Off course it gets harder when the media choose not to report events because it contradicts the narrative they are pushing, and we all saw the result in the latest election. They can't blame "fake news" yet suppress information when it doesn't fit their narrative.

So obviously i'm not a journalist, but you could say I'm biased against MSM. More accurately I don't give them the benefit of the doubt just because they are professionals.

<comment><USER5> > Actually it is extremely easy. It is easy to tell if an article has an angle > or as not

When I'm talking about bias it's something which would not be provable based on one article. I'm talking about something which would span multiple years and many articles. That kind of bias is very difficult to prove.

> Now calling something "fake news" is biased too.

Do not conflate non-mainstream news sources with news which has been completely fabricated. These are two completely different things. Furthermore I would argue that the mainstream news media holds itself to certain journalistic standards that a random internet blog may not.

> NYT, Wapo and co ... it is extremely dishonest, since these media are > propaganda organs.

Using the term "propaganda organs" makes me think you believe them to be biased sources? What exactly makes their news articles badly written?

> The easiest way to tell facts from opinion is to take multiple sources > opposed politically

A good news article will already include multiple sources and try and have both points of view.

I think something which is increasingly becoming a problem is people getting their only news from online sources which they agree with or align with politically. This is different from how we consumed news media in the past since we couldn't control what was in the paper or on the TV (and is probably what has led to the increased polarization in politics).

<comment><USER10> > Using the term "propaganda organs" makes me think you believe them to be > biased sources? What exactly makes their news articles badly written?

The build up to the Iraq war is a perfect example of how these august, mainstream news organizations were rendered to be mouthpieces of the then ruling administration. Another example is the NYT constant villification of Edward Snowden.

I agree with you in principle that Google, Facebook, et al do have a serious problem. The revenue models incentivize getting the maximum number of clicks and this leads to clickbait articles and outright lies posed as reputable stories targeted towards those of a certain political persuasion. This model should be changed.

However, the various proposals being bandied about would effectively elevate traditional media outlets to the status of true gatekeepers of news and re- centralize power into their hands.

</comment></comment></comment></comment><comment><USER11> How do we know that this article isn't Russian propaganda? I just prefer to go full solipsism.

<comment><USER12> How do we know you aren't working for the KGB?

<comment><USER13> We can be pretty sure that he does not, since KGB no longer exists.

<comment><USER14> That sounds like something a KGB agent would say.... Comrade....

</comment></comment></comment></comment><comment><USER15> It seems dangerous to view "fake news" as a binary value. All news is just approximation of the truth.

<comment><USER16> >> All news is just approximation of the truth.

No. There is something called truth. Did it rain yesterday or did it not? What was the temperature last week? These are things that are scientifically true or not true and yet over the last election cycle these facts were up for "debate". I don't mean climate change. They argued about whether it was hot or cold on a specific day at a specific place.

The weather is easy. Where the grey areas start are with things like budgets. Did a particular department spend more or less this year that last? Such statements would seem objectively true or untrue yet we saw grown men and women fight for years, one side seeing surplus and the other deficit. In the middle is an accountant who knows the objective truth, the real story from the fake news. When the accountant comes forward, that story is then slotted into real or fake. If there are never any objectively true or untrue facts, why do we even bother talking?

<comment><USER17> Let's say I'm reporting on a study that sets out to see if there is wage discrimination against earthling immigrants in favor of native martians. The study shows that earthlings earn more than martians on average, but once you control for occupation and education and all that, earthlings actually make less for doing the same thing.

It's true to report that earthlings tend to earn more than martians. I could use that fact from the study to form the basis of a totally factual story that goes completely against the study itself. My readers will come away thinking discrimination happens in earthlings' favor instead of against them. Totally true facts can support a totally false conclusion, depending on presentation.

</comment></comment></comment><comment><USER18> All of this fake news discussion makes me think about McLuhan. It's as if we are forgetting the point of internet.

> _McLuhan 's central thesis, encapsulated in the famous phrase "the medium is > the message", was that the technologies through which we take in information > - the media, broadly defined - become "extensions" of our bodies, exerting a > profound influence over us. When an important new medium arrives, it can > reshape who we are as individuals and as a society._ [1]

> _Electric media, being cool technologies that promote interaction, would > bring back our lost tribal consciousness, McLuhan believed. But our tribes > would no longer be small, isolated groups. Because the new media spanned the > planet, we would become members of a "global village"._ [1]

[1] [https://www.theguardian.com/technology/2007/nov/01/comment.i...](https://www.theguardian.com/technology/2007/nov/01/comment.internet)

<comment><USER19> It's curious how people like McLuhan and Lippman and Bernays are hardly ever discussed, even though they hold the keys to understanding what's going on.

</comment></comment><comment><USER20> I have a hard time accepting twitter quotes as a source for use in real journalism.

</comment><comment><USER21> The washington post recently printed a staggering hypocritical article on fake news: [https://www.washingtonpost.com/business/economy/russian- prop...](https://www.washingtonpost.com/business/economy/russian-propaganda- effort-helped-spread-fake-news-during-election-experts- say/2016/11/24/793903b6-8a40-4ca9-b712-716af66098fe_story.html)

This is a rag that has been a sock puppet for the DNC since forever.

<comment><USER22> Greenwald's piece on it: [https://theintercept.com/2016/11/26/washington-post- disgrace...](https://theintercept.com/2016/11/26/washington-post- disgracefully-promotes-a-mccarthyite-blacklist-from-a-new-hidden-and-very- shady-group/)

<comment><USER23> HN discussion of Greenwald's article: [https://news.ycombinator.com/item?id=13045381](https://news.ycombinator.com/item?id=13045381)

</comment></comment></comment><comment><USER24> Don't believe everything you read on the internet, or the newspapers. Nobody has the monopoly on truth.

Healthy scepticism is a life skill. More so now when rumours can spread globally in seconds. The solution is not control but education and discernment. Our prejudices and current knowledge often determine to a large degree what we more easily believe.

The traditional media is not only losing reach but also its power to 'shape and influence events' ie propaganda and in desperation the 'free press' has been reduced to the sorry state of seeking 'official source of truth status'. Sort of like a ministry of truth.

</comment><comment><USER25> This is an odd sentiment. Most of the reporting around fake new shows it comes from entrepreneurial people, some I would say are morally suspect, but I have never seen any reporting show it going back to nation states. Its just clever people take advantage of people awful biases.

<comment><USER26> Really? here are a few stories [https://www.washingtonpost.com/politics/report-russian- propa...](https://www.washingtonpost.com/politics/report-russian-propaganda- effort-spread-fake- news/2016/11/25/72eb461c-b33a-11e6-bc2d-19b3d759cfe7_story.html)

[http://www.pbs.org/newshour/rundown/russian-propaganda- effor...](http://www.pbs.org/newshour/rundown/russian-propaganda-effort- behind-flood-fake-news-preceded-election/)

Edit: a few more

[http://www.usatoday.com/story/news/2016/11/25/reports- russia...](http://www.usatoday.com/story/news/2016/11/25/reports-russian- operation-boosted-fake-news-phenomenon/94424206/)

[http://www.theverge.com/2016/11/25/13746250/us-election- russ...](http://www.theverge.com/2016/11/25/13746250/us-election-russia- influence-fake-news-propaganda)

[http://www.seattlepi.com/business/technology/article/Report-...](http://www.seattlepi.com/business/technology/article/Report- Russian-propaganda-effort-spread-fake-news-10635910.php)

<comment><USER22> You are actually citing that WaPo propOrnot piece? You trust independent, anonymous experts to tell you what is legit? There is zero fact checking in that thing. You might as well be sourcing 4chan.

Talk about playing to people's biases...

EDIT: I'm retarded.

<comment><USER26> You misunderstood me. Parent mentioned that he had not seen this narrative about Russian propaganda anywhere. I saw a whole bunch just yesterday so I posted them there. I personally find them ridiculous and agree with fortune piece which I believe was a reaction to WaPo piece.

<comment><USER22> Ah, apologies!

</comment></comment></comment></comment></comment><comment><USER27> I do not google a term to get a false result that I happen to agree with. Especially not one that is doing so just to sell ads. I am not interested in soothing nonsense on _any_ topic not just political ones.

For some reason it is being argued that google's (and facebook's) algorithms, which obviously must rank their results, should not take accuracy into account at all.

The value of google is exactly proportional to the accuracy of its results, not its random interspersing of fabrications. If I want insane conspiracy theories I can always take 3 seconds to type bullshit.com in the url line.

And ironically, somehow observations like this one are ruthlessly down voted by defenders of free speech.

<comment><USER28> Google indexes content, and displays it by relevance to your query and overall relevance (e.g. PageRank). The idea that Google and FB should be in the business of discriminating truth from falsehood is one of the most naive ideas to come out of this election cycle.

<comment><USER27> On the contrary.

The discovery that social media, as currently implemented, is in the process of annihilating rational discourse is the single most important discovery of this election cycle. Possibly of the last decade.

Even now, because of this problem, there is no longer agreement on many basic facts. The notion that a people so polarized on basic truths will ever be able to function together democratically is what is utterly naive. Under such circumstances, ultimately one side will have to force its view of reality on the other.

Indeed it may not be a coincidence that the sudden success of extremists parties in the west, driving voters to vote so conspicuously against their own self interest, corresponds almost exactly to the emergence of social media in general and facebook in particular.

What is naive is the belief that truth will somehow win out when the bulk of mankind sees the world predominantly through a facebook feed that explicitly selects for spectacular self confirming nonsense algorithmicly suppressing anything unpleasant to know.

<comment><USER28> You overestimate the health of rational discourse before this election. Disagreements on basic facts are not new.

Social media has created a kind of nervous system which amplifies and spreads fears and foolishness. Yes, this is a problem that we should address. (Perhaps allowing interpersonal communications to be dominated by a handful of companies was not the best idea.)

What is naive is to solve this problem by putting the determination of truth and falsehood in the hands of a centralized authority.

<comment><USER27> >Perhaps allowing interpersonal communications to be dominated by a handful of companies was not the best idea.

Absolutely. Especially since tabloid like self confirming facile nonsense has broader user retention and thus profitability than difficult or subtle factual information.

>What is naive is to solve this problem by putting the determination of truth and falsehood in the hands of a centralized authority.

Indeed that would be catastrophic. But the ordering of the facebook timeline is intentionally emphasizing self confirming propagandistic nonsense: We already live in a highly propagandized facebook steam. Just remove that (highly profitable) bias from the algorithm.

>Disagreements on basic facts are not new.

Now however it is far worse, indeed essentially nonexistent. As a result the width of the division has grown enormously in the last 10 years. In the, US for example, the vast majority of counties are now either landslide red or landslide blue. The most polarized since the civil war. And there has never been this level of overt hostility to science at the top of the political ladder.

On the larger scale, it is a mistake to think that the experiment of enlightenment democracy that is only present in a minority of countries and in those, only for a fraction of history, is somehow permanent and can somehow endure when the public no longer has access to accurate information. Indeed, nascent democracies are already being snuffed out one by one.

<comment><USER28> > But the ordering of the facebook timeline is intentionally emphasizing self > confirming propagandistic nonsense: We already live in a highly > propagandized facebook steam. Just remove that (highly profitable) bias from > the algorithm.

We agree on the problem, just not the solution.

Let's consider two sub-problems. First, the advertising-based business model benefits from compulsive behavior and engaging, viral content. The most engaging content is not always the most true, and the second problem: some of it is entirely fabricated.

Put these two parts together and you get the current situation. Now we see many crocodile tears shed over the fact that the second problem spread so much untrue content that it may have swung the election. Whether that's realistic or not, it creates an excellent opportunity to establish an authority that determines which stories are true, or how true a story is. Who wouldn't want that job? What a way to make a difference. So we have this:

> Under such circumstances, ultimately one side will have to force its view of > reality on the other.

And what better way to do that than by establishing a ministry of truth. (Whether established by the government or private enterprise is practically immaterial.) But might it be worth stepping back for a moment first?

If the only way democracy can work is by the elite directly controlling the perception of truth, then what is it that democracy still offers?

> Just remove that (highly profitable) bias from the algorithm.

Sell that to the shareholders! "Engagement is down but our metrics show the electorate is slightly better informed due to our efforts this quarter." So here you get at the first problem, which is that Facebook's monetization is directly at odds with many of the things that people should be doing to make a democracy work, such as getting off Facebook, reading a history book, engaging with their neighbors, and so on. Of course nobody is talking about solving this, because none of the solutions to this problem allow anyone to amass money or power.

<comment><USER29> _We agree on the problem, just not the solution._

Thank you! This has been weighing on my mind ever since the fakenews meme has infected HN, I just haven't been able to express it constructively. It seems that admitting any common ground feels like surrender in the current political climate. It would be great if we could start from this statement and look for some workable, agreeable solution.

<comment><USER28> Absolutely. Smart people disagreeing and working together anyway is our best hope for getting out of this mess.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER30> The global media (and the US ones disproportionately) is behind almost every piece of fake news people all over the world see -- only it's called just "news".

</comment><comment><USER31> Pot, meet Kettle.

</comment><comment><USER32> Why people are so stupid? No one would say that "Russians" wrote this or that shitpost. What they did - they developed and successfully used "innovative" techniques of producing and spreading catchy memes along with trolling and shitposting into any meaningful discussion on popular public forums. These methods nowadays are used on a "factory scale". The famous "Troll factories" were a great success.

Basically, it is a next level of jamming techniques used against western radio stations in old times - active trolling instead of passive jamming, using paid people instead of dumb noise generators.

What Russians developed is very efficient (optimized, if you wish) set of methods to manipulate the minds of idiots, to implant naive, unverified (and unquestioned by wast majority of idiots) assumptions and false premises, in forms of emotionally charged but primitive memes (textual or graphical) - "Shillary will start WW3". Fucking degenerates.

In the same way all the Russian media propaganda is organized nowadays - meme- like framed "news", tailored to the vocabulary and mentality of uneducated, primitive majority, along with marketing-like targeting. You do not have to have a PhD to do that.

What happened is that almost everyone else nowadays are re-using these simple ready-made methods. All this is nothing but virus-like meme contagion, plus applied manipulative psychology of advertising on internet scale. That's all. No KGB agents or anything like that.

How a third-world, corruption-ridden country with stagnant economy could be so powerful? It obviously wasn't. The cause is these "information viruses" (memes) spreading through human stupidity. Russians didn't invent them, of course, only used on a large scale.

</comment><comment><USER33> I guess Putin's hackers tricked BBC to publish that the late owner of Coronita beer factory left enormous amounts of money to the village where he was born.

This fake news meme is so boring by now. The mainstream media try to blame others for their falure to manipulate the people in a direction desired by the elites, but actually nowadays mainstream media is no better than the fake news sources.

</comment><comment><USER34> Don't worry everyone, Fake News is the biggest problem the world has faced ever and we will not hear the end of this very, very serious issue.

If we keep repeating this message enough, include it on everything from t-shirts, billboards to radio and tv ads (maybe even handing out steep discounts at stores everywhere if you have people regurgitate some of the websites to not visit from the very reputable list that the extremely unbiased and trustworthy Jeff Bezos' Washington Post has helped distribute recently), everyone will eventually come around, the world will be saved and everything everyone dead or alive has ever wished for will come true.

<comment><USER35> Or shorter: "Don't feed the (WaPo) trolls" :)</comment></comment>


The China Study: Fact or Fallacy? - <AUTHOR> http://rawfoodsos.com/2010/07/07/the-china-study-fact-or-fallac/

<comment><USER2> Can someone tldr this? Basically meat is bad, but whey is ok, and soy isn't quite as good as once thought?

<comment><USER3> tldr: "Fallacy"

Slightly longer version: some dude published a book which said that eating meat is bad. This article says said book is largely BS. So go eat some meat.

My opinion: Complete BS outweighs good science by a factor of several thousand to one, when it comes to nutrition. There's no particularly strong evidence that one diet is _significant_ better than any other, so as long as you're not fat or malnourished you're probably doing 90% as well as you possibly could be. So relax and stop reading fifteen-thousand-word articles debunking the statistics in some diet book.

<comment><USER4> I really agree with you when you say trying to find correlation with a specific food ingredient and one or several diseases is a fallacy in itself.

That being said there is some strong evidence that the _Okinawa diet_ is extremely good for the health. Note that the key word is "diet", meaning it's not just one single food ingredient. Actually that "diet" is actually more a lifestyle than a mere diet.

Generally speaking a healthy life isn't difficult: regular exercise, don't overeat, avoid refined sugars and alcohol, no soda, no fast food and most important: enjoy yourself!

<comment><USER5> _That being said there is some strong evidence that the _Okinawa diet_ is extremely good for the health._

Actually, I tried to find out what the Okinawa diet is, but I found contradictory information. For example, some sources say that it is nearly vegetarian, some say that they eat significant amount of pork and cook on pork lard. Also, I found some articles about research on Okinawa centenarians genetics and some genes were identified that correlate with longevity. So it might not be just about the diet.

I would appreciate any links on Okinawa diet, as long as they can be considered credible.</comment></comment></comment></comment>


Prius saves gas money? (follow-up) - <AUTHOR> http://aaronhawley.livejournal.com/29976.html <comment><USER2> "A convenient and fair comparison is the Corolla also made by Toyota."

No a fair comparison would be a non-hybrid Prius. I don't know if Toyota makes such a vehicle, but Honda makes the Civic in both a hybrid and conventional model, so a direct comparison would be fair. I'd actually like to see that analysis as I drive a Civic.

<comment><USER3> To borrow from Joel Spolsky, what this shows you is that you can prove anything with a contrived example. More specifically, for any comparison that isn't night-and-day, you can almost always find a reasonable set of assumptions that leads to your preferred conclusion.

> I don't know if Toyota makes such a vehicle

They don't, and actually the Prius sits between the Corolla and the Camry in size and features. The cost premium over a Camry is only about $2k, and the fuel economy improvement is more dramatic. (The Camry has substantially more power, but the Prius's flat torque curve probably reduces the real-world difference).

Another problem with this analysis is that on the one hand it ignores resale value, but on the other hand assumes that the owner won't keep the car more than a few years. Maybe Priuses have crap resale value, but the author doesn't state that much less back it up.

But the biggest problem is that only businesses buy cars this way.

People don't buy sports cars because they get them to work faster, and people don't generally buy SUVs because they have to drive off-road to get to the grocery store. People buy cars (or forego them) to tell themselves a story about what kind of person they are (and there are many comical instances where it tells an entirely different story to others).

Business do buy cars this way, and if you look at those that do a lot of city driving in an area with even moderately high gas prices (say, Vancouver, with gas at US$5.15 a US gallon), you'll find that, for instance, 90% of the taxis are Priuses.

<comment><USER4> > (say, Vancouver, with gas at US$5.15 a US gallon), you'll find that, for > instance, 90% of the taxis are Priuses.

That appears to be an exaggeration, according to this [http://www.calgaryjournalonline.ca/news/34-news/1281-calgary...](http://www.calgaryjournalonline.ca/news/34-news/1281-calgary- taxi-companies-embrace-hybrid-vehicles) as of mid-2010, 50% of the fleet was hybrid. I suppose it's possible, but unlikely they ramped up that quick to over 90% hybrid (I imagine there would be some Fusions in the mix like in SF and Boston).

</comment></comment></comment><comment><USER5> The same comparison can be done (for estimating savings) between diesel and unleaded gas in France. I did a similar calculation years ago and you had to drive your diesel vehicle more than 10 years before making any savings. Basically the initial investment trumps the ongoing costs.

<comment><USER6> The price of fuel looms large in people's minds, since it is so visible, but if you actually keep track, it just doesn't add up to much compared to the cost of the average new car. Just goes to show, measure before you optimize :)

<comment><USER5> Yeah. it's all about perception. Since you refuel often, you see the fuel cost much more often than the actual initial car costs. It's constantly in a "top of mind" position for consumers. You can find the same phenomena across different fields. Like when you hear about people dying from cancer the whole time on TV, you may feel there is a great risk to actually get it anytime, while in fact, the main key factor to develop cancer is simply "age" (there are, of course, young folks diagnosed with cancer, but it's a clear minority).

</comment></comment></comment><comment><USER7> Like the article says, saving money isn't the only reason to drive a hybrid.

If you care about the environment, and are wealthy enough, you might be willing to pay a premium to reduce your carbon footprint.

<comment><USER8> Like the article also says, you don't actually know if you're reducing your "carbon footprint" or not.

You don't know how much more carbon is produced in the manufacture of a prius than the manufacture of a non-hybrid.

<comment><USER3> Price is generally a pretty good indication of resource intensity if you're comparing two things with generally similar level of externalities and subsidies/taxes.

If anything I would say the fuel cost understates the externalities, at least in the US.

</comment></comment></comment><comment><USER9> This argument only works when comparing a Prius to a Corolla... Which is the entry level bottom tier sedan from Toyota. I think it would be a bit more fair to use the Camry as the alternative car.

I'm a Prius owner and I didn't decide to spend an extra $6k just to get a Prius for the gas savings, otherwise I'd have a Camry. If I didn't buy a Prius I would have bought a similarly priced car because... that was the price range I was shopping in.

</comment><comment><USER10> it would be interesting to add resale value to this calculation.</comment>


Ask HN: What is a good way to host a Rails app for free? Or very cheap? - <AUTHOR>

This is for a personal website.
Heroku's free tier sucks because servers take 5-10 seconds to spin up every time they are used. Amazon EC2 isn't working out for me either.
Any advice? <comment><USER2> FREE:

appfog - <https://www.appfog.com/>

dotcloud - <http://dotcloud.com>

CHEAP:

webbynode, $15 entry level plan - <http://webbynode.com>

digital ocean, $5 entry level plan - <https://www.digitalocean.com/>

I've used appfog and it works alright. I am a paying customer for webbynode and like it a lot. Easy to deploy, ssh access, flexibility. $15 might be more than you want to pay, I've not tried digital ocean, but it seems like a nice entry level price point.

<comment><USER3> I like Digital Ocean (use them for <http://bngal.com> and <http://cheergram.com>) but be warned: I wasn't even able to compile nginx on the $5 plan. Kept getting memory-related errors. YMMV

</comment></comment><comment><USER4> <http://www.cloudvps.com/> A Dutch company that I've used for the last 18 months to run my VPS without issues. It also provides VPS's to the largest online accounting software company here (Exact software) has reasonable customer support

11 Euro (14 dollars) a month for: 1GB Ram, 20 GB HDD, 250 GB traffic and a single core.

If you want even cheaper you could try hetzner.de with 8 euro/month, 512MB VPS.

</comment><comment><USER5> You could check out lowendbox for cheap VPS. I had one for a year, only paid $50 for a 512MB Xen VPS. It ran a low volume rails site just fine. I did have to do a trick with mysql, and that was to disable TCP in MySQL and only use the local socket (/tmp/mysql.sock). That reduces the memory footprint of mysqld significantly.

</comment><comment><USER6> Linode. The 20USD/month plan is guaranteed to meet your needs, and their service and support are top notch. The LISH shell is also really really useful - esp. if you are new to sysadmin stuff, and has saved my ass a lot of times. Their collection of tutorials on sysadmin stuff is another huge benefit.

</comment><comment><USER7> When I had a rails app on Heroku (a while ago), I solved that by setting up a free Pingdom account to ping a full stack (hit the DB, hit any external services I was using) "heartbeat" route. I'm not sure if this works any more, though - they may have done something to work around it.

</comment><comment><USER8> What's the problem with the 5-10 seconds wake up time in Heroku? This does NOT happen every time they are used, it only happens ONCE every an hour or more of inactivity. This limitation becomes problematic however if your app has cron jobs every so often, or states (e.g. node app).

<comment><AUTHOR> It's fine for developing but I want to have this as something I can send to employers and friends. I don't want them to have to wait up to 10s to see a site that I spend a lot of time trying to speed up lol

<comment><USER9> there is a way around that. just install the new relic addon, and configure it to check your sites availability once every minute.(on a production site you want that anyway to get alerts if the app is down, and might need a restart)

like Donito said, it only needs to wake up if there hasn't been a request in over an hour -> which will never happen with the new relic addon, because it sends a request every minute.

</comment></comment></comment><comment><USER10> Another solution is CloudControl: <https://www.cloudcontrol.com/for- developers>. This is a provider from Berlin / Germany.

</comment><comment><USER11> OpenShift and AppFog are two PAASs I've used that both support Rails. Their free tiers will be more than adequate for a personal website.

<comment><AUTHOR> I'm going to try those next, thanks.

</comment></comment><comment><USER12> Can i have a question why Amazon EC2 doesn't work for you ?

<comment><USER13> Might not be a technical issue, maybe his free tier ran out? After that, a micro instance isn't really that cheap anymore.

<comment><AUTHOR> Yeah its a technical issue. I've been trying to set it up for the past couple days with rubber, its not worked once.

<comment><USER14> Elastic Beanstalk now supports rails..use that, and you should be up and running in a few hours.

[http://ruby.awsblog.com/post/Tx2AK2MFX0QHRIO/Deploying- Ruby-...](http://ruby.awsblog.com/post/Tx2AK2MFX0QHRIO/Deploying-Ruby- Applications-to-AWS-Elastic-Beanstalk-with-Git)

</comment></comment></comment></comment><comment><USER15> Heroku with 3 unicorn workers + NewRelic uptime pings

<comment><USER16> Pingdom or even a cronjob also works.

</comment></comment><comment><USER17> Dotcloud[1] is ideal for this sort of thing (to me). If it's a production app and you're just looking for a free/cheap ride, you might need to look elsewhere, but if the complaint is that the spinup time is too slow during development, I would definitely look into Dotcloud, which lets you have development instances free, and only production apps are charged.

<comment><AUTHOR> Might actually work for my site. I don't need a big or fast db as I will be the only one using the rails interface. External users are just served non- interactive pages.</comment></comment>


Confessions of a Used Programming Language Salesman (2007) - <AUTHOR> https://dl.acm.org/doi/pdf/10.1145/1297027.1297078 <comment><USER2> Speaking of Haskell and laziness.. I found myself wondering last night, what was the issue with laziness again? I think it was that it's making it difficult to parallelize things. Why would that be? Is it just an implementation artefact, or are the rules of lazy evaluation actually disallowing the compiler/runtime from batching things appropriately for parallel computation?

EDIT: Guess I was thinking of this: [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.36.3...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.36.3806)

> we have shown that lazy languages, even when implemented using a framework > well-suited for parallelism, can generate only small amount of parallelism. > [..] this result, when combined with the result concerning the low > parallelism for really lazy programs, can also be interpreted as saying that > there is not much hope for lazy languages as far as parallelism is > concerned: in effect, they are saying that we can hope to obtain parallelism > from programs written in a lazy language only when the programs _do not > need_ to be written in a lazy language!

That's a really old paper though (1995), if there are more recent relevant studies or results, I'd be interested..

<comment><USER3> The big problem with laziness is that it becomes impossible to reason about performance, because performance becomes noncompositional. If y = g(x) then we expect the computation time of f(g(x)) to be the computation time of g(x) plus the computation time of f(y), but in the presence of laziness that's no longer true. Any sufficiently large codebase hits a performance problem (usually a trivial one that ought to be easily resolved) eventually, and at that point you lose the ability to refactor fearlessly that's the great advantage of Haskell-like languages.

<comment><USER4> Hmm, I'm not sure that's quite right. The time taken by f(g(x)) under lazy evaluation is known to be no more than the time taken under strict evaluation. It's the _space_ usage that can be hard to predict.

<comment><USER3> > The time taken by f(g(x)) under lazy evaluation is known to be no more than > the time taken under strict evaluation.

True but irrelevant (particularly given that idiomatic Haskell involves a lot of constructions that would take infinite time under strict evaluation). If x < a, y < b, z < c and a + b = c, that doesn't give you any useful relationship between x, y and z.

<comment><USER4> A fair point. An example would be something like

        take 10 (map (+1) [1..])

</comment></comment></comment></comment></comment><comment><USER5> > _To appreciate the difficulties of XML, let's make a short excursion to the > world of XML schema [...], which must be one of the most complex artifacts > invented by mankind. The complexity of XSD is especially baffling since the > existing solution, DTDs, is already a perfectly fine solution._

Ah, the period where it was attempted to integrate XML as first-class type/literal into programming languages. I guess the conclusion can only be that what SGML/XML is designed to represent (namely, semistructured text documents with regular content models) was fundamentally misunderstood by folks wanting to represent (relatively benign) business records-like structures for service-oriented apps. The complexity results for statically type-checking programming languages having XML as literals were disappointing, and the whole thing got out of fashion. Then, shortly after Mozilla dropped E4X (XML literals in JavaScript), JSX and React happened. I honestly don't know what conclusion should be drawn here :)

<comment><USER3> My conclusions would be: usability in the small matters. Overengineering is the natural fate of standards and must be fought at every turn. Insist ruthlessly on conciseness at the cost of virtually anything else (but not consistency; remember Perl).

<comment><USER5> Yes, and it's all the more ironic that XML started out as a simplified subset of SGML - only to turn into a monstrosity shortly after with W3C's XML Schema and WS-* "death star" series of standards.

</comment></comment></comment><comment><USER6> Orthogonal to the topic but Erik Meijer is hilarious

</comment><comment><USER7> The author compares himself with the prophet and apparently believes that the acceptance of functional programming languages requires intensive missionary work - just like with an ideology or religion. However, the history of science shows that the right knowledge always prevails in the end. If it is so difficult to get functional programming languages to market, either the time is not yet ripe, or there are better solutions.

<comment><USER8> Programming languages are seldom chosen because of the merit of their core semantics. Successful programming languages are coupled to some platform or ecosystem and it is the platform that is chosen. For example JavaScript could have been functional or imperative or whatever - it wouldn't have mattered for it's success. Objective C was obscure until the success of the iOS platform, C# was pushed by MS as the way to write Windows apps, VB was the way to script office applications. None of these languages became successful due to their core semantics.

Haskell will always remain niche since it is not coupled to a successful platform. But compared to the thousands of other "second tier" languages it is doing pretty well.

<comment><USER9> I believe that you're right about why languages aren't chosen, but miss the alternative explanation. Languages are chosen for speed of deployment and ease of use, almost nothing else. From the dawn of time, when C and Unix beat out Lisp, all the way to modern JavaScript, the tool that solves a given set of problem quickest and easiest wins regardless of 'merit'. Worse is better, all the way down.

<comment><USER10> I agree, but there's one more twist: Problems are different. Languages are chosen for ease of writing _specific programs_ , not for "general programming".

I think, then, that FP is great for certain kinds of problems, and not so good for others. If you've got an FP problem, reach for an FP language. If you don't, don't.

What's an FP problem? Someone here (I wish I remembered who, I'd give credit where due) said "if you can think of your program as a pipe", then FP is appropriate. If the nature of your problem has a lot of state, FP probably isn't the answer. (Note well: _problems themselves_ can have a lot of state. It isn't always just the implementation - the problem itself may demand it.)

</comment></comment></comment></comment><comment><USER11> That is a one misleading title.</comment>


Storage Pod 6.0: Building a 60 Drive 480TB Storage Server - <AUTHOR> https://www.backblaze.com/blog/open-source-data-storage-server/ <comment><USER2> It's always great to see storage hardware being opened up, thanks Backblaze.

I have a question after reading the Vault overview here:

[https://www.backblaze.com/blog/vault-cloud-storage- architect...](https://www.backblaze.com/blog/vault-cloud-storage- architecture/)

I'm curious what the bandwidth demand is when an entire host fails and you have to drop a new replacement host in, since you would have 60 Tomes in the Vault that need to be rebuilt at once. You have at least two other parity copies when a host dies (assuming no other drives in the affected host's Tomes are dead on other hosts in the Vault set), so I'm guessing you can afford to wait the handful of days it would take to rebuild the host. I'm still curious to know what rebuild speed you plan for, since I'm guessing you'd be looking at 40Gbps NICs eventually. I was surprised to see only 2x10Gbps.

<comment><USER3> Brian from Backblaze here. If we do a full pod swap for a blank, we are disk / CPU limited in the rebuild, not 10 Gbit/sec limited. We have the rebuilds up to taking about 3/4ths of the 10 Gbit/sec link but we continue to try to change the software and buffering to get this higher.

It's MUCH better to lose only one drive, because we distribute the rebuild CPU task across all 20 CPUs in the tome and we can get it synced back up in a few hours. All the chattering back and forth still doesn't come anywhere close to maxing out the 10 Gbit.

For uploads, each individual pod has a 10 Gbit/sec connection, so ACTUALLY the vault has an aggregate of 200 Gbit/sec facing towards the internet. However, every byte that a pod accepts it has to ALSO retransmit that internally so that cuts the actual theoretical upload rate to 100 Gbit/sec for a properly parallelized application.

<comment><USER4> > If we do a full pod swap for a blank, we are disk / CPU limited in the > rebuild, not 10 Gbit/sec limited

Wow. This is pretty shocking TBH. My basic ceph nodes will crush a 40gbps link doing a rebuild/rebalance. What's your bottleneck?

<comment><USER5> Backblaze use an in-house Java pseudo-RAID layer based on Reed-Solomon codes, with 17 data shards and 3 parity shards. There are some performance numbers here [https://www.backblaze.com/blog/reed- solomon/](https://www.backblaze.com/blog/reed-solomon/) (2015).

<comment><USER6> I wouldn't mind understanding why, I'm assuming at the time ceph wasn't mature enough (and still somewhat is due to btrfs, yes I know it's optional).

<comment><USER4> btrfs isn't a requirement nor really a recommendation for production ceph clusters. It was really only there for weirdo benchers.

Bluestore and rocks DB really changes the back-end story altogether. No more POSIX filesystem junk with journals and such.

</comment></comment></comment></comment></comment></comment><comment><USER7> Hey Yev or Brian,

First I think it is awesome that you guys come on HN to respond when backblaze makes the news. Thank you.

If you bought a prebuilt storage pod (from somewhere like [http://www.backuppods.com/](http://www.backuppods.com/)) could you start out at less than capacity and add more as you go? Say you started with one drive -- would it need to go in a specific location? When you add more would there be a significant reconfiguration process? Is it possible to hot-expand? I guess location and config has a lot to do with software involved. Has there been a blog on the software side of managing that much storage? Can it be as simple as a spanning partition to treat all drives as a single drive?

Also, are you tracking the cost of SSD drives? Do you expect cost parity soon? It seems that capacity parity is getting close, but cost parity (especially at that capacity) is farther off. Do you have an estimate of when it might be or is it "far off, check again in 5 years". I'm sure you all found the ssd endurance test at techreport interesting ([http://techreport.com/review/27909/the-ssd-endurance- experim...](http://techreport.com/review/27909/the-ssd-endurance-experiment- theyre-all-dead)).

<comment><USER3> Yes, and we do this internally in our QA and load testing lab. It's an expensive waste to put $200,000 worth of drives in a 20 pod vault when all you need is 1/60th of that!

> are you tracking the cost of SSD drives? Do you expect cost parity soon?

Yes we track SSDs, and this is a hotly contested topic inside Backblaze. Personally I'm predicting they "cross over" in cost effectiveness within 2 years. But I don't have any more insight than you do, and many inside Backblaze disagree with me.

When I say "cost effective" that is total cost of ownership, which includes paying for less electricity to power the SSD, possibly the higher density of SSDs, and whatever failure rates of the SSDs forcing us to purchase more (might be better or worse than hard drives). So whatever our little spread sheet kicks out as cheaper -> that's the one we will purchase!

One random nice thing about SSDs is there will be less case vibration, and the drives shouldn't be affected AT ALL by the small amount of fan vibration.

<comment><USER8> Am I correct in assuming that the performance improvements of SSDs are worthless for your use case?

It seems like the workload of Backblaze (like many "big data" workloads) is such that the extra IOPS don't actually help.

With thousands of drive spindles, and a very small portion of the data on on the spindles being needed at any given moment, I don't imagine you are actually limited by the performance of spinning rust.

<comment><USER3> > the performance improvements of SSDs are worthless

No way! We are disk I/O limited right now. We think we can approximately double our performance simply by using SSDs instead of traditional 7200 RPM drives. I'm advocating for building an "SSD Pod" built with 1 or 2 TByte SSDs.

> and a very small portion of the data on on the spindles being needed at any > given moment

With our original Personal Backup product that was true. Everybody was happy with waiting a few hours for their restores to complete. But with our new product line of B2 (competes with Amazon S3) then suddenly performance can matter much more, because programmers using B2 might be implementing all sorts of different access patterns. If those programmers are using B2 to implement online backup, then yes, still fine. But if the programmers are building hacker news or reddit or Dropbox the access patterns might require faster IOPS to be more responsive.

<comment><USER8> I guess the B2 product is growing quickly or is where you are looking to drive growth? Or are you using different storage pools for different products?

It's surprising to hear this. I would have thought that your backup workloads would leave enough unused IOPS that you wouldn't be anywhere close to hitting a performance ceiling.

What metrics do you optimize towards? Tail latency?

</comment></comment></comment></comment></comment><comment><USER9> So a hard drive weighs about 1.5 pounds, so 60 of them in a pod is about 100 pounds before you even include the pod itself. That doesn't sound particularly pleasant for the technician to install the first time...

You put 10 of those in a rack and now we're up to 1000 pounds. At what point does this become challenging for the _floor_?

<comment><USER10> > That doesn't sound particularly pleasant for the technician to install the > first time...

I've pretty much reconciled I'm going have to buy something from [http://serverlift.com](http://serverlift.com) at some point if I keep doing this job and stuff keeps getting heavier.

<comment> mark-r Just fill the pod with those helium drives, it will float right into place!

<comment><USER11> Tried a few once...had to tether them to the ground :P

</comment></comment></comment></comment><comment><USER12> One note - looks like you can get 8TB WD Helium drives for way cheaper ($250 vs $340) right now by shucking externals: [http://www.amazon.com/Book-Desktop- External-Drive-WDBFJK0080...](http://www.amazon.com/Book-Desktop-External- Drive-WDBFJK0080HBK-NESN/dp/B01B6BN0Q2)

<comment><USER13> Thanks for the tip. Though Backblaze has spoiled us with so much good reliability data, I now feel completely paralyzed in the face of buying a new drive if Backblaze hasn't told me how much I can trust it!

<comment><USER11> Yev from Backblaze here -> You can trust it. As long as you're OK with it dying and have other backups in place ;-)

<comment><USER13> Ha! Well, even if I have backups, I don't like buying replacements, so there's that...

</comment></comment></comment></comment><comment><USER14> Now at $0.036/GB. So basically for the cost of storing the data for a single month in the cloud - and cloud storage is one of the more price competitive options in the cloud.

<comment><USER11> Yev from Backblaze here -> We actually offer Backblaze B2 at $0.005/GB/Month, so slightly less expensive! -> [https://www.backblaze.com/b2/cloud-storage- pricing.html](https://www.backblaze.com/b2/cloud-storage-pricing.html)

<comment><USER15> Are you still in a single DC? I'd love to suggest you to clients.

<comment><USER11> Currently yup! But we're actively looking to expand ;-)

</comment></comment></comment></comment><comment><USER16> At the rate of 0.036/GB, I'm just paying Backblaze USD50/year. Meaning if I store more than 115.74GB per month, Backblaze is losing money? And this is the cost only for storage, it could be 80GB to break even the cost and revenue. How do you guys make sure you can sustain in the long run? (I don't want to face another 'Copy case' T.T)

<comment><USER11> Yev from Backblaze here -> Accurate. We live off the averages. More people store less data than store more data. A lot of folks that have multiple terabytes backed up often tell their friends with less data and eventually it all averages out. We're bootstrapped so we have to be cash-flow positive, and at our $5/month price-point we are. We've been at this for 9 years too and so far its worked out OK! No reason to charge a lot for a service if you don't have to. We might need to move stuff around or change things in the future, but so far we've never had to raise prices and instead just keep adding features.

Tell your friends? :)

</comment></comment><comment><USER17> I've never heard about SMR before. I wonder - does it mean that a write to one area must be done in multiple disk rotations? Ie: if a sequential write to one file be aggregated into a single location on the disk or will it be spread around the whole plate. Just wondering how a software utilizing this should behave.

[http://www.storagereview.com/what_is_shingled_magnetic_recor...](http://www.storagereview.com/what_is_shingled_magnetic_recording_smr)

[https://en.wikipedia.org/wiki/Shingled_magnetic_recording](https://en.wikipedia.org/wiki/Shingled_magnetic_recording)

<comment><USER18> Heads up - if you've bought a high density disk recently, it's probably SMR internally anyways.

A write to one area can be done in a single disc rotation, as long as it's sequential.

SMR disks added what are called ZBC, Zoned Block Commands to the SCSI spec. The disk itself exposes the ability to interact with zones several ways:

\- REPORT ZONES - What zones there are, how many, what type, and state of the zone.

\- OPEN / CLOSE / FINISH ZONE - zone write state management

So essentially, it's like a drive exposes ~8 files, opened with O_APPEND, and you have to manage how you stripe your writes between those files to handle your workload. Your disk can also have zones that aren't shingled, but those seem to have been fairly unpopular.

</comment></comment><comment><USER19> I also found an older post describing how they use Scrum to design the hardware to be pretty interesting: [https://www.backblaze.com/blog/designing- the-next-storage-po...](https://www.backblaze.com/blog/designing-the-next- storage-pod).

I wonder what limitations the 6-month production run requirement put on design - it would seem you would have to be a lot more conservative in your "definition of done" (a bad decision can be expensive when you buy a half-year worht of gear).

</comment><comment><USER20> Whenever i see a post on Storagepod, i always fantasize how much it will cost Apple to provide free iCloud Backup to its users. If you have 600M active users and 30GB per backup @ 0.036/Gb. That is 648M, so even triple redundancy will only cost less then 2Billion.

If Apple could afford to give free OSX update, free iWork and ilife, Why not free iOS Backup?

Or at least offer a Time Capsule that i can backup my iOS devices without going to the cloud.

<comment><USER11> Yea. Right?

</comment></comment><comment><USER4> Any reason on 60 drives instead of 70 or more? At that depth, there are several OEM designs that with that many drives (70).

No 25/40gig networking?

It's nice to point out the depth issue, as older racks are shallower than the newer versions. This blocks things like the 0U PDU channels and cable management channels. That said, the newer, deeper (and more popular) racks fit everything just fine.

How tall are your racks? How many pods are you getting in there? 3ph 50A power?

<comment><USER3> > Any reason on 60 drives instead of 70 or more?

One step at a time. :-) We think we can put a few more drives in the motherboard half of the POD without many other changes, and connect them to already existing SATA connectors on the motherboard. To get all the way to 75 drives we will need another row of port multipliers, which then means we need even more SATA cards, etc.

> No 25/40gig networking?

We're currently not able to saturate the 10 Gbit so for us it is wasted money to go faster. Remember that we run these in "vaults of 20" so we have 20 pods EACH with 10 Gbits so the vault hoovers in data at 200 Gbits/sec if you can thread the application.

> How tall are your racks?

Most of our racks are 45U tall where the PDU takes 4U, so we can fit (not necessarily power) 10 pods per rack and still fit a network switch in the 1U left at the top. In the past (with 45 drive pods) we ran 2 circuits of 30A 208V power (three phase). The two circuits are redundant - our power strips can flip over from one to the other if there is a loss of power on the currently used power. But I think they only put 8 or 9 of the 45 drive pods in a cabinet because it slightly exceeds what the datacenter likes. And now that we're moving up to all 60 drive pods, we have to rejigger the whole thing.

It is also worth mentioning that when we bring a 20 pod vault online, we put each of the 20 pods in different racks in different parts of the datacenter. This helps keep the vault completely online if any one component fails. For example, if the power strip feeding the pods in 1 rack power dies, it only takes 1 pod offline out of several different vaults, so ALL the vaults stay online both reading and writing data. (I hope that made sense.)

<comment><USER4> Makes sense. Thanks for the reply!

As to spreading the vaults around the DC, it sounds like that's more to prepare for ToR switch failure -- its sounds like an ATS handles your PDU and power circuit failover.

Yeah it sounds like you're mostly CPU bound calculating EC if you're going to rebalance and replace failed pods so 25 gbps ethernet probably isn't interesting (yet). That said, it's only marginally more expensive than 10gbps twinax. Interesting that you're running 10gig baseT. Interesting from a power and latency perspective, though I'd imagine you're not worried too much about latency.

</comment></comment></comment><comment><USER21> Any idea what software is used to make the Build Book? [https://f001.backblaze.com/file/Backblaze_Blog/Storage- Pod-6...](https://f001.backblaze.com/file/Backblaze_Blog/Storage- Pod-6/Build+Book+Backblaze+60+Drive.pdf)

<comment><USER11> Yev from Backblaze here -> You're not going to believe this, but the folks at Evolve used Powerpoint and a template...seriously.

<comment><USER22> You would be astounded to learn how much use MS Office still gets today. Excel...that thing runs the world.

</comment></comment></comment><comment><USER23> Where or how do you find places that make chassis? I'd love to buy 100 or so completely basic cases to our design.

</comment><comment><USER24> Dropbox manage to do 1PB per server, why doesn't backblaze do it?

<comment><USER25> Backblaze are/were bootstrapped. They don't have the money to burn to move outside of the sweetspot $/GB - so their logic starts with which size drive is expected to be cheapest per GB in the next 12 months. I remember some of their old posts explaining why they made the jump from 2 to 3tb drives - they are slightly more expensive today, but we expect them to go down in price soon.

<comment><USER11> This is accurate. Unlike many Silicon Valley companies, we live within our means :D

</comment></comment></comment><comment><USER6> Whoa wait they don't use ecc ram?

</comment><comment><AUTHOR> I accidentally misspelled 'BackBlaze' in the title; can someone fix it please?

Interesting decision by BackBlaze to stuff an extra row of drives in, lengthening the chassis to 33" instead of just under 29". It is _their_ design, so they're free to do what they need for their own racks, but this does mean anyone trying to install a pod based on the 6.0 design couldn't use many closed-back racks or other kinds of racks.

<comment><USER26> Are they offering the 6.0 controllers, etc. with the standard length housing? I would think that's an option soon, if not now. Perhaps the 6.0 controllers have no benefit with fewer drives.

<comment><USER3> Brian from Backblaze here. Just to be absolutely clear, you cannot "buy" one of these from Backblaze. We don't sell them. You are free to take these instructions and assemble one yourself though! And so yes, you could simply drop the 6.0 controllers into an older chassis and it will work fine.

Also, the 6.0 housing will fit in most racks. If somebody can either measure their racks or include a link where the 33" chassis will no longer fit, we really want to hear about it!

<comment><USER27> Protocase, the vendor Backblaze sourced to fabricate the enclosures, has their own subsidiary that'll sell you turnkey pods:

[http://www.45drives.com/](http://www.45drives.com/)

<comment><USER28> Surely they should be rebranding to 60drives now? ;)</comment></comment></comment></comment></comment>