Avoiding overflow trick: log(a+b) from log(a) and log(b) - <AUTHOR> https://facwiki.cs.byu.edu/nlp/index.php/Log_Domain_Computations

<comment><USER2> I was dealing and implementing this last week. From what I understood, calculating log(a+b) from log(a) and log(b) has reasons other than underflow- overflow problem.

1- it is usually a necessity because For example in Speech Recognition and many NLP applications people always deal with log probabilities and log likelihoods.Yes they do it for running away from the underflow problem but all formulation and data is based on log.

2- Also, when dealing with log probabilities, it is possible to generate a look-up table to make this calculation much faster (Around 30 folds) with a good approximation instead of calculating the log(1+exp(log(b)-log(a))) part.

This is helpful for example when trying to find how likely a piece of speech signal is generated from an utterance of a letter (not formally speaking) using Gaussian Mixture Models. LogSum is extensively used there. It is also used in Forward-Backward algorithm used in Hidden Markov Model generation and decoding.

Edited for some grammar and missing information.

<comment><USER3> Useful to you?

[https://github.com/akhudek/feast/blob/master/src/btl/trunk/b...](https://github.com/akhudek/feast/blob/master/src/btl/trunk/btl/logint.h)

<comment><USER2> Pretty good, thanks. But because I am using Java this may not be that useful for me. I will definitely check the math tough. I will send it to my colleague as well who deals with C++ GMMs.

</comment></comment></comment><comment><USER4> Don't try this at home, there are nasty edge cases around Infs/NaNs.

Use your math library instead; in Python, that'd be _numpy.logaddexp_ or _scipy.misc.logsumexp_.

</comment><comment><USER5> Site is down, Google cache:

[http://webcache.googleusercontent.com/search?q=cache:https:/...](http://webcache.googleusercontent.com/search?q=cache:https://facwiki.cs.byu.edu/nlp/index.php/Log_Domain_Computations&hl=en&biw=1528&bih=859&strip=1)

</comment><comment><USER6> I've run into this problem often when dealing with log probabilities in maximum likelihood estimation where the optimizer can try some numbers that are way off from the optimal value and you still want to return an accurate number. There is an package in R for handling these operations automatically [1].

[1] [http://cran.r-project.org/web/packages/Brobdingnag/index.htm...](http://cran.r-project.org/web/packages/Brobdingnag/index.html)

</comment><comment><USER7> See also <http://en.wikipedia.org/wiki/Logarithmic_number_system></comment>


Liberate Your EveryDNS Records - <AUTHOR> http://blog.cloudflare.com/everydns-magic-data-liberation-tool-redux

<comment><USER2> EveryDNS was the best. I switched last week to Rackspace Cloud DNS which, as best I can tell, is free(ish).

The DNS manager is hidden away within Hosting -> Cloud Servers -> (any random server) but accepts arbitrary records. No promises here that the records live on when you delete your last Rackspace Cloud instance.

Kudos to CloudFlare for offering export tools despite obstacles.

</comment><comment><USER3> I love what CloudFlare is doing. From offering a free service to basically cache and speed up your web site for free with minimal set up, to hosting CDNjs, to doing deals with MediaTemple to helping peeps migrate from EveryDNS. Major props.</comment>


Financial Illiteracy Is Killing Us - <AUTHOR> http://blogs.forbes.com/brettnelson/2010/08/24/financial-illiteracy-is-killing-us/?boxes=Homepagelighttop

<comment><USER2> "Financial literacy" is often a cover-up for schemes to brainwash consumers into believing "stocks always go up", and that you should sell your dreams down the river, give as much money to wall street as you can with your 401K, and kid yourself that you're going to be able to afford to send your kids to college and then retire comfortably some day.

That's just such a cock-and-bull story that people conclude "finance is hard, let's go shopping" and pull out the credit cards.

<comment><USER3> I agree with you that there great myths about finance. However, I do not find it fair to conclude that there is a mass belief that is all a lie.

I think people just don't think. They are fine to just stretch their finances to whatever whims them think they can handle.

<comment><USER2> Of course.

The situationists have long asked the question of "Why don't people listen to revolutionary movements?"

The answer is that people have a protective mechanism against BS in the media -- a kind of radical disengagement that keeps TV preachers and other demagogues from being much more dangerous than they are.

People aren't offered a story that makes sense; the average person certainly doesn't have a coherent criticism of Wall Street, but, given no framework that works, they just live for the day.

<comment><USER3> I agree with you that given the percentage changes in the way we view money, there has to be massive social change in thought.

I also agree that people filter what the news tells them.

However, I disagree that the two are related. I don't believe the news media causes people to spend more because they aren't having some sort satisfaction.

Other causes seem more likely to me.

1) People believe that their income will continually increase over their lifetime.

2) The increase in the number of advertisements over the last 50 years.

3)People have an accustomed lifestyle, but are making less money. Therefore, instead of cutting back they go into a little bit of debt. Which becomes alot of debt.

And of course, all of the above. 3)

</comment></comment></comment></comment><comment><USER4> Financial literacy is not going to be taught in schools until it appears on some standardized test.

Parents need to be picking up the slack and teaching their children these skills. For all of the helicoptering modern parents do, you'd think watching over finances would be part of it.

<comment><USER5> Or teaching their kids that producing or saving a dollar can be more rewarding than spending one.

<comment><USER6> That might be a hot or controvercial subject depending what potential banking or other corporate interests it might go against.

Banks might not like the fact that a new generation of consumers wants to save too much, don't want to carry a credit card ballance, don't want to buy homes with mortgages they have to pay for 30 years, would rather buy a used car with saved money instead of finanancing it and so on.

As much as the media harps about "financial illiteracy" there are also industries that thrive on that illiteracy.

In general whenever there is illiteracy someone will take advantage of it. In this case I think it is a multi-billion dollar business. That business will try to go against education.

It could very well be that lobbyist for large financial institutions would try to influence the curriculum. It would "court" the state education systems to make sure the "right" stuff is taught in the financial literacy course.

I also can see this subject be as hot as religion and politics and perhaps that is also why it is not taught.

<comment><USER5> I should hope that parents don't consider the interests of multi-national corporations when talking with their children. I assume that you meant the curriculum of public education, but even that should ( _should_ ) be driven by local opinion and not corporate interests.

</comment></comment></comment></comment><comment><USER7> "Economic ignorance is the death of cultures; it is presently eating away at the US, and is sadly most advanced in medicine and medical research."

[http://www.fightaging.org/archives/2006/02/death-for- everyon...](http://www.fightaging.org/archives/2006/02/death-for-everyone- before-inequality-for-anyone.php)

[http://www.fightaging.org/archives/2007/09/jealousy-fear- env...](http://www.fightaging.org/archives/2007/09/jealousy-fear-envy-and- economic-ignorance-powerful-poisons.php)

[http://www.fightaging.org/archives/2009/02/the-decline-is- al...](http://www.fightaging.org/archives/2009/02/the-decline-is-always-slow- until-the-rapid-collapse-at-the-end.php)

</comment><comment><USER8> Say that I am financially illiterate. How do I fix this?

<comment><USER9> The same way you fix the problem of now knowing how to code, not knowing brain surgery, not knowing how to drive a car - buy some books and study.

I'm not trying to be sarcastic, just to point out the strangeness of it all. Most people wouldn't turn the ignition key and start trying to drive a car without taking some instructon first, but they are happy to whip out that credit card whenever the chance allows them.

<comment><USER8> Which books? I didn't learn to drive a car by reading books: I paid for lessons. I wouldn't learn brain surgery by reading books, I'd go to medical school.

A rephrasing of this question: what level of financial literacy can a person reasonably expect to attain by just applying "common sense", and what is the baseline level of financial literacy that we should strive for?

<comment><USER9> Well, OK, I just chucked a few examples out there without thinking them through. In retrospect I don't think you'd learn to code much just by reading books either.

What I wanted to say was that to learn about managing your finances you need to take an active role in learning how things work - whether by reading books, going on a course, whatever.

As a base level of financial literacy I don't think people need that much to make a big difference in their lives. Teach people budgeting, how to track their spending, and a basic understanding of savings and credit interest rates/compound growth etc.

Teaching people to live within their means and use credit for "needs" instead of "wants" will be one of the hardest lessons to impart. I don't know how you'd accomplish that.

</comment></comment></comment></comment><comment> loup-vaillant I'm not sure the ones which could have the power to make that happen actually want everyone to be financially literate. If we all were, our financial system would probably changes deeply (if not violently) within the decade.

Same thing goes about the Internet and the food, by the way.

<comment><USER10> What's wrong with the financial system and internet? I have a pretty intimate knowledge of both, and I'm OK with them (I don't know anything about food though ...)

<comment><USER11> The primary focus of the US financial system is sales. In large part this is based on the ignorance of the average American. Take a 25 year old who makes 30k / year and invests 3k/year in their 401k. Now figure out how much they pay in fees on that investment over the next 40 years.

Now play the same game with a 25 year old who would like to invest 10 million and if anything they get worse service even if they get a lot more face time.

<comment><USER12> I'm not sure what you're getting at here, partly because I've never invested in a 401K _or_ tried to invest $10 million cash. How much fees does each of these guys pay?

<comment><USER11> For a 401k a little over 1/2 a % per year or around (1.065 ^40/ 1.07 ^40) 17% of their final return over 40 years.

For 10 million investment ~1% per year is not uncommon so something like (1.06^40 / 1.07^40) = 32% of their return over 40 years.

Now plenty of people pay far more or less than these numbers, but that's a reasonable ballpark.

For more details from someone with a little money: <http://philip.greenspun.com/materialism/money>

"Jeffrey Greene owned $500 million in Southern California real estate and was getting nervous about its value. He was an old friend of Paulson's and, in 2006, let Paulson pitch him on the virtues of investing in his new fund to bet against mortgages. Instead of agreeing to _pay the 2-and-20 fee (2 percent annually raked off the top regardless of performance, plus 20 percent of any profits),_ Greene decided that he could run the trade himself. He pestered his Merrill Lynch broker until the guy got approval from top executives to allow a rich individual to participate in the illiquid and confusing credit default swap market. " [http://philip.greenspun.com/book-reviews/the-greatest- trade-...](http://philip.greenspun.com/book-reviews/the-greatest-trade-ever)

</comment></comment></comment></comment></comment><comment><USER13> I can vouch for this personally.

tl;dr - Financial illiteracy really is screwing some people with their pants on.

By any reasonable measure, my mom should be all set. Nothing more than a GED but she has started more than one successful business. Generally, she can drop herself into any town, open a shop for her services (dog grooming), and do great business. She's just really good at it.

She's in her 40's now and made a boatload of cash between '03 and '09, before the recession really kicked her business in the teeth. It should be no big deal -- she should have over a $100,000 saved away to smooth out a rough patch like this one.

Instead, she lost her house. Instead of keeping the nice, affordable big home she had three minutes from her shop, she increased her commute by 20 minutes in a bigger, more expensive house (with a pool!) she ended up unable to afford. Instead of saving, she bought big TV's, expensive pets, gambling and a Mercedes. Even Christmas of '09, when things were drum-tight, she was buying needless expensive gifts for people.

My mom is financially illiterate. When she should have been saving for a rainy day and setting up her retirement, she was blowing a considerable amount of money on stupid things. Worst of all, she has already been down this path – she had to declare bankruptcy back when I was a kid.

The other part that sucks: This crap gets passed on. I was financially an idiot for years. Only in the last couple of years have I gotten a frugal grip on my life, and now I get to dig out of some stupid debt decisions.

</comment><comment><USER14> As consumers, our biggest problem is living well within our means.

</comment><comment><USER15> Actually, it's not financial illiteracy that's killing us.

Image for a moment that we were financially literate, what would be able to do that we are not able to do today and what would be doing differently? Being financially literate would mean we would be able to _read_. However, as soon as we would be able to do that we would be shocked and we would walk away.

What is killing us is not being able to financially _write_. Money is a medium that has been designed so that it can hardly be _read_ let alone _written_. We need to redesign money. Money is a medium like all other media and it can be redesigned.

<comment><USER16> > Money is a medium that has been designed so that it can hardly be read let > alone written. We need to redesign money. Money is a medium like all other > media and it can be redesigned.

I have no idea what that means. When was money designed? Who designed it?

Money is just something we use so that we don't need to directly trade chickens for laptops. The current financial structure has some problems, but humanity hasn't come up with any better method for exchanging wealth.

<comment><USER17> Debit cards come to mind, if only they were managed by the government and not by private entities.

<comment><USER18> > Debit cards come to mind, if only they were managed by the government and > not by private entities.

What govt would resist the temptation to do dumb things with debit cards?

For example, the combination of school locations and laws about how close some people can live to schools means that there are many cities where such people can't live. (The specific category that I'm thinking of is registered sex offenders but there are others.)

Govt means that the majority gets to decide what happens to the minority. What could possibly go wrong....</comment></comment></comment></comment>


Introducing, Braintrust, my bootstrapped lean startup - <AUTHOR> http://www.tawheedkader.com/2010/05/introducing-braintrust-my-bootstrapped-lean-startup/

<comment><USER2> My criticism:

\- You don't have a "try here" or "demo" button. Like many people I won't spend the time creating a free account. And I am not interested in watching a video.

\- no other criticism (as I haven't tried it).

<comment><AUTHOR> Thanks. This is good feedback. I've just now added a one click demo button. <http://braintrust.io/demo>

</comment></comment><comment><USER3> I think you've basically created a simple version of Quora but you dont have the experts who can actually answer interesting questions.

Most people just email their friends if they want to figure something out. Braintrust = gmail conversation thread. Plus that is completely private.

I think you need to pivot.

You need to find some type of job which answers questions. And create a way to allow the "answerer" to bill clients per conversation thread or per time or something. There are other solutions to this problem but its just an idea ...

Eg: Lawyers,

Best of luck with it!

</comment><comment><USER4> I've been casually using this for the last couple weeks. It's really slick for what it does (multiple simultaneous conversations with the same group of people) and has some neat touches like upgrading comments to standalone discussions when people start to discuss them.

Congrats on the launch.

</comment><comment><USER5> You said that people kept asking for project management features, but your vision was to focus on conversation management. Do you think you will ever add in the project management aspect?

I can imagine that after using this service, people will eventually have to stop talking and actually do something! For example, we use Jira at work, and I find it lacking in the discussion dept. I would love to have your app's conversation tracking about specific revisions of code, QA cycles, production pushes, etc.

<comment><USER3> Extending byoung2's thought... Right now there is no way to tie in comments/programmer conversation and code. It would be great to have a tool that could be added to svn,git etc which would allow programmers to attach comments to code in such a way that I could then ask for all conversations for a particular feature since that feature was created to any enhancments done on it. ...

<comment><USER6> I believe github allows you to comment on commits. I don't know how well that is integrated with their bug tracker, though.

<comment><USER5> So does Jira. But it's just basic comments. Full conversation tracking like this app (including image and video embedding) would be amazing. For example, I would like QA to be able to embed a screenshot or video walkthrough when evaluating a specific revision.

</comment></comment></comment></comment><comment><USER7> This is only a little nit-picky but I found the image next to "Real time conversations" on your homepage very _very_ distracting. Otherwise looks pretty cool!

<comment><USER8> I kept feeling like some ajaxy widget was loading on the side and kept causing me to look over there. So I'd agree.

<comment><AUTHOR> I agree. I've changed the icon. Thanks for the feedback.

<comment><USER7> Awesome, looks good!

</comment></comment></comment></comment><comment><USER9> I like your idea - sort of wave-esque. The way you addressed the encapsulation of topics in an uncluttered interface is definitely important. You might want to put some thought into a logo redesign since it looks so much like Ubuntu's. Aside from that, congratulations on the launch! I'll be taking it for a spin later this weekend.

</comment><comment><USER10> Very good demo video on the product page. I'm actually surprised that I didn't kill it after 30 seconds as I usually do.

<comment><AUTHOR> Thanks. This is about the 8th iteration of me doing a demo video. I think I've got the length and the content just right this time, but it was a long road to get to this point.

</comment></comment><comment><USER11> You borrowed a lot of design from basecamp & co. No necessarily a bad thing though :)

</comment><comment><USER12> Why the comma after "Introducing"?

</comment><comment><USER13> maybe also get brainstrust.io

</comment><comment><USER14> Looks really cool, just from a sales / marketing perspective I'd change "Braintrust Features:" to "Braintrust Benefits:"

<comment><AUTHOR> Agreed and changed! Thanks for the feedback.</comment></comment>


Are Professors Obsolete? - <AUTHOR> http://www.antipope.org/charlie/blog-static/2011/10/are-professors-obsolete.html <comment><USER2> Facts are the machine language of knowledge. Hopefully you didn't spend university, in whatever major, memorizing long binary strings. You were meant to be climbing the abstraction ladder (say, to higher level programming and language design) and pushing the boundaries of what can be expressed and known.

DRY is for every discipline, not just software.

</comment><comment><USER3> Even if I can google programming answers or use stackoverflow, it probably takes 5 minute for me to get my answer. In other words, just because knowledge exists on the internet, it will take me a non-trivial time to retrieve these information.

Rote memorization allows me to bypass the latency and allows me to get things done faster. The rest of my brain is focused on retrieving all my experiences on how to debug or knowledge or how the framework I am using interact. I use Google and stackoverflow as needed.

My suggestion is to use SRS software like anki or supermemo and build your own deck of things to memorize such as how to get the size of an array, how to capitalize a string, etc. A good rule of thumb is to memorize anything you have to google twice. This will save you many 5 minutes of development time in the future.

<comment><USER4> Even more important: You need to know a lot before you can ask the interesting questions. You need knowledge in advance about algorithms and tools to have an idea about possible solutions and thus what you should Google or ask stackoverflow about.

</comment></comment><comment><USER5> No. A _good_ professor (and I'll leave it as a lemma to determine if the majority of professors meet this standard) is able to package up information and ideas in such a way that they take root and flourish in the student. A professor can take years of mastery, introspection, and mistakes and present a simple idea to a student in 5 minutes which will keep that student up all night.

I have very rarely found this on the Internet.

</comment><comment><USER6> Going beyond the question, one could say that Professors provide a view into the avenue beyond just business. I have personally found that being able to chat with professors about stuff beyond just Q&A and `what should I do` to be very insightful/helpful.

</comment><comment><USER7> Professors are not obsolete. Professors who spend their students' tuition on research and professional equivalencies instead of teaching and putting a majority of their efforts into their classes are.</comment>


Google endorses the Trans-Pacific Partnership - <AUTHOR> https://publicpolicy.googleblog.com/2016/06/the-trans-pacific-partnership-step.html <comment><USER3> This endorsement surprises me just because the TPP is so blatantly anti- consumer, pro-corporation.

I'm under no illusion that Google's anything other massive tech-savvy advertising company, but the self-serving corporate side of the company is usually hidden behind the pretty veneer of "tech for the people". Supporting a trade agreement like the TPP (especially given the lack of transparency in the process) lacks their usual subtlety.

<comment><USER4> Is it anti-consumer?

Convince me. I'm open. But I also want you to include counter-arguments. I want to hear both sides to reach a conclusion.

So far most anti-TPP arguments I've read are high on rhetoric and low on substance.

<comment><USER5> One specific part I don't like is that it would require signatories to enforce a DMCA-like ban on circumvention of DRM mechanisms. The US already has such a ban, but putting it in a treaty makes it worse (harder to repeal), and it would create one in countries which don't already have one. In general the intellectual-property related provisions seem not good to me, almost exclusively in the direction of more restrictions on what regular people can do with their data and devices.

<comment><USER6> There's essentially no enforcement of copyright law in many of the signatory states. Yes DRM kind of sucks, but rampant piracy also kind of sucks. The way the US does it I think is closer to optimal than the way Vietnam does it. That's why Google calls the provisions "balanced copyright enforcement."

<comment><USER7> > The way the US does it I think is closer to optimal than the way Vietnam > does it

Specifically with regard to DRM I'd say they are just as ineffective, but more consumer hostile.

</comment></comment></comment></comment></comment><comment><USER8> Wow. Even as the resident Google critic, I never expected Google would sink down to the level of endorsing the TPP. :/

This is a clear break from the EFF's position ([https://www.eff.org/issues/tpp](https://www.eff.org/issues/tpp)) and a strong indication of their increasing closed door ties with the Obama administration.

<comment><USER9> Or it could be an indication that Google is considering a bigger picture than the EFF is. TPP is a comprehensive agreement, covering a wide range of topics and activities. Here are the titles of the major components:

        Competition & Business Facilitation
        Competition Policy
        Cooperating & Capacity Building
        Cross Border Trade in Services
        Customs Administration & Trade Facilitation
        Development
        Dispute Settlement
        Electronic Commerce
        Environment
        Financial Services
        Government Procurement
        Intellectual Property
        Investment
        Labour
        National Treatment & Market Access for Goods
        Regulatory Coherence
        Rules of Origin and Origin Procedures
        Sanitary & Phytosanitary Measures
        Small & Medium-sized Businesses
        State-Owned Enterprises
        Technical Barriers to Trade
        Telecommunications
        Temporary Entry for Business Persons
        Textiles & Apparel
        Trade Remedies
        Transparency & Anti-Corruption

The EFF's opposition seems to be almost all based on just one of those components (Intellectual Property). It is quite possible that others, including Google, could see enough positives in the other components to outweigh the negatives the EFF sees in the IP component even if they agree with the EFF that those IP provisions are serious negatives.

This is especially true when you consider that much of the EFF's criticism of the IP section is over provisions that are already in effect in the US, and to a lesser extent in Europe. For instance, the US and Europe already have the 70 year copyright term, and the US and Europe already have anti-circumvention measures (DMCA in the US, the Copyright Directive [1] in the EU). Basically, many of the things the EFF objects to ALREADY APPLY to Google in most of Google's markets.

[1] [https://en.wikipedia.org/wiki/Copyright_Directive](https://en.wikipedia.org/wiki/Copyright_Directive)

<comment><USER10> _This is especially true when you consider that much of the EFF 's criticism of the IP section is over provisions that are already in effect in the US..._

A core purpose of the EFF is to keep IP law from spinning out of control, and to roll back the places where it already has (which is pretty much everywhere). This would be more nearly impossible than it already is if the US laws calcify into treaty form, which is a very good reason for them to oppose the TPP.

<comment><USER11> I doubt it is even in Google's business interests to do this either.

</comment></comment></comment></comment><comment><USER12> If these benefits are legitimate, why can't they focus on 5 or 10 specific objectives, and write them in some concise form like the Bill of Rights? Why does it need to be 2,000 pages and full of complex, obsfucated concepts and details?

Is the only way for progress to happen?

And even if the TPP is completely beneficial, it is still part of the philosophy of economic integration. Which at its heart is promoting the centralization of power and the diminishment of classical liberalism.

[https://en.wikipedia.org/wiki/Economic_integration#Stages](https://en.wikipedia.org/wiki/Economic_integration#Stages)

<comment><USER13> I don't really know, but I'm guessing it's a comprehensive deal. With tons of players with their own goals. So there's probably plenty of stuff like "your eggs aren't round enough to be labelled like that, but we'll allow them if we can call our meat like this". And "we don't like your copyright law, but in exchange for ... we'll allow it". Layer on exceptions and whatever else and the page count goes up real quick.

Even selling a company for just 8-figures is more than 5-10 pages and these deals far eclipse that amount and probably directly involve hundreds of people if not thousands.

<comment><USER14> I would be okay with minutia or details like these, making it a highly complex trade agreement. What i can never swallow is the secrecy. If it really is a good trade deal, why not make it transparent? They should be publishing it online, searchable and indexable with consice summaries for every section. This deal will literally affect billions of people, the people have a right to know.

<comment><USER13> The reasoning provided is that negotiators can't do their jobs well if every single thing they ask for is publicized then politicized. Unfortunately this ends up with a treaty that no one wants to renegotiate, a take-it-or-leave-it deal with a bunch of sunk costs.

<comment><USER15> There is a lot of middle ground between keeping something secret until there's something take-it-or-leave-it vs. publishing every "what if we do X, will you then let us do Y" back and forth. If more intermediate work products are unfit for public eyes, perhaps they shouldn't be going there in the first place.

<comment><USER13> OK but what would they publish? General ideas? The details being published would only lead to a mess as each company could advertise and make a fuss over things. Or other countries might do the same to shift things in their favor. The end goal for each country is to do things to benefit themselves, even if it means making hard decisions.

It's not like spy exchange negotiations or all other sorts of diplomatic processes are public. And it's not that things are unfit for the public per se, just that publishing each move isn't beneficial.

Yikes I never thought I'd be defending TPP-like processes. Personally I think we're (the world) not yet ready for this level of globalisation, unfortunately.

<comment><USER15> Spy exhanges and most diplomatic processes rarely gets enshrined in law. At least not without the resulting law being written in a far more open process afterwards (e.g. consider changes after the Troubles in Northern Ireland - <USER2> negotiations certainly didn't happen in the limelight, but the subsequent legal changes in the UK did happen in the open).

Whether or not "publishing each move" is beneficial depends on what "each move" is. I agree that at the most fine grained level, you don't want everyone to find out how far you were willing to go in conversations with one opposing negotiator, in case you need to negotiate similar clauses with someone else, for example.

But once you have agreed that you're giving X in return for Y, publishing that should not do much harm unless giving X is controversial. In which case the "harm" might be exactly what ought to happen.

Here's another example, which is more relevant:

During Norways EEA negotiations with the EU, the Norwegian press reported details pretty much daily. I just checked the archive of one of Norways largest papers to confirm my memory, and from '90 to '92, EEA was mentioned about 5000 times, and included things like government ministers informing the press about likely contents of Norwegian negotiation positions that had not yet been presented to the EU (but where the overall lines of the EU position was known - in one case I looked at, the minister confirmed that the Norwegian position would overall come close to what the EU had asked for in terms of regulation of granting operating licenses for EU companies in certain areas).

They certainly did not get access to everything, but they got regular briefings, and it contributed to ensuring the debate over whether or not to join the EEA shaped the negotiations, as the sitting, pro-EEA government got very clear signals about which concessions would cause the biggest problems with the opposition and/or cost them voters.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> > prohibits [participating countries] from requiring local storage of data

> It prohibits discrimination against foreign Internet services

As a Dutchman I can see why Google likes this.

<comment><USER17> Of the very many _terrible_ things in these trade treaties (most notably the massive expansions of copyright and patents), this is one of the few that seems helpful. It's a step towards not having to care (as much) about national borders on the Internet. A company providing a service accessible to the entire Internet should not have to worry about jurisdictions other than their own.

<comment><USER18> To offer an alternative viewpoint.

In Canada we have laws that protect personal health data. For instance my doctor cannot save my patient data outside of the country. This is a major pain for SAAS vendors, and I understand google's position. However, as a patient I appreciate knowing that my personal health data is not being sent around the world to other jurisdictions that may not have the same privacy and data protection laws that we do here.

So, I have to wonder do we have a problem of countries requiring local storage for reasons that are questionable? Or is this simply a preemptive strike ensuring companies like google continue their easy access to global markets? Given my example above I suggest its the latter.

<comment><USER5> I agree with this as an American as well. Obviously worries about the NSA getting your data are moot if the data was American to begin with, but there are a lot of other concerns, too. I wouldn't feel comfortable with my medical data being stored just anywhere, and I think it's entirely reasonable that the U.S. government should be able to put some restrictions on where it's stored.

It may be possible to come up with a list of reasonably similar and friendly countries where such restrictions are dropped, because they agree to similar protections, and form a kind of common data-storage zone. For example, I am probably okay with reputable Canadian or German companies being contracted to store sensitive data. We could have a treaty formalizing what that zone is and what common protections will be applied. But the TPP is not by any reasonable stretch of the imagination that zone. The TPP includes countries where being homosexual is a serious crime, for example. Is it really wise for the U.S. government to agree to be bound by treaty not to take measures to keep its citizens' medical data, student records, employment records, etc. out of those countries' hands?

<comment><USER6> That's the price of freedom: putting individuals in control of their data means they have responsibility to protect it. But is that such a bad trade- off?

Let me put it another way. _Someone_ has to decide where it's safe to send and store private data. Right now, in many cases, it's the government. Why do we think that the government is properly equipped to make that decision for us? Make sure you don't give your medical records to any shady companies. Corporations with international operations and teams of lawyers are probably better equipped to determine where data can be safely secured than some random regulator.

<comment><USER5> How do you propose that I personally reach agreements with corporations on storage of my medical records? I've visited a hospital once in my life, which involved doing business with dozens of different "businesses", from the hospital to the anesthesiologist, all of whom collected records, and none of whom I was in a condition to negotiate with on an individual basis (nor would they have been interested in negotiating with me regarding their data- protection policies anyway). And I don't have any faith in corporations determining where data can be safely stored; I think they will primarily go with where it can be most profitably stored, which is not necessarily the same question. I would feel much more comfortable if the hospital, anesthesiologist, and everyone else were required to store my data in accordance with a regulator's oversight. And preferably in the USA or a friendly western country. The companies can of course exercise _additional_ caution if they feel it warranted, but I'd like the government to legislate some minimums.

</comment></comment></comment></comment></comment></comment><comment><USER14> Its always funny when people naively forget that Google, Apple, etc. are megacorps, not some startup ventures. They are the man. This endorsement doesn't surprise me one bit.

<comment><USER19> Is this a satirical comment or are you actually using the word "megacorps" seriously? Just because a company is big doesn't mean it is nefarious. You should explain why these companies are evil and how them endorsing TPP is a bad thing.

<comment><USER20> I take "megacorps" to mean large, established companies with strong political connections. Companies that exert influence both over people's lives and governments; not necessarily malicious.

<comment><USER14> Precisely. Case in point: Google's various attempts at tax avoidance in Europe. I respect them for trying it, I'm sure every other established corporation is doing the same thing, or trying to at least.

</comment></comment></comment></comment><comment><USER21> The announcement starts by patronizing the reader ("most of us imagine container ships ..."), then proceeds to use a "for the children"-type argument ("small business"), directly followed by "for freedom!" argument with little to no substance.

      Trade agreements like the Trans-Pacific Partnership (TPP) are beginning to recognize the Internet's transformative impact on trade.

Sure, in the sense that the TPP is designed by and for corporations, like Google, who don't want stuff like, say, the EU, getting in their way when trying to defend the users' rights (e.g. right to be forgotten). The paranoid in me also thinks this is in fact an agreement between organizations like Google and the US government to "legally" syphon foreign data directly from US soil, but there is of course absolutely not enough transparency to substantiate this either way.

I clearly don't support TPP in the first place, but I find this announcement from Google borderline insulting, TPP or not. The condescending tone alone makes me cringe, but the whole "we're fighting for freedom" cover story is just disgusting.

</comment><comment><USER22> I would better trust EFF on this: [https://www.eff.org/issues/tpp](https://www.eff.org/issues/tpp)

<comment><USER23> Everyone links to that general page, but I'd think the EFF analysis on how it affects digital rights specifically might be even more persuasive for this crowd:

[https://www.eff.org/deeplinks/2015/12/how-tpp-will-affect- yo...](https://www.eff.org/deeplinks/2015/12/how-tpp-will-affect-you-and-your- digital-rights)

</comment></comment><comment><USER24> >The TPP advances other important Internet policy goals. It prohibits discrimination against foreign Internet services

This is really it, I think. Makes sense. They don't want another China to happen to them.

</comment><comment><USER25> I see that the point about countries not being allowed to require local storage because

>These provisions will support the Internet's open architecture and make it more difficult for TPP countries to block Internet sites -- so that users have access to a web that is global, not just local.

I do however believe that there is a aspect that has been overlooked here. Many cooperations, private and governmentally controlled, handle sensitive data. Having a SLA with Microsoft, Amazon or Google that states that this highly sensitive data is _ONLY_ to be stored in specific data centers is the only way for a lot of non flexible IT departments to regain some control over their data. This is a widely popular demand. I can only see this as the "X eyes" with USA in charge removing one more hurdle in their way to total information control.

<comment><USER26> TPP binds governments from requiring local data storage, not private contracts.

<comment><USER25> Thank you for clarifying. The argumentation still stands though.

<comment><USER27> Not really, there's a big difference between state enforced rules about where data can be and client requested rules.

<comment><USER28> Except that in a functioning democracy, state-enforced rules are client requests from an entire population.

<comment><USER27> It's nice to have the freedom to store your data where you'd like.

<comment><USER28> It's nice to know my business relations store my data where I'd like them to.

<comment><USER29> Well then pick business relations that do so, that's what private contracts are for.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER30> In another discussion about the TPP I was pointed to this link: [http://economixcomix.com/home/tpp/](http://economixcomix.com/home/tpp/)

I thought it was a very enjoyable way to learn about the agreement and why one might oppose it.

</comment><comment><USER31> Disappointing. The were a loud voice against SOPA/PIPA. All it takes for them to do an about-face is a bill that includes a few bits about protecting multinational internet companies.

</comment><comment><USER32> It's sad to see that most of the commenters here decided to discuss it as a news tweet ("Google endorses TPP which, as we know, is terrible") instead of a text that makes some points on the matter. Seems like most don't even address the text in any way — which means people don't stop for a second to doubt their opinion on TPP, or even discuss arguments of the "enemy camp".

</comment><comment><USER33> Of course they do. They were probably involved in drafting it.

<comment><USER34> Julian Assange and Edward Snowden second that. The extent Google is tied to Obama administration is astonishing.

<comment><USER35> I am sorry, but hours-old accts commenting on highly controversial threads, spouting poorly cited 'almost conspiracies' doesn't seem helpful. I may be completely off, but that is how I feel; this is something that has become far too common on HN. Am I in my own world on this?

<comment><USER34> Judging other people by the age of their accts instead of their reasoning, you are in your own perfect world. Here is a simple example which shows how liberal Google is (there are plenty of them): [https://www.youtube.com/watch?v=PFxFRqNmXKg](https://www.youtube.com/watch?v=PFxFRqNmXKg)

</comment></comment></comment></comment><comment><USER36> This comes across to me as nothing more than proof that Google is doubling down on HRC being elected now.

<comment><USER8> I'm sure they are, and Google's definitely worked closely with Hillary before. Though at last public claim, even Hillary said she was against the TPP.

<comment><USER36> She said so after she said she supported it. It will be a matter of time until she "evolves" on that matter again. Mark my words, if she's elected.

<comment><USER37> > She said so after she said she supported it.

She said she supported it while it was being negotiated, and later (as a candidate, when the final text was available) said she opposed the final form, and that it no longer had the specific features she expected the final form would have when she supported it.

<comment><USER36> We can debate the "he said she said" until the cows come home. It is no mystery that HRC supports and champions oligarchic policies. The TPP is being pushed by the Obama administration who have added their support to her presumptive nomination. There's no fooling anyone here about what may very well happen.

The timing of this is peculiar, but if it has nothing to do with HRC, we'll simply have to wait and see if that is the case.

<comment><USER37> > The timing of this is peculiar, but if it has nothing to do with HRC

If it had to do with HRC, they could wait; if anything connected to the election, it has to do with their actual support for TPP and concern that no successor to Obama will approve it, so their best chance is for it to get passed and signed this year (given the statements already made by Republican leaders that that isn't going to happen _before_ the general election, probably in the lame duck session after the election.)

<comment><USER36> >If it had to do with HRC, they _could_ wait (Emphasis mine.)

But they won't, because they're _doubling down_.

</comment></comment></comment></comment></comment></comment></comment><comment><USER38> What are some alternatives to google search. I think I can get away from the rest of their products in the coming few months.

<comment><USER8> DuckDuckGo is pretty fantastic all around.

Bing is alright, to be honest (some of their data widgets are more comprehensive than Google's), but you will find times with Bing that you are sad with the results.

<comment><USER38> DuckDuckGo it is then. Bing is still Microsoft's product, so thats a bummer.

<comment><USER39> DuckDuckGo's killer feature is its thousands of !bang search shortcuts. They integrate with content sites' own search forms, so the results are similar to but often better than Google's site:example.com searches because the sites' have more domain knowledge over their content.

[https://duckduckgo.com/bang](https://duckduckgo.com/bang)

</comment></comment></comment></comment><comment><USER40> Disappointing. But that's Google these days.

</comment><comment><USER41> I support TPP for geopolitical reasons, I am sure the Elites in the corridors of Power in Google do it for "access" and ability to sue the shit out of the gubmints that come in their way.

<comment><USER42> What are your geopolitical reasons?

<comment><USER41> I concur with Ian Bremmer's view on TPP. [https://www.youtube.com/watch?v=jniOW9G9TQA](https://www.youtube.com/watch?v=jniOW9G9TQA)

At a fundamental level, US defining and guiding the world economic order as it has done with Bretton woods.

<comment><USER43> I couldn't listen to that video for long, it wasn't quite bearable. Stopped after 2 minutes.

Can you explain it from your point of view? These first 2 minutes repeated 'strong United States' about five times already or so. Is that what you consider 'geopolitical reasons'?

<comment><USER44> This comment seems very holier-than-thou to me. Did you intend it that way? If not, it still seems a bit inconsiderate to ask someone to summarize a video a mere _8 minutes_ in length, which they were kind enough to link.

</comment></comment></comment></comment></comment><comment><USER45> Unlike most of the corporations in America, at least Google is transparent about its decisions.

</comment><comment><USER46> Given the intense regulatory pressure (especially in the EU) that Google is finding itself under, this is unsurprising. The TPP would greatly help in easing 3 strategic concerns, which are:

1) Data protection regulation that would severely impact Google.

Google is kinda caught in the crossfire here. The EU is upset at the US intelligence community (Post-Snowden), and wishes to have some assurance of protection for their citizen's communication. The USG does not see why it should do this, and the data protection regulation is the EU's way of saying: If we can't get you to cooperate by being friendly, we will have to pass legislation that will hurt US companies; they can then perhaps pay lobbyists to change your mind. So it is not really Googles fault, but they are caught in the middle.

2) Anti-Trust considerations.

Google is extremely dominant in the European market, much more so than in the US. See #3. The momentum for an Antitrust case in the EU is high.

3) Industrial policy considerations.

Right now, the EU does not have an "Internet powerhouse"; there simply are no modern Internet giants in the EU. Interestingly, both China and Russia have "inadvertently" created local giants (mostly by censorship, which has had the strange side-effect of being equivalent to import tariffs, but for Internet services -- e.g. an accidental protectionist measure that allowed local competition to emerge). Some voices have been heard in recent months that advocate that the only way the EU won't get entirely left behind is industrial policy toward creating an internet giant. This could even mean blocking US- based giants for a while.

If TPP passes, Google will have a very powerful tool to wield against these three concerns. It is entirely rational of them to support it.

</comment><comment><USER47> Speaking of Google and politics, Julian Assange claims that Google has made a deal with Hillary Clinton and is backing her [0]. That's not far-fetched considering how intertwined Google and White House are. The Intercept has a really detailed analysis and a chart showing how Google and WH share so many lobbyists and executives [1]. Google wants to keep Democrats in the White House so they can continue their influence over various decisions and FTC (which squashed the investigation into Google's anticompetitive practices).

And just today, Google was accused of whitewashing Hillary Clinton's record and not showing negative autocomplete terms [2].

Google's "don't be evil" mantra is just a way to inoculate themselves from criticism. If anyone accuses them of anything unethical or evil or inappropriate, they can always claim how they are "not doing anything evil" because that's their policy. It becomes circular logic.

Assange has exposed Google's close ties to the State Department in the past as well. For example, give this NY Times article a read [3]. If you want to see how deep the rabbit hole goes, Assange also published a book about his meeting with Eric Schmidt and Jared Cohen of Google [4] and he published the complete transcript of their meeting. He also covered how Jared Cohen, formerly a State Department employee, was involved in regime changes around the world while at Google.

Assange also states that in 2015 former Google CEO Eric Schmidt launched "The Groundwork," a startup specifically designed to get Clinton elected. [5]

Finally, Clinton's email release also confirmed that Google was involved in helping Syrian rebels who were trying to bring down the Syrian regime [6].

[0] [http://www.belfasttelegraph.co.uk/news/world-news/google- wor...](http://www.belfasttelegraph.co.uk/news/world-news/google-working- closely-with-hillary-clinton-presidential-campaign-julian- assange-34780998.html)

[1] [https://theintercept.com/2016/04/22/googles-remarkably- close...](https://theintercept.com/2016/04/22/googles-remarkably-close- relationship-with-the-obama-white-house-in-two-charts/)

[2] [http://www.businessinsider.com/donald-trump-google-search- vi...](http://www.businessinsider.com/donald-trump-google-search-viceo-2016-6)

[3] [http://www.nytimes.com/2013/06/02/opinion/sunday/the- banalit...](http://www.nytimes.com/2013/06/02/opinion/sunday/the-banality-of- googles-dont-be-evil.html)

[4] [https://wikileaks.org/google-is-not-what-it- seems/](https://wikileaks.org/google-is-not-what-it-seems/)

[5] [http://dailycaller.com/2016/06/09/julian-assange-google- is-i...](http://dailycaller.com/2016/06/09/julian-assange-google-is-in-bed- with-hillarys-campaign-video/)

[6] [http://www.independent.co.uk/life-style/gadgets-and- tech/new...](http://www.independent.co.uk/life-style/gadgets-and- tech/news/google-syria-rebels-defection-hillary-clinton-emails- wikileaks-a6946121.html)

<comment><USER48> That autocomplete thing looks innocent to me. They seem to block all "firstname lastname liar" and "firstname lastname criminal" autocompletes.

<comment><USER47> Why did they block the word "indictment" but not "racist"?

<comment><USER48> Maybe they overlooked it. I wouldn't be surprised if it gets blocked in the future.

</comment></comment></comment></comment><comment><USER49> Let me translate this into truthiness: even though TPP and TPIP favor multinational corporations over the sovernty of countries and what is best for people, Google is promoting what is best for their profits.

</comment><comment><USER50> How many Google engineers would have to strike at once to get them to change their tune? 80%?

Things only start changing when it gets bad enough to walk away.

</comment><comment><USER51> What? So Google supports all the horrible IP provisions there?

TPP must be ditched because it's an abomination in the current form, and it's not fixable because of "fast track".

Google should officially change from "don't be evil" to "be evil" now.

</comment><comment><USER52> Seems like they agreed to the deal to have a bigger influence on future trade deals. I don't think they like this deal particularly. But they are swallowing the pill for having a say in the next one.

</comment><comment><USER53> The free flow of information must be why they released this tidbit on Friday afternoon?

If they are so proud, surely a banner on Google is appropriate. It's for a good cause after all.

<comment><USER8> This is how you dump a story. Everyone's on the way out the door for the weekend. Google will publish something "amazing" on Monday morning to excite everyone to the level that people forget Google sold them out three days prior.

<comment><USER53> It's how you publish _bad_ news, specifically.

</comment></comment></comment><comment><USER42> "Don't be evil" seems to be more of a general guideline than a rule these days.

<comment><USER54> Motto. Or slogan.

</comment></comment><comment><USER55> I guess they want to pay even less taxes... Ireland is getting too expensive perhaps?

</comment><comment><USER56> if Trump wins, there will be no TPP, at least if he sticks to his campaign promises. We all accidentally get a gift if he makes it to the Oval Office.

<comment><USER57> According to [https://ballotpedia.org/2016_presidential_candidates_on_the_...](https://ballotpedia.org/2016_presidential_candidates_on_the_Trans- Pacific_Partnership_trade_deal) all of the remaining candidates oppose the TPP.

<comment><USER58> _At the moment._

 _Senator_ Obama opposed quite a few things as well.

<comment><USER57> >At the moment.

Well, yes, people lie. My point was that Donald Trump wasn't a special case in terms of his position on the TPP.

</comment></comment></comment></comment><comment><USER59> "okay, be evil."

</comment><comment><USER22> First they manipulated search suggestions in favor of Hillary and now this :(

<comment><USER60> [citation needed]

<comment><USER13> Probably referring to this: [https://www.youtube.com/watch?v=PFxFRqNmXKg](https://www.youtube.com/watch?v=PFxFRqNmXKg) Was on HN but flagged. Google replies: "Google Autocomplete does not favor any candidate or cause. Claims to the contrary simply misunderstand how Autocomplete works. Our Autocomplete algorithm will not show a predicted query that is offensive or disparaging when displayed in conjunction with a person's name. More generally, our autocomplete predictions are produced based on a number of factors including the popularity of search terms."

Seems like "criminal" and "indictment" are filtered out, which happens to benefit Hillary but seems to apply to other people. I tried a indicted people and it didn't autocomplete.

</comment></comment></comment><comment><USER61> Boycott Google</comment>


Learning Machine Learning from 187 Quora Questions - <AUTHOR> https://unsupervisedmethods.com/learning-machine-learning-and-nlp-from-185-quora-questions-cebe42e47da8#fklj4 <comment><USER2> I really thought this was going to be a post where someone trained a model to output "ML code" from Quora answer snippets, though I guess in that case StackOverflow would've been a better source.

<comment><AUTHOR> Interesting thought, but there isn't enough ML code on Quora to do something like that even on a lark.

</comment></comment><comment><USER3> Upvoted for later reading, to never actually read like a hundred other lists and tutorials before - _in case anyone was still wondering why these kinda posts are frequently on the front page._</comment>


Backblaze Hard Drive Stats Q3 2019 - <AUTHOR> https://www.backblaze.com/blog/backblaze-hard-drive-stats-q3-2019/ <comment><USER2> These stats are wonderful and make me really appreciate the culture of the company. I've been considering becoming a customer because of these posts, since they reflect a lot of pride in the craft and care for the community.

But I'm stuck on one thing. Does Backblaze offer a solution for Linux backup? I've got an NFS server running that I use for home storage that I want to back up - but looks like Backblaze is only offering a Windows or Mac client.

Maybe the business version would work, since it claims to support NAS backup. But then the pricing seems lower than the personal edition (60$/computer/year = 5$/month < 6$/month) - unless that's implying that every computer that accesses the NAS is part of the fee?

So I guess: is there a reasonable Linux offering for home users from Backblaze? If not, what service do folks suggest?

<comment><USER3> I've been using restic with the Backblaze B2 backend for a home server backup, which seems to be as close as Backblaze will ever get to having a Linux client.

My rough numbers are 450GB stored monthly, 4GB downloaded monthly, 90k stored files, 385,000 individual transactions, which ends up costing about $2.25 in storage fees, $0.25 for transactions, and $0.10 for download bandwidth.

<comment><USER4> One thing that has been keeping from backing up more content to B2 has been finding an appropriate encryption strategy, and it looks like Restic manages encryption the remote backup repository automatically?

If I'm reading this correctly, Restic + B2 sounds like an absolutely godsend!

<comment><USER3> That's correct, restic operates under the assumption that you cannot trust where it's storing the data.

Some more (informal) analysis on restic's crypto was done here:

[https://blog.filippo.io/restic-cryptography/](https://blog.filippo.io/restic- cryptography/)

</comment></comment></comment></comment><comment><USER5> There was a post from BackBlaze a year (?) or so back where they commented on the Toshiba low failure rate with something along the lines of "they seem really reliable, but we buy in bulk and just don't have enough offers at low price for those, otherwise we would buy a lot of them".

Well I run a couple dozen Synology NAS in professionnal setup, as well as two in personnal setup (mine and my parents'), and ever since that post I made the experiment of having almost 50% of all drives be Toshibas, and I have to say they do seem to be much more reliable (on the scale of "why do every other drive from Seagate and WD keep dying first, and often their replacement dies first too").

It is still a scale of use where it's mostly anecdotical rather than verifiable data, so don't take this fun comment for more than that. But I suspect a lot of people reading these posts are not interested for some large scale setup or anything like that but rather to know which drives to put in their home computer or NAS, and honestly I can highly recommend the Toshiba for that. They do tend to be a bit more expensive (around 10% more ? I buy them from ldlc.com and grosbill.com , french IT stores, no bulk buying or anything like that)

Of course no matter the brand never expect no failure and a Toshiba drive may just as much die in the first ten minutes so always plan for it.

<comment><USER6> Yev from Backblaze here -> Yea, the Toshiba drives are great! If the price was lower, they'd likely play a larger role in our hard drive mix!

<comment><USER7> If they need replacing less often, then presumably you'd be willing to spend more on them, so do I infer that they are sufficiently more expensive to wipe out the benefits of greater reliability?

<comment><USER6> Yev here -> Yes, that's basically the trade-off for us. Is the drive available and reliable vs. how much does it cost. One of the reasons we have so many Seagate drives is because they are available in abundance, are affordable for us, and have failure rates in-line with the other drives that we're testing.

<comment><USER8> I'm theorizing that one of the things Backblaze has optimized is the labor cost of drive replacement, yes?

So the reliability may be "worth more" to someone who pays a lot for remote- hands in a server farm somewhere, but not in this case.

<comment><USER6> Yev here -> we do have remote hands available in some cases, but we try to have our own teams in place where possible, but yes - it's one of the things we optimize on our end!

</comment></comment></comment></comment></comment></comment><comment><USER9> This is my favorite kind of content marketing.

</comment><comment><USER10> Shout out to Seagate, every drive that my friends and I have bought from them have eventually failed, good to see that they fail in non-consumer use too, not just me! Stick to the Western Digitals.

<comment><USER6> Yev here from Backblaze -> The Western Digital drives that you've purchased will fail too. That's part of the whole point of these reports, all of the drives eventually fail out or reach a state where we have to replace them - it's not any one specific manufacturer. That's part of why having a backup is so important, even the SSDs in newer machines will eventually go wonky.

<comment><USER10> Then I stand corrected. Is there anyway to avoid drive failures or should we accept the fact that they will break eventually? Also, the data is interesting and appreciated.

<comment><USER11> Buy a RAID enclosure or setup a home RAIDed server and do backups. You cannot avoid it but you can mitigate the damage.

<comment><USER12> I really wish that consumer OSs would mandate RAID (mirroring at the minimum), requiring additional steps to install on unprotected storage. This should have been implemented even back during the early days of consumer hard drives, or at the minimum have a warning on boot that "Your data is stored on temporary storage -- go [here] to configure redundancy". At least something that puts it front and center that storing data on a single hard drive will eventually lead to data loss.

Also would be nice to have a similar warning to "Your data has not been backed up in [x] days".

<comment><USER13> Ugh, god why? Why must people in our industry daydream about over-complicating things all the time? Very few use-cases benefit significantly from mirrored disks. The amount of data a user actually cares about having a backup of is often significantly smaller than the OS and related garbage on a disk that they can do without. Besides which, a local mirror isn't a good backup anyway!

<comment><USER12> My thinking is that this would serve a similar purpose to the trend for web browsers to warn users of insecure websites -- the more "in your face" the warnings are, the more incentive there is for providers to be secure.

I've had friends / relatives experience drive failure a few time in the past, and the look of horror they have when there is very little I can do to help them recover their photos etc. is something that I hate seeing.

And having the OS give a simple warning (that can be dismissed, with a "don't show me this any more" checkbox) would not over complicate things, and may end up saving some people's data.

Really no different than the current warning that Windows has, when you don't have antivirus installed. Or the fasten seatbelt signal that comes on the dashboard of your car when you start it. Or the flashing red light that gets added to some stop sign controlled intersections.

Also it would be nice if there was a standard way that backup software could inform the OS of backup status (this way it would serve as a secondary check in case the backup software's internal reporting fails to notify the user of bad backups). Just a little nice-to-have.

(I'm not advocating for this to be a legal requirement, just a nice feature if any OS vendor wants to add it).

<comment><USER13> > My thinking is that this would serve a similar purpose to the trend for web > browsers to warn users of insecure websites -- the more "in your face" the > warnings are, the more incentive there is for providers to be secure.

Another thing I hate about the industry today: being hostile to the user and trying to force them to use their computer how you want them to use it.

> I've had friends / relatives experience drive failure a few time in the > past, and the look of horror they have when there is very little I can do to > help them recover their photos etc. is something that I hate seeing.

Then teach them about proper backups. It doesn't take a rocket scientist to understand "don't keep all your eggs in one basket". Or if you're going to implement some stupid forced user-hostile scheme at least use something that actually qualifies as a backup.

> And having the OS give a simple warning (that can be dismissed, with a > "don't show me this any more" checkbox) would not over complicate things, > and may end up saving some people's data.

Here's what will happen: the user will dismiss the dialog without even reading it. We have decades of experience showing us this. Users have learned that "warning" dialogs are meaningless precisely because of crap like this. Oh yeah, and they're super annoying.

> Really no different than the current warning that Windows has, when you > don't have antivirus installed.

Exactly my point.

<comment><USER14> I've looked into RAID at home but came to the conclusion that there are plenty of easier/cheaper ways to do backup. I just use USB drives, have a couple rotating Time Machine backups plus Backblaze. (Plus, when I think of it once a year or so, I have one more copy of my main data disk that I keep in a fire box.)

I don't really use Windows but you can do something similar--though in my experience it's not as simple.

I don't really care if I avoid _any_ downtime. So long as I have belt and suspenders backups, I'm pretty comfortable.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> I love these articles - I'd also love an update to the cost curves: [https://www.backblaze.com/blog/hard-drive-cost-per- gigabyte/](https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/) (2017)

It looks like after stalling in 2017-2018, $/GB has dropped again - [https://jcmit.net/diskprice.htm](https://jcmit.net/diskprice.htm) \- but JCM doesn't have the large sample sizes Backblaze does.

</comment><comment><USER16> Interesting how the checksumming process sounds very much like zfs's scrubbing process. One of the reasons I trust zfs with my large data volumes is because it proactively looks for problems and fixes them. (and most filesystems really can't look/check)

</comment><comment><USER17> > By increasing the shard integrity check rate, we potentially moved failures > that were going to be found in the future into Q3. While discovering > potential problems earlier is a good thing, it is possible that the hard > drive failures recorded in Q3 could then be artificially high as future > failures were dragged forward into the quarter. Given that our Annualized > Failure Rate calculation is based on Drive Days and Drive Failures, > potentially moving up some number of failures into Q3 could cause an > artificial spike in the Q3 Annualized Failure Rates. This is what we will be > monitoring over the coming quarters.

Wouldn't survival analysis on interval-censored data handle this problem automatically? All of your observations of failure presumably are actually interval data, where all you know is that the drive failed sometime in between the last good check and the first bad check. Then it doesn't matter if some time periods have large intervals and others have small intervals, that just affects the precision of estimates.

</comment><comment><USER18> The only reason I use other storage provider than backblaze is simply because of the benchmark done by one of the more modern backup tool author.

[https://github.com/gilbertchen/cloud-storage- comparison/blob...](https://github.com/gilbertchen/cloud-storage- comparison/blob/master/README.md)

Can anyone from backblaze say anything about their performance compared to other vendors?

The pricing is certainly ahead of others, so I would use if the performance is comparable to some of the leading group tested there.

<comment><USER6> Yev here -> well that chart hasn't been updated in a while. For starters we're just $0.01/GB for downloads (we dropped the price last year). Our performance is generally pretty good, and we're partnered with cloudflare (free egress) if you need more umph. But most of the time folks don't have any issues with just our regular service.

</comment></comment><comment><USER19> I must have missed it somehow, but what is the difference between boot drives and data drives in a typical Backblaze server other than the boot drives store the OS? Obviously you don't need 8TB capacity solely for the OS, I'd assume you also store user data on boot drives? In which case why is there a distinction?

<comment><USER6> Yev here from Backblaze -> Mainly it's the OS and log files - the reason we make a distinction is that the boot drives typically do not have as much load as the data drives, so it wouldn't be a real 1:1 comparison.

<comment><USER19> Thanks! What's the capacity and utilization rate of those boot drives, then?

<comment><USER20> Andy from Backblaze here: The capacity of the boot drives ranges from 80 to 500gb typically. They are mostly hard drives with some SSDs added recently. We are switching over to SSDs for boot drives. The workload is reasonable, but on the higher side as they not only boot the systems, they also are used to store log files temporarily - so lots of reads, writes, and deletes. Since we only have a little over 2,000, any data we published not be very accurate. If you are really interested, the boot drive data is in the data files we publish each quarter.

<comment><USER19> Thanks for the pointer! :)

</comment></comment></comment></comment></comment><comment><USER21> I wonder If Backblaze will eventually offer some sort of consumer solution to Backup. An App on iOS, Android, Mac and Windows that simplifies backup to your Backblaze NAS, that Backup to B2 as well.

</comment><comment><USER22> I've been following these since maybe 2016? I don't remember when they started. It's striking to note that for as long as I recall, HGST still holds the crown of lowest annualised failure rate across the board.

</comment><comment><USER23> Off-topic but sort of related: is there a hard drive tower designed specifically for a pool of SSDs? The one I have for 8 3.5 bays could easily fit twice as many of the little SSDs...

<comment><USER8> The term "mobile rack" will find some of the densest ways to shove, for instance, eight 2.5" drives into a single 5.25" bay. That can get you some serious density in one of those "cdrom duplicator" tower cases that's all drive-bays, but heaven help you on the controller side.

</comment></comment><comment><USER24> And is there _any_ reason to not trust WD Red drives in my home NAS?

<comment><USER5> Your question makes no sense. Any drive can and will fail, and no matter the brand it may fail immediately or it may last years.

The rule is to avoid all of your drive being from the same factory date/batch, even if they're all from the same brand order from different stores or whatever to help make sure in case of a defect they're not all affected.

Also for home NAS ensure you have redundancy (a proper raid level, avoid JBOD and Raid-0), and have backups. Raid is not a backup.

<comment><USER25> I would add data integrity to the list, in addition to redundancy and backups.

ZFS is great in this regard: redundancy, data scrubbing to ensure data integrity and built-in snapshotting and data replication features (zfs send- receive). Can't believe how I got by without ZFS in my earlier years of data hoarding.

<comment><USER5> Absolutely right, most home Nas don't have zfs but can use btrfs, which helps protect against bit rot and offers proper data scrubbing.

</comment></comment></comment></comment><comment><USER26> Is Blackblaze used mainly as a backup solution? Can anyone give me a use case?

<comment><USER6> Yev here from Backblaze -> the company started by providing unlimited online backup, and that's a great industry for us. About 4 years ago we released Backblaze B2 Cloud Storage, which allows developers or sysadmins or enthusiasts to directly upload/retrieve data to/from our data centers. Our core competency is data storage - so while most folks do use us for a backup (either of their Mac or PC on the consumer side or servers/NAS devices with B2 Cloud storage) - what we really do is store and retrieve data.

<comment><USER27> So you guys basically wrote your own Ceph?

<comment><USER6> Sort of, but not really, we just wrote APIs that let people talk to our pods directly. You can read about our architecture here ([https://www.backblaze.com/blog/vault-cloud-storage- architect...](https://www.backblaze.com/blog/vault-cloud-storage- architecture/)) and check out the APIs and how we built them here (backblaze.com/b2/docs/).</comment></comment></comment></comment>


Vitamin D Supplements Aren't Living Up to Their Hype - <AUTHOR> https://www.sciencenews.org/article/vitamin-d-supplements-lose-luster?tgt=nr <comment><USER2> Note, that it actually doesn't say _it 's all hype_ it really does say it's more complicated. And vitamin D for bone loss in elderly people is a thing. This article doesn't say "stop taking vitamin D"</comment>


Vim Cheat Sheet - <AUTHOR> https://linuxacademy.com/blog/linux/a-vim-reference-guide/ <comment><USER2> Wait, there's something wrong with this picture. It says just "Ctrl" instead of "Ctrl-R" and "Ctrl-V".</comment>


Bitcoin Still Confuses Bankers - <AUTHOR> http://www.bloombergview.com/articles/2015-03-03/bitcoin-still-confuses-bankers

<comment><AUTHOR> "The sooner people give up the hope that bitcoin will skyrocket in price, the sooner they will be willing to spend bitcoins in everyday life, the way they now spend dollars. The quicker bitcoin as an investment dies, the quicker bitcoins as currency can come to life."</comment>


What would you do? Or turning a website into a business. - <AUTHOR>

A couple months ago I was approached by a good friend (we will call him Sam) who was taking a job with a large company with strict IP rules. Sam owned a free classifieds website that he had been maintaining for years but he needed to get rid of it due to the IP rules at his new company. So I purchased it in full for a very nominal amount of money.
The site is located at http://www.frimp.net and it currently has 5k registered users, 17000 postings and 25-30k hits per month. The current operational costs are about 45.00 per month. It is not stellar but I was thinking it was better than starting from scratch.
Does the community at large have any words of advice or suggestions on how to turn this web site into something valuable?
It is written in Java/Spring and I am about 75% through with a rewrite of the existing functionality  in python/django.
Any help will be very much appreciated. <comment><AUTHOR> Anything other than UI?

</comment><comment><USER2> Better UI.

<comment><AUTHOR> This is a secret... but... Here is a very, very rough version of the new UI.

<comment><USER3> It is such a secret that we cant even see it?!

<comment><AUTHOR> Yes, top secret... lol

Here is a link to my prototype: <http://www.thesaverssecret.com/>

One think I forgot to mention is that this website has very good rankings in google for the following terms:

online yard sale - #1 online garage sale - #3 online sale - #4

<comment><USER2> Honestly. Your best bet would be to get a designer to design it for you, rather than use something that reeks of a template.

Best way I've found when working with designers, is to sketch exactly what _you_ think it should look like. Include colors or themes (from pictures if you can) and then draw some sort of 'mascot', like a cuddly animal, or a lightbulb. This gives the designer most bits of the puzzle, which in turn allows them to (more easily) design you a unique website.</comment></comment></comment></comment></comment>


Show HN: Avoid posting something your friends have seen (Built today on Singly) - <AUTHOR> https://chrome.google.com/webstore/detail/biigofehldaoepafmpeibadambofikaf

<comment><AUTHOR> Currently of limited utility; depending how many people you follow on Twitter, links can fall out of history pretty fast. More networks and speed improvements will come as the Singly API improves.

Singly's API, by the way, is currently super-alpha. Apologies for confusing things and any load issues.

</comment><comment><USER2> If you avoid dupes, how will any topic trend?

<comment><AUTHOR> If the topic is cool enough, you'll post it anyway, which would be a win for signal:noise</comment></comment>


WorkiLeaks: How to Be a Workplace Leaker Without Getting Caught - <AUTHOR> http://www.wired.com/threatlevel/2012/04/workileaks/?utm_source=twitter&utm_medium=socialmedia&utm_campaign=twitterclickthru

<comment><AUTHOR> Can a mod strip all the social media junk in the URL? I didn't think of that when I submitted it.</comment>


How Do You Stack Up Against Other Developers? - <AUTHOR> https://strictlydev.com/welcome

<comment><USER2> These things are interesting and good ego-candy (or not), but one thing that always bothers me about sites that import your projects from Github is that they never seem to look at the open-source projects for which I'm a core team member (unless it happens to be one with its canonical repo is under my personal account).

Like, when someone is a core team member working on rails, that's a big difference from simply having a fork of rails under your own username. But these sites would never look at the core rails members because the official rails repo is under the rails namespace, not anyone's personal account.

Because of this (and many many other factors), it would be hard to ever use something like this to actually assess a given developer's chops.

<comment><USER3> Hell, what about non-GitHub? I have tons of code on <http://git.xfce.org> that presumably this site wouldn't pick up at all.

<comment><USER4> We're working on adding BitBucket and Ohloh soon

</comment></comment></comment><comment><USER5> I was very interested to see this. I have been working on a similar idea but without the github "centric-ness".

My one is called codefamous.com and has a similar tag idea to yours but I've gone a bit more flexible and allowed users to create tags and tag groups that they like and pinch them from each other's profiles. See my profile on it to give you an idea: <http://codefamous.com/bcg>

Yes everyone, I know I need the services of a designer and gravitar and auth api integration. It early days yet as I'v only just knocked her up over the last couple of weeks.

At the moment I'm just really interested in what tags and groups evolve.

</comment><comment><USER6> Please don't make me create yet-another-account to see a number.

</comment><comment><USER7> Why are they emailing developers out of the blue to "claim" their profile?

Please don't base your service off of crawling sites and emailing people who don't ask for either.

<comment><USER8> I just got one of these spam emails too. Gordon L. Hempton, I believe you are promoting your business in violation of the CAN-SPAM Act. Please stop.</comment></comment>


Networking for the Shy Entrepreneur - <AUTHOR> http://www.nytimes.com/2009/06/23/business/smallbusiness/23toolkit.html <comment><USER5> A dating coach has a 3C's rule that rates a venue for meeting women that also may apply to networking for shyer people:

Conversation-friendliness: How easy is it to hold a conversation (no shouting, no loud music, seating (if needed) )

Community - What brings these people to this venue? (e.g. a bar - <USER2> dropins vs art showing - people interested in art). A general networking event may rate low on this index but a Ruby brigade meetup (shared interests) may be high

Continuity - How likely are you to meet these people again? How involved can you get?

By this rule, local monthly _small_ entrepreneur, coder, hacker meetups may be a much better use of your time than trying a general purpose networking event. Every time I go to a big networking-type event (where I meet people whose job it is to network), I get reminded that its not really my thing; I really go just to observe how bad/well people can make small talk. It's good practice. If you really want to observe good networkers, go to a political fundraising event (these people are pros, for better or worse).

I have met people at networking events - the key thing is you'll know pretty quickly if you hit if off with someone - the whole point of networking (as explained to me by the best networker I know - a woman president of a tech company) is to setup a "1st date" to follow-up if you hit it off - you don't go to networking events to have a 20-min conversations with one person but to meet a number of potential people who you might hit it off with for lunch or business.

If you go to a networking event and participate, good luck and remember: "One of the most difficult social tasks is to join an activity that is already in progress." Basically that means going up to a group of people who are chatting. There are techniques for doing this but theory is cheap, execution counts.

<comment><USER6> _"One of the most difficult social tasks is to join an activity that is already in progress." Basically that means going up to a group of people who are chatting. There are techniques for doing this but theory is cheap, execution counts._

Can you please give some tips on how to do this? This is something I definitely need to learn. A friend once invited me to a programmers/founders meet, which was in a bar. [little dark, loud music] I was a little late, tried "hard" to get into the already chatting crowd. They had formed a tight circle, standing, with their backs to me from every fucking angle. The friend just smiled at me and nodded and didn't do much else. I went around them like a moron, not finding any gap to step in, felt really foolish and frustrated, faked a phone call and fled. In my defense, I think I could have 'got in' only by pinching someone's ass. I rationalized it thinking "Fuck them. I'd better be coding right now anyways."

<comment><USER5> > Can you please give some tips on how to do this?

I used to think I was being clever by hovering next to groups when I didn't know anyone - close but not too close so it would seem like I was with that group. Not so - you just look like a wallflower.

It's not something you need to learn. It's something everyone can do. Unfortunately, it is all about commitment. By hovering, you are not committing. You have to be willing to interrupt the group/sidetrack them (although it is better to start side conversations inside the group then to completely derail it).

It is quite a rush/good feeling to be able to approach a group of strangers.

The basic concept is of leadership. If you approach a group of people and just kind of stand there, you're playing a passive role/not taking ownership of the situation. You're leaving it up to the other person to do something - <USER3> since they're in the group - they can easily do nothing/blow you off. Believe me, I think one of the most uncomfortable feelings is hovering around groups and knowing that the people in the group know you are hovering.

So, when you have the courage (note I'm not saying confidence - <USER4> sometimes implies you feel good before you do something whereas with courage you feel the fear and do it anyway) to approach a group - pick a specific person to engage. There is always someone who is momentarily left out of a group - for example, someone might start texting or looking around the room - go to them - pick them to engage. If you can engage them in conversation, you've got an in. And, if you pull more people in the proximity of you who are in the group into that conversation, you start to divide and conquer.

However, if you don't engage them properly and the group kind of stalls [e.g. proverbial music stops playing] - what you have to do is even more courage- taking - introduce yourself to each person calmly but quickly with brief eye contact. In every group, there will be someone who will ask you a question to interrupt you - whatever you do - do not answer the question - if you do - you've lost (e.g. shown that you weren't committed to introducing yourself to everyone). But don't ignore them, tell them that's a good question (whatever it was - even if it was something like - 'what do you want?!') and go on to the next person. If you do this right, you will effectively control the group by interrupting whatever it was they were talking about and making you the focus. The key, once you've done this, is to put the focus back on the group, engage them - get other people talking. By focusing on other people, even just one, you're engaging the principle of reciprocity (e.g. he's paying attention to me, maybe I could pay attention to him).

We all know that it's so easy to meet people through friends (because we're all pre-vetted right?). A good exercise to work on anxiety talking with strangers (which we all have) is to talk to someone at a retail store [ideally opposite sex] and try to get the conversation out of the standard customer- clerk role [e.g. make it personal - ask them why they cut their hair that way].

Actually, one of the best ways to practice what I call social skydiving (I read this term somewhere - very fitting) is to go to a networking event of the type described in the NYT article.

> The friend just smiled at me and nodded and didn't do much else.

The friend could have easily smoothed it for you with a 'X, this is my friend Y.' But don't fault him - he was probably not feeling much like he was part of that "group".

If I were in your situation, after that happened, I would have left the scene momentarily - then gone back in a couple minutes and basically hug the friend and/or put my arm around the friend (yes, its awkward) and then introduce myself. By leaving, you can sometimes reboot the situation enough so an initial bad impression doesn't matter.

<comment><USER7> It also helps to break the ice with some smart/funny comment at your friend, but also hoping the comment reaches to the others as well. Sense of humor or good observation can be your ticket in to the network.

</comment></comment></comment></comment><comment><USER8> I've never found networking "events" to be of any use. What has worked is asking people you're working with (say investors) for introductions any chance you get. The conversation starts off much warmer and you tend to get to people who are usually too busy to attend conferences for the sake of it.

</comment><comment><USER9> What I want to know: How do you start a conversation with a stranger?

<comment><USER10> -Nice weather today.. :P

Seriously, my best method of starting a chat with a stranger is making a comment about something in the surrounding environment quickly followed by a question about the other person. Try not to talk about the weather as this usually raises a "red flag", ie I really want to talk to you but dont know how.

After that, it's mostly questions/answers as you start to get acquainted. Question/answer ratio in your speaking time should be around 80/20. Try to keep your answers relatively short, especially in the beggining, so that you don't bore your conversational partner. You can usually tell if he/she is genuinely interested in a subject as he will request more information about it. As for the questions let him/her finish his answer, nod, have an open body posture, copy his/hers with a 5sec delay and while he/she answers think of the next one. Give some time between his answer and the next question (his/hers or yours) but be ready to fire in case the conversation drifts off to that "awkward silence".

In any case try to remember that people like to talk about themselves (hence the question/answer ratio) but this isn't always the case, e.g. you have met an introvert (takes time to start opening up) or someone who's having a bad day (difficult to change his/her mood).

Finally, if you fail, don't sweat it. Find what/where the conversation went wrong change your tactic a bit (alter talking speed, think of better generally applied questions, humor) and try again. And don't forget to smile :)

<comment><USER11> "Try not to talk about the weather as this usually raises a "red flag", ie I really want to talk to you but dont know how."

This varies depending on the region. If you live somewhere with interesting enough weather (i.e. eastern Washington) it's always a topic of discussion. (in June) "Did you see the snow this morning? That was crazy!" (in July) "Man, it's 100 degrees out. Can you believe it snowed last month?"</comment></comment></comment>