Why 37signals is falling into obsolescence - <AUTHOR> http://www.seancoleman.net/post/3052050668/why-37signals-is-falling-into-obsolescence

<comment><USER3> While a mobile app can certainly deliver a rich user experience, the availability of a standard, universally supported platform (read: a browser) is a more sustainable approach to application deployment.

We've been down this road before. In the pre-browser era, desktop application were king, but at a high cost to IT departments as well as end-users. The same game is being played out on mobile platforms now.

From a business perspective, do you really want to support multiple development efforts for each and every mobile platform? Will it be worth it in the long run?

The advantages of the mobile app platform must be balanced with the cost and maintenance overhead as well. Give me a great browser platform, and I'll gladly take its shortcomings over the long-term sustainment costs of a dedicated mobile app.

<comment><AUTHOR> As a customer though, what would you prefer? That was my main point is that the customer demand is really heavy on native apps.

Evernote is an example of a successful startup that has native apps for every major platform, and they consider it a strategic decision.

<comment><USER4> Only 37Signals is a B2B company that has a lot of customers that are developers. They have an API. Their source of income is their subscription and not apps for native platforms.

In other words, many of their customers who want a native app can scratch their own itch pretty easily, and 37Signals can cut down their own development efforts and focus on making their platform better.

The reason Evernote has apps for every platform is simple: that's the only thing they have going for them. Their interface isn't the best, and there are tons of other ways to save notes. They just happen to have really good cross- device integration. Other note-taking tools have users for different reasons.

I agree with 37Signals. App Stores are an obsolete concept that are only making a comeback because we have a new wave of devices without a solid set of mature cross-platform development tools.

EDIT: In other words, 37Signals wants to differentiate themselves from other applications in more unique ways than just "works natively on every device."

</comment></comment></comment><comment><USER5> Having only a mobile web version isn't necessarily obsolete.

The current demand for native apps on mobile devices stems from the fact that it is easier to get high quality user experience on the native apps than on mobile web browsers. This was true of desktop apps, too, 5-10 years ago. However, web-apps offer a host of other advantages and so long the UX can be brought up to a decent quality, I'd take a mobile web-app over a native app that I have to go find and install for the same reasons I prefer web apps to desktop apps (both to use and to create).

So, 37s's decision to do a mobile web-app is either obsolete or visionary. I'd bet on the latter.

<comment><USER6> It always amazes me that people don't see we're repeating desktop history.

Look back to when desktop apps were the norm. People eventually realized it was counter intuitive to write the same program 5 times for 5 different platforms so they started moving to frameworks (Java/.Net). Then as the web gained the power to host comparable apps people started moving to the web.

It was a natural progression of societal knowledge based on trial and error.

So why are we now making the same mistakes on mobile platforms? Especially since web apps can already match the functionality needed by 99% of the applications out there? Seems a little silly

<comment><USER7> That 1% still covers:

Access to the device's GPS. Access to the devices Accelerometer. Hardware Accelerated 3d Graphics (Although this is possible with WebGL, I would wager the tools are more focused towards native clients) Access to the device's Camera / Film Roll

<comment><USER8> You can access the GPS information with Javascript. <http://diveintohtml5.org/geolocation.html>

</comment></comment></comment></comment><comment><USER9> Ugh, I could have read something else but I read that instead.

</comment><comment><USER10> linkbait headlines anyone? Next, learn why some blogger is quitting javascript forever and switching to Erlang: bit.ly/hTNYCW

<comment><USER11> Come on. It'd only be linkbait if the meat of the article didn't address the title. I thought his title was very relevant to the content.

<comment><AUTHOR> Thanks, my intention was definitely not to create a linkbait title.

</comment></comment></comment><comment><USER12> Maybe I'm missing the point, but aren't there tons of 3rd party clients available as well as their own Highrise client?

<http://basecamphq.com/extras>

I'm not fan of Basecamp, but I think they've got a good ecosystem going.

<comment><AUTHOR> They have their Highrise client that they hired a contractor to develop on iOS, and decided that it wasn't worth repeating for their other products.

</comment></comment><comment><USER13> He's right. Many people bet on the web, there is a lot of legacy on the web, and the web is obviously easier for developers to make.

But the _customers_ want native apps (as thick clients to internet services). Native apps work better. They are faster and have more features.

Companies that have so much invested in the web, and choose to stick with this type of technology will find that they were wrong.

<comment><USER14> I think you are confusing the average smartphone user with the average Basecamp user. The former is interested in shiny apps that make them feel like Tom Cruise in Minority Report. The latter wants something that is fast, easy to use, and saves time.

37 Signals has made a pile of money by ignoring all the cw about what customers want. I wouldn't write them off just yet.

<comment><USER13> Native apps are faster, easier to use and save time. Web apps are easier to develop for them, that's why they chose them.

<comment><USER15> This is the argument everyone pushes, but it's not true. Yes, JavaScript is slower than Objective-C. No, it doesn't matter in most cases. Speed of almost all consumer apps is measured in user time, not in ms, and you can achieve top notch perceived speed in a web app for most of the types of apps on the app store.

<comment><USER13> The problem is not javascript, the problem is server round-trips. Local HTML5 is practically indistinguishable from a native client - but that's not what they are doing. They are making a proper server based system.

<comment><USER14> All of their products are web apps. Those round-trips are getting made regardless.

</comment></comment></comment></comment></comment></comment><comment><USER16> A lot of desktop computer based software migrated to the web where it ran on the server. Made things simpler and avoided "updates downloads" since the updated could be pushed to the code running on the server. With the different platforms, and so many versions of Andriod floating around, doesn't what 37 signals is doing, make things that much simpler? Basecamp is about online collaboration. How could a "native app" do this without the internet?

</comment><comment><USER17> I think I disagree with the premise, which is that 37Signals is making some sort of strategic mistake by not implementing native apps. My perspective is that back in the '90s, I wrote internal desktop apps for corporations when the web came along, setting back the user experience a decade (at least). Web apps have only recently begun to offer the kind of user experience that desktop apps provide. Because of this, I was surprised when web apps won, and won so quickly and thoroughly. But it turns out that the user's experience - <USER2> important, and especially so for consumer apps - is only one of many factors that drive technology adoption, even in the consumer market.

Certainly 37Signals might pick up some users with smoother client apps, but at least for the time being, they have to balance that against the cost of developing those apps. My guess is that most users will care about clunky HTML vs. smooth native apps about as much as they cared about clunky HTML vs. smooth native apps back in '98. Which is to say, hardly at all.

</comment><comment><USER18> 37signals has an extensive API built around all their apps with tonnes of 3rd- party native apps for desktop and mobile. If there isn't a native app for your platform, go write one and stop complaining.

And arguing that a web app isn't useful as _a freaking web app_ on your mobile is a little wild to me. There may be deficiencies in areas like file uploads on mobiles, but largely, you interact with Basecamp through a browser on your desktop. Why doesn't a mobile work well? (I've used many iOS Basecamp clients and none were _better_ than the mobile version launched today by 37s).

I love native apps too, and I'm glad that 37signals encourages their existence by providing a solid, well-documented API. And while they do ship some native apps, I totally buy them building really awesome web apps on the desktop, and now on mobiles.

<comment><USER11> It's frustrating to me when people say something along the lines of "go write one and stop complaining." It's hardly a useful retort. I think his main thesis is an important one to take note of. One that anyone who runs a software company should consider.

</comment></comment><comment><USER19> I can't seem to find OrangeSlyce (the author's company) on the iOS App Store. Maybe his company has been obsolete since day one? Our little company doesn't have a native iOS app either. We do make sure our web app works in mobile WebKit based browsers though. Are we obsolete already?

The relevance of a native app is strictly dependent on your users' demands. If I'm going to make an investment in a native app for iOS, Android, Blackberry, etc, there had better be a business case. How many users would sign up for my service based on the fact that I have a native mobile client? What percentage of my users are even using a mobile device to access my service?

It's simply not rational to look out across the tech landscape and declare every company without a native mobile app to be "obsolete".

<comment><USER11> His title is clearly calling out 37 Signals, not every company. Are you a bit defensive? 37 Signals has thousands of users, maybe even more. It's my understanding that there is constant demand for a native mobile app.

If your business doesn't need a native app, then don't create one. It's really that simple.

<comment><USER19> So assuming we're talking about 37 Signals only, I'd still offer the rebuttal that claiming obsolescence because of a lack of native app is a stretch. It's hyperbole. Yes, there is a vocal community of users who want a native app, but what are the costs associated with building the native app? Would 37 Signals see any substantial benefit by releasing a native app? What percentage of their user base are dissatisfied with the mobile web app?

There are a vocal community of users that want Flash on their iPad/iPhone/iPod Touch. There are a vocal community of users who want a Facebook iPad app. Apple and Facebook aren't on the path to obsolescence because they don't bend to the whim of their users.

In order for something to be obsolete, it must be operable, but unused. To be on the path to obsolescence, one would expect a decline in use. That's pretty hard to prove in the case of 37 Signals, because they don't publish user-base numbers. Or maybe they do. Does anyone have a source for this information?

<comment><USER11> You've presented a very good rebuttal that's hard to disagree with.

</comment></comment></comment></comment><comment><USER20> 37signals provides APIs for its apps and there are many 3rd party native apps available based on them. For example Propane - a native Mac Campfire app is very popular.

</comment><comment><USER21> If the argument is "they are obsolete because they don't do native mobile", that is a poor argument to make. Where is the data to support it? It's simply a theory wrapped up as a conclusion.

The more important question is, "are they positioned well for the future?" That is really what matters. Don't focus on a particular type of technology as the end-all-be-all of development.

</comment><comment><USER22> Using the cache manifest, you can still run your web apps offline...

</comment><comment><USER23> I think he makes a very lucid argument ...

"The fact that 37signals is unwilling to adapt to a changing market (i.e. demand for native apps) shows their archaic mindset and lack of innovation"

Mobile Development seems to be the next frontier in the evolution of the interwebs, and they could very well be betting on the wrong horse.

<comment><USER24> I think apps are the flavor of the day, but the web will ultimately win.

<comment><USER25> Agree. Mediocrity always wins.

<comment><USER26> No. Better economics wins:

Web sites are cheaper to maintain upgrade, and add features to than native applications, which means that for the same price, you can provide more to your users. Think how much simpler it is to provide a web app that works on iPhone, Android, Blackberry and Symbian, compared with building it natively for each one.

</comment></comment></comment></comment><comment><AUTHOR> I mention that companies must _provide_ for distributed client access. There are other ways of doing this than developing all clients in-house. Twitter has demonstrated this excellently through their fantastic API. I think a key for startups to focus on is API first, client acces second.

<comment><USER27> I couldn't vouch for its speed or coverage, but Basecamp has an API which several existing third party native apps use...

<comment><AUTHOR> I've used a lot of those "extras" but they are all very limited and I think mostly because of their API.

</comment></comment></comment><comment><USER28> Is Google also obsolete?</comment>


YouTube - iPhone4 vs HTC Evo - <AUTHOR> http://www.youtube.com/watch?v=FL7yD-0pqZg <comment><USER2> I know, I know, it is meant to be funny (it is) and it's lame to make a serious comment but:

The problem is by now many households have hand-me-down iphone chains in the family and have spent much money not only in the handsets but also in apps. So unless really this is your first smartphone, the comparison is not a level playing field - you are not selecting purely on the basis of feature comparison.</comment>


New study affirms that fewer copyright restrictions benefit the economy - <AUTHOR> https://www.eff.org/deeplinks/2012/09/copyright-and-campaign-misinformation-new-study-affirms-less-copyright

<comment><USER2> There has been plenty of evidence about how intellectual property is actually counter productive and hinders innovation for a long time:

[http://levine.sscnet.ucla.edu/general/intellectual/against.h...](http://levine.sscnet.ucla.edu/general/intellectual/against.htm)

And is not surprising when you realize that it is basically the government going around granting monopolies to private entities, and what incentives that creates (the main incentive is to expand the monopoly power rather than to create and innovate, innovation is contrary to monopolies).

</comment><comment><USER3> The article is _very_ light on details about how the "study" arrived at their conclusions and figures (I hesitate to call it a _study_ , because it didn't actually test any hypotheses).

The report is published by a group that lobbies for "copyright reform that enables fair access to content", with a membership is predominantly tertiary institutes.

They argue that a number of industries are _dependent_ on some "limitations and exceptions" to copyright law. These industries include: research and education, publishing, broadcasting, film, music, ISPs, and the performing arts. The industries currently employ some people, make some money, and contribute some amount to GDP. Their extend of this dependence on copyright limitations is not quantified by thre report and is quite vague (focussing mainly on quotation, citation, and critique).

It concludes that if Australia implements "better crafted limitations and exceptions", these industries can increase their job/income/GDP metrics, and reduce the amount paid as licensees of copyright protected works to foreign companies.

</comment><comment><USER4> The methodology of this report is a joke. For example, to come up with the $600 million figure, they say:

"We simulated an increase in productivity growth in the exceptions using industries of just one one-hundredth of its otherwise experienced growth in the three years from 2007 through 2010 – a very conservative estimate of additional growth."

i.e., they pulled 1% (net) increase in growth out of the air, and then simply calculate the economic benefits of that hypothetical increased growth.

(Per <http://www.digital.org.au/media/166>)

</comment><comment><USER5> _Meanwhile, creativity—far from being stifled without more copyright laws on the books—is currently thriving. There's been a market increase in the amount of movies, music, and books produced over the last decade, as this comprehensive study done by CCIA and Techdirt's Mike Masnick shows._

That's a poor argument for loosening copyright restrictions - it makes it seem as if the status quo is OK. I personally think existing copyright terms need to be significantly shortened, and that a legal challenge might succeed on the basis that Congress is empowered to grant copyrights for 'a limited time.' Insofar as copyright terms exceed the average human lifetime, they are effectively endless for all but the longest-lived members of society, promoting a creative gerontocracy.

<comment><USER6> Technological advancement (eg: music software, cheap HD cameras, etc.) and significantly better distribution platforms (eg: the internet) can account for an increase in production... they need to account for that, otherwise its kinda nonsensical to say what they say.

But then, it seems this is ideologically driven from the beginning.

It is much like the "studies" in the 1990s, published by microsoft (and you see them now from google) that say Microsoft (or android) is winning. (despite, for instance, Samsung never giving sales details, and when they were forced to by the recent lawsuit, it turned out they were pitiful... so you have press claiming huge sales to further and ideological position that "Android is winning" (and thus "open" works, even though Android is no more open than iOS) using essentially made up data and then other sources repeat it to the point where large numbers of (frankly, uninformed) people believe it.

This is how propaganda works.

<comment><USER7> _...even though Android is no more open than iOS..._

Wait a minute. We all know from reading your comments on which side of the Apple/Google divide you fall, but how on earth do you come to that conclusion? Can you download and redistribute the complete source code to iOS?

</comment></comment></comment><comment><USER8> Also this TED talk by Johanna Blakley, shows how the lack of copyright and patent protection in the fashion industry actually works to benefit both innovation and sales: [http://www.ted.com/talks/johanna_blakley_lessons_from_fashio...](http://www.ted.com/talks/johanna_blakley_lessons_from_fashion_s_free_culture.html)

</comment><comment><USER9> I'd like to see a quantitative study on the effects of ridiculous patent laws. Most HNers know that US patent law, and the entire patent system here is broken, but a quantitative study might be able to convince people who otherwise wouldn't pay any attention to the issue to give it some thought.

<comment><USER6> The US Patent law, and the "entire patent system" are not broken. A lot of very vocal HNers like to go on ideological rants along these lines, but when pressed for specifics they generally cannot back them up (eg: they haven't even read the patents in question, or don't know the difference between a utility patent and a design patent, or even what design or utility patents cover.) In fact, I have yet to meet someone who opposes the current patent system who understood what patents are.

Anti-IP propaganda claiming that the patent system is "doing damage" by measuring the enforcement of patents against people will be good for ideological arguments but actually be pointless.

Because it comes from that lack of understanding of the benefit patents provide. Patents incentivize the publication of inventions so that others, who may not have been up to speed on the state of the art, can come up to speed and start competing.

They also disincentivize the need to obfuscate or protect your inventions using physical or technological means, which improves repairability and in fact, enabled the entire standard-component industrial sector we have that has allowed integrated circuits to be such a big part of our lives.

Without patents, technological progress would be severely restricted-- not because there "would be no incentive to invent" (a straw-man that anti-IP people seem fond of knocking down) but because there would be vastly less communication and understanding of how other people's inventions work.

And like Global Warming and "Quantitative Easing" and other nonsensical, anti- scientific, ideological positions, the proponents, rather than make or defend logical arguments for their positions, tend to use signaling like "most HNers know the US Patent law... is broken". You just presume your conclusion, and a lot of others do as well, which is why they can't make arguments and instead are forced to downvote or shout down those who make counter arguments. Since having a good argument (with well researched citations) is not a defense against downvotes on HN, your claim becomes self fulfilling-- anyone who might challenge the ideology of anti-IP with facts, logic or reason, is downvoted so you don't have to see them.

Which, ironically, is why I'm posting this here. Becuase I really, really, hate to see the single mindedness, and cheering that groupthink.

So, if you want to claim that the system is broken, feel free to make an argument, and feel free to account for the factors I briefly touched on.

<comment><USER7> _In fact, I have yet to meet someone who opposes the current patent system who understood what patents are._

Are you talking about meeting someone on HN? How about HN user grellas, a Silicon Valley business lawyer? I doubt his arguments against the status quo are made in ignorance:

[http://www.hnsearch.com/search#request/all&q=grellas+pat...](http://www.hnsearch.com/search#request/all&q=grellas+patent)

Edit to add:

 _Without patents, technological progress would be severely restricted-- not because there "would be no incentive to invent" (a straw-man that anti-IP people seem fond of knocking down) but because there would be vastly less communication and understanding of how other people's inventions work._

It is difficult to accept this argument when the majority of software patents we see in lawsuits are written as obtusely as possible. Software patent language is the antithesis of "communicat[ing]... how other people's inventions work."

<comment><USER6> Your response is exactly what I'm talking about. You believe, but you don't know. You can't make an argument, you point to someone else, and advocate faith.

Imagine if a religious person had used that kind of "argument" to you, and referenced a priest regarding the law?

It is ok to not know. And it is ok to take things on faith. The problem is, when people think that their faith means they are right and people who make arguments they can't counter are "Wrong" because it goes against their faith.

<comment><USER7> Claiming that nobody who disagrees with you knows what they are talking about is also a symptom of religious thinking. You sound exactly like your description. Have you made any arguments yourself that don't beg the question by saying that patents are good because they were intended to be?

You said you've never met someone who knows what patents are and still opposes them, implying that nobody who knows is opposed to patents. I provided a proof by counterexample of someone it would be difficult for you to argue doesn't know what he is talking about.

And you haven't. Instead you claim that I don't know what patents are. I do know the difference between a design patent and a utility patent (and a trademark, and copyright). I've seen good and bad examples of each. Yet I still oppose software patents, support reduced terms for all of the above except trademarks, and want to see a much higher burden of proof for a patent to be granted, and higher still for a lawsuit to proceed. I also argue that independent invention should be a defense against patent infringement that doesn't require an army of lawyers to raise. Yet somehow you will try to claim I don't know what I am talking about.

The real test to distinguish rational from religious thought is the willingness to change one's mind. I'll tell you what would change mine: a plethora of incontrovetible, scientifically sound studies that prove that the harm done to and risk imposed upon small innovators and market competition is worth the benefit to society; or a solid implementation of the changes I'd like to see, that subsequently results in a collapse of innovation (collapse of any business alone does not count as a collapse of innovation). What would change your mind?

</comment></comment></comment></comment></comment><comment><USER10> This is a point that has also been made by German historian Eckhard Höffner[1]. He claims that one of the main reasons for Germany's rise as an industrial power during the 19th century was its very lax copyright regulations that allowed a rapid distribution of pirate prints of publications to every corner of Germany, thus creating a very large reading public that would later flock to universities and engineering schools and fuel Germany's innovative economy.

So far there hasn't been an English translation, although Ars Technica, Wired and the English edition of the Spiegel carried good reviews/summaries[2][3][4]

[1] Eckhard Höffner: Geschichte und Wesen des Urheberrechts. Verlag Europäische Wirtschaft, 2010, ISBN 978-3930893164 (2 Tomes).

[2] [http://arstechnica.com/tech-policy/2010/08/drool- britannia-d...](http://arstechnica.com/tech-policy/2010/08/drool-britannia- did-weak-copyright-laws-help-germany-outpace-the-united-kingdom/)

[3] [http://www.wired.com/business/2010/08/copyright-germany- brit...](http://www.wired.com/business/2010/08/copyright-germany-britain/all/)

[4] [http://www.spiegel.de/international/zeitgeist/no- copyright-l...](http://www.spiegel.de/international/zeitgeist/no-copyright- law-the-real-reason-for-germany-s-industrial-expansion-a-710976.html)

</comment><comment><USER11> Hardly surprising, the less government interference there is in the market the better the economy does. Look at the industries that are tanking such as finance, manufacturing, education, healthcare, airlines and you'll find no end of regulation and interference.

<comment><USER5> For heaven's sake, extension of copyright terms comes at the behest of the publishing industry. Governments would likely welcome the increased economic activity and lower administrative burden of lower copyright protection, but publishers are going to support whatever politicians promise them the most revenue protection. Your various comparative examples are driven by wildly different factors (not least demographic ones) and don't support your argument.

<comment><USER2> > For heaven's sake, extension of copyright terms comes at the behest of the > publishing industry.

This is known as _regulatory capture_ , and is what is to be expected once the government starts to grant monopolies: instead of competing and innovating, corporations dedicate themselves to much more lucrative rent-seeking.

When the government is the one who decides who can do business and how, it becomes much more lucrative to put your energy in lobbying the government for rules favorable to you, than to put the energy in competing and pleasing consumers.

Without patents the only thing that matters is execution: pleasing consumers, with patents all that matters is how you can milk the system and use it to crush your competitors that might have more competitive products.

<comment><USER5> You're assuming the government initiates regulation. In reality, merchants and artisans tend to form into guilds and then demand new entrants to that market be licensed. Adam Smith noted this phenomenon in _The Wealth of Nations_ in the famous passage: 'People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices. It is impossible indeed to prevent such meetings, by any law which either could be executed, or would be consistent with liberty and justice. But though the law cannot hinder people of the same trade from sometimes assembling together, it ought to do nothing to facilitate such assemblies; much less to render them necessary.'

A more recent treatment of the subject appears in nplusone magazine, but the site is down just at the moment: <http://nplusonemag.com/death-by-degrees> Another recent example from NPR: [http://www.npr.org/blogs/money/2012/06/21/154826233/why- its-...](http://www.npr.org/blogs/money/2012/06/21/154826233/why-its-illegal- to-braid-hair-without-a-license) (you'll be happy to know that Clayton recently won her court case, and plans to reopen her business).

</comment></comment></comment></comment><comment><USER12> No surprise that reducing copyright protections drives economic growth. It acts like a reduction in cost for the raw material of other creative and distribution businesses. What is not as clear is the impact on long-term innovation--in particular the very capital-intensive kind.

The Lord of the Rings movies would have been less expensive to make if they did not have to pay for the rights to use the story and characters. But would the studio have given the green light to such an expensive shoot if they had a lower expectation of return?

<comment><USER13> Why should they expect a lower return? While I understand that value in being unique, I can't imagine a high quality production of a public work would be damaged that much by low quality productions of a similar work unless it's just profanely overdone. And in that case, why would they have sunk that much money into it?

I mean, every time a zombie film comes out, three cheaper ones come straight to video. The new Dawn of the Dead did fine. And even in copyright, Snakes on a Train was in video stores three days before Snakes on a Plane came to theaters!

Sure, they may have had to think of a unique name to identify their product to fans (even just "Peter Jackson's LotR",) but I'm not convinced that they would've suffered, had lower quality renditions been widely available. It may have made them stand out more.</comment></comment>


How Dropbox securely stores your passwords - <AUTHOR> https://blogs.dropbox.com/tech/2016/09/how-dropbox-securely-stores-your-passwords/ <comment><USER3> I cannot see any obvious weaknesses in this scheme.

It seems to address a known pain point in bcrypt (max length), implements a pepper in a secure way (which cannot inadvertently degrade security), and is otherwise doing things which are best practices (high work factor, per user salt, etc).

I know peppers remain controversial (some people claim they're pointless, and make a good argument). But ultimately nothing Dropbox is doing with peppers in this article makes your password easier to break, only harder.

I'd call this scheme 10/10.

<comment><USER4> It's a good system, especially compared with the current best practice of simply hashing passwords with bcrypt and calling it a day.

I can't recall it off the top of my head, but Facebook has a similarly impressive system with more secret sauce involved for performance at scale. I believe what they do is the following:

1\. Hash the password with MD5(password).

2\. Generate a 20-byte (160-bit) random salt (this is well over the 64 bits you'd need to defend against birthday attack collisions).

3\. Hash with hmac_sha1(hash, salt).

4\. Send this value to a separate server for further operations (mitigates offline brute-forcing).

5\. Hash in a secret key with hmac_256(hash, secret). Note this operation is on a separate server. The secret key might be colloquially termed a "pepper".

6\. Hash with scrypt(hash, salt) to make local computation slower.

7\. Shrink the final value with hmac_256(hash, salt) for efficient database storage.

If any Facebook engineers are around, please correct me if I've missed or misinterpreted any part of that.

<comment><USER5> The current best practice of simply hashing passwords with bcrypt is fine, and anything past that which doesn't doesn't involve (probably per-password) use of an HSM adds only marginal value.

I wouldn't want anyone to read this and get the impression that "secret peppers" and multiple hashing rounds and HMAC were important components of a password storage system. They are not: they're things that message board nerds come up with.

If you want to be a step better than just storing passwords with bcrypt, your next step is to create an authentication service that runs on separate hardware with an "is this password valid" and "enroll this password" API and nothing else. The stuff people do instead of this is basically cosmetic.

<comment><USER6> Yep, Tom, that's the FB solution - or rather, halfway up the stack of hashing is a callout to a service where HMACs are involved. These bring their own challenges: [https://video.adm.ntnu.no/pres/54b660049af94](https://video.adm.ntnu.no/pres/54b660049af94)

<comment><USER5> No, according to your talk, Facebook's solution is that password validation is pushed out to the front-end servers, who use back-end KMS services to do some (but not all) of the crypto.

I think this is a suboptimal approach. Can you tell me what the benefit of your layered approach is over simply adding to your KMS servers the APIs for validate(user, password) and change(user, oldpassword, newpassword)?

If your KMS service did password validation directly, you wouldn't need any of the layers in this architecture. HMAC would add nothing (it would be tautological, since anyone who could directly attack the hashes must have also owned up the KMS service). I still don't totally understand the MD5 step. You could just use scrypt and nothing else, and you could probably ratchet the work factor up because you wouldn't be billing the front end servers for those cycles.

I generally like, have recommended, and have built a few times the "software HSM" KMS approach you're describing here --- but only for "seal/unseal" and "sign/verify" APIs.

<comment><USER6> >No, according to your talk, Facebook's solution is that password validation is pushed out to the front-end servers,

Yes, although I don't work for Facebook any more, that was and probably still is the case.

>who use back-end KMS services to do some (but not all) of the crypto.

In fact, the backend service does a tiny amount of the crypto.

>I think this is a suboptimal approach.

Of course you do, it's not like I've not argued with you before, Thomas. :-)

>Can you tell me what the benefit of your layered approach

...Facebook's layered approach...

>is over simply adding to your KMS servers the APIs for validate(user, password) and change(user, oldpassword, newpassword)?

If you want your hash to be ... hashed, rather than lodged in some questionable silicon, not putting a fat crypto load onto the backend avoids the "thundering herd" problem when some fraction of 1.7 billion people want to log in.

>If your KMS service did password validation directly, you wouldn't need any of the layers in this architecture.

Quite, and if everyone flew instead of drove, we wouldn't need cars with layers of bumpers, crumplezones, seatbelts and airbags; but it's merely shifting the problem for 1.7 billion people.

Have I mentioned "scale"? I should mention scale. Scale is a thing.

>HMAC would add nothing (it would be tautological, since anyone who could directly attack the hashes must have also owned up the KMS service).

Yes. Fucking huge KMS service. HUGE. Trumpiness levels of -HUGE- and eating lots of power and redundancy.

1.7 billion people. That's a lot. Like 0.01% of it is 170,000 people. All logging in together. All over the world.

>I still don't totally understand the MD5 step.

Yeah, but I'm not betting that Mark's/whomever's coding at the time was focused on the future of password authentication.

>You could just use scrypt and nothing else

Yes, but where's the fun in that?

>and you could probably ratchet the work factor up because you wouldn't be billing the front end servers for those cycles.

The frontend servers are approximately precisely where you want the cost/chokepoint. There's a metric fucktonne of them and they are closest to the request, so by definition they are scaled to the load.

>I generally like, have recommended, and have built a few times the "software HSM" KMS approach you're describing here --- but only for "seal/unseal" and "sign/verify" APIs.

Cool. What's your biggest deployment?

<comment><USER5> Last question first: large, but not Facebook large (I spent 10 years consulting on this kind of thing).†

You comment early in the talk that the MD5 step is somehow helpful for password dumps. That was the bit I didn't follow. If it's there because that's how password hashing worked before your team got to it, that makes a lot more sense. But then: it's not a "layer" of the onion so much as a sheen of dirt that needs to be washed off the onion. :)

I get that your auth problem is huge. Yuge. So big you wouldn't believe it. I totally believe you. No, wait, I don't believe you, that's how big I know your authN problem to be: unbelievably huge.

But here's the thing: you're already scrypting passwords. We're not debating whether you can use expensive password hashes. You already use expensive password hashes. I'm saying: the model where the KMS does a small bit of the password hash step and defers the heavy lifting to front-end servers seems like a suboptimal way to structure this:

* You have to bill cycles from the front end to do it

* You can't change password hashing without updating all the front-end servers

* It's harder to track usage because it's spread across a zillion machines

* You're more constrained in how you scale it (for instance, if you wanted to double or triple the work factor) because whatever your new scheme is, it has to fit with the existing front-end resources.

I'm not saying "wow, it's dumb that you built it this way". I'm saying, if other people are reading this thread thinking about how to do it:

* DO split authentication out into its own service

* DON'T have that authentication service be "HMAC as a service" and then do scrypt on your front-end service

YOUR MOVE, ALEC MUFFETT. I keep going until you unfriend me on Facebook so I can't see you wincing about these posts.

† _I 've_ assessed _Facebook-large variants of this, though._

<comment><USER6> >Last question first: large, but not Facebook large (I spent 10 years consulting on this kind of thing).

I just spent 3 years living it for 50h/week. Hence why I am taking a vacation.

>If it's there because that's how password hashing worked before your team got to it, that makes a lot more sense.

That. It wasn't even me, it was done before I arrived, but it was done by a team of geeks with a tremendous nose for making the best of the database that they had available to them without pulling the old password-migration "log in with one password, parallel-encrypt with a new algorithm, and save the new hashes" \- thing, because some of those billion people might never log in again for years. You would never stop migrating people.

I remember internal pasword algorithm migrations at Sun, at least there you could force the matter for 10,000..40,000 people.

But you can't force everyone to migrate at FB scale.

>But then: it's not a "layer" of the onion so much as a sheen of dirt that needs to be washed off the onion. :)

You can take that approach, but - again - when will you finish the task? Whereas wrapping one algorithm in the next is a finite task which is completable in a reasonable amount of time.

>I get that your

...Facebook's...

>auth problem is huge. Yuge. So big you wouldn't believe it. I totally believe you. No, wait, I don't believe you, that's how big I know your authN problem to be: unbelievably huge.

Well channeled. :-)

>But here's the thing: you're already scrypting passwords. We're not debating whether you can use expensive password hashes. You already use expensive password hashes. I'm saying: the model where the KMS does a small bit of the password hash step and defers the heavy lifting to front-end servers seems like a suboptimal way to structure this:

>* You have to bill cycles from the front end to do it

Yes. 0.1% of frontend cycles. <blank expression> And?

>* You can't change password hashing without updating all the front-end servers

...which happens three times a day, weekdays, and is moving to moreso.

>* It's harder to track usage because it's spread across a zillion machines

"Facebook has tools for that sort of thing":

[http://conferences.oreilly.com/strata/stratany2012/public/sc...](http://conferences.oreilly.com/strata/stratany2012/public/schedule/detail/25540)

...and to be honest it's not hard to do anyway.

>* You're more constrained in how you scale it (for instance, if you wanted to double or triple the work factor) because whatever your new scheme is, it has to fit with the existing front-end resources.

Yes. For a site with wildly heterogeneous architectures in front-end deployments, I can see how that might be a concern; but even AWS leads people to standardise on having approximately-the-same-kinds-of-hardware-doing- approximately-the-same-things.

>I'm not saying "wow, it's dumb that you

...Facebook...

>built it this way". I'm saying, if other people are reading this thread thinking about how to do it:

>* DO split authentication out into its own service

...or some component of it...

>* DON'T have that authentication service be "HMAC as a service" and then do scrypt on your front-end service

Why not?

>YOUR MOVE, ALEC MUFFETT. I keep going until you unfriend me on Facebook so I can't see you wincing about these posts.

Wince?

> I've assessed Facebook-large variants of this, though.

Did they buy it?

<comment><USER5> Sorry for the delay. A Jazzercise class suddenly appeared in the coworking space I work out of, so I fled, and then I had to give a talk about Starfighter.

Responses: _(when I say "your" let's just stipulate I mean Facebook)_

* Your password validation overhead is .1% of _current_ front end resources, but could be ratcheted up, and would be easier to ratchet up if they weren't shared by other things.

* I totally understand why you keep the old MD5 cruft around --- but would add that it's cruft that would be even less obvious if it lived behind an authentication server.

* I think it's safer, simpler, cleaner, probably easier to scale, and definitely easier to change authentication if it lives in its own service rather than being implemented (in part) on a generic application server. As usage shifts from HTML front-end to all API, you might even be able to keep app servers from even _seeing_ passwords.

* By "assessed", I mean, worked on other people's systems at this scale.

So I guess I'd wrap up with a question: if you had this to do over again, from scratch, the way you wanted to, would you have app servers do a password hash and then entangle it somehow with an HMAC operation from a crypto service, or would you have the whole password hash done on the crypto service directly?

<comment><USER6> tl;dr for me:

Putting the authentication service into a nice tidy centralised box does not actually achieve much, and may have architectural downsides.

Not the least of which is: if it's wholly in a service, then you have to authenticate the service; that's not such a big step from "if the hashed passwords are stored in a directory, then you have to authenticate the directory" of course - but if we were to equate the two systems because of the need to authenticate the {directory, service} then the service-based solution still has the downside of being a CPU hotspot and a potential single point of failure.

We're much better at distributing directories of data which is self-protected / needs no special treatment, than we are at building humongous scalable "secure" services with an enormous TCB and a physically enormous attack surface / footprint.

Yes. If I was doing this, sure, I would do this again. Curiously I am a big fan of password hashing rather than all-singing, all-dancing authentication services.

General notes: [http://dropsafe.crypticide.com/muffett- passwords](http://dropsafe.crypticide.com/muffett-passwords)

<comment><USER5> Not sure I understand. If you have to authenticate the password verifying service, you also have to authenticate the HMAC-providing service.

I like password hashing too. That's all my password validating service would do.

<comment><USER6> > you also have to authenticate the HMAC-providing service

Yes.

But, to look at the Facebook approach, what is the risk surface presented by the HMAC service?

Done properly in the FB approach the password is irreversibly hashed before it arrives at the HMAC component, and cheaply HMAC'ed and returned, where the onion of hashing is completed.

It's good to bidirectionally authenticate access to the HMAC service, but in terms of protocol it strikes me as less critical than in your scenario.

Either the HMAC is done properly (in which case the eventual hashes will verify for legitimate users) or - if someone inserts a "fake" hashing service - the HMAC'ed results will not validate, and a bunch of legitimate users will experience login failure.

( edit: there's a risk of exfiltrating the input to the service, but it's meant to be a shitload of work to achieve any evil with that input anyway, which also can be shorn of user-metadata and other clues thereby making it a bit less valuable )

Maybe I have missed something but to my mind this threat scenario fails (by dint of fake services, exfiltration, etc) in a "safe" manner.

=== Now === consider your "authentication service" approach.

Plaintext goes into... what?

The real service?

A fake service that returns "true" in all circumstances?

A MITM that exfiltrates the plaintext?

Where do you put the root of the trust chain to this service? In an SSL Certificate? Pinned? From which CA?

Simply: I feel that in centralised password authentication services there are a lot more potential shenanigans to defend against.

( ps/edit: and - I know you would not, but - don't get me started on "three strikes": [http://www.crypticide.com/article/42](http://www.crypticide.com/article/42) )

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> A note about combining SHA512 with bcrypt: Don't feed the raw binary output of SHA512 into bcrypt. Use the hexadecimal or base64-encoded form instead. (Dropbox probably does this already, since they mention base64 in passing.)

bcrypt is known to choke on null bytes. Each SHA512 hash has a 25% chance of containing a null byte if you use the raw binary format.

Using hex or base64, of course, decreases the amount of entropy that you can fit into bcrypt's 72-byte limit. But you can still fit 288 to 432 bits of entropy in that space, which is more than enough for the foreseeable future.

<comment><USER8> Thanks for the reminder. You could encode with a "base255" algorithm that just excludes the null byte to retain more entropy within 72 bytes.

<comment><USER9> Much faster than all of those multiplications that invertibly maps 64 bytes to 65 bytes without nulls: replace first null with 255. Every later null, replace with the index of the previous null. Make the final byte the index of the last null (or 255 if no nulls were replaced). In this way, you've replaced the nulls with a linked list of the locations where nulls used to be. To invert the transformation, just start at the final byte and walk the linked list backward until you hit a 255. (You'd never do the inversion in practice, but the existence of the inversion algorithm proves that no entropy was discarded.)

Use the time saved by not doing base 255 conversion to increase your iteration count.

<comment><USER7> Any time you spend or save on these kinds of transformations is going to pale in comparison to the bcrypt step. It might not even amount to a single iteration.

</comment></comment></comment></comment><comment><USER10> It's nice to store passwords securely, but it's also important to remember to, you know, actually verify them afterwards ;)

[https://techcrunch.com/2011/06/20/dropbox-security-bug- made-...](https://techcrunch.com/2011/06/20/dropbox-security-bug-made- passwords-optional-for-four-hours/)

<comment><USER11> That is literally five years old at this point and is at best, a cheap shot. Let's be better.

<comment><USER12> Also, Dropbox has more people working on security today than the total number employees Dropbox had in 2011.

<comment><USER13> Also, they have written a unit test to make sure this cannot happen ever again.

</comment></comment></comment></comment><comment><USER14> As someone who exclusively uses a password manager with random unique passwords for each service it always amuses me to see posts like this.

Years ago I relieved myself from the stress by using a password manager. Now for all I care they could be storing it in plaintext and it wouldn't make a damn difference to me. Problem solved.

<comment><USER15> Just curious which one do you use?

<comment><USER16> I (not OP) use pwsafe on macOS and iOS, a port of Password Safe [1] designed by Bruce Schneier.

It syncs the encrypted blob to iCloud, so I can pull up passwords on the iPhone if necessary. It is fairly simple, not much browser/OS integration - you just open the app, choose a safe/blob, enter the master password, and can then browse/edit your list of passwords, and in particular copy a password.

Simple, not too much functionality - not too much that can go wrong, I hope. (Often it has been the browser integration that lead to exploits in LastPass/1Pass, if I'm not mistaken)

[1] [https://www.pwsafe.org/relatedprojects.shtml](https://www.pwsafe.org/relatedprojects.shtml)

</comment></comment></comment><comment><USER17> _We considered using scrypt, but we had more experience using bcrypt._

Ok, fair enough...

 _The debate over which algorithm is better is still open, and most security experts agree that scrypt and bcrypt provide similar protections._

... wait, what?

<comment><USER5> They're badly expressing the idea that there may be only marginal benefits to optimizing the genus of password hashes used, so long as you're using a serious construction designed for storing (or generating keys from) passwords.

The debate over whether scrypt is better than bcrypt is not really still open. The debate over whether the difference matters that much in practice might be.

For what it's worth: for new systems, I use scrypt. But if someone asked, and they didn't have a very specialized application, I'd tell them that switching to scrypt from bcrypt, or even PBKDF2, would be a waste of money.

<comment><USER17> Right, if they had stopped at "we used bcrypt because we're familiar with it and we think it's good enough for our purposes", I wouldn't have said anything. But the second sentence, claiming that there's an open debate about which provides more protection...

<comment><USER5> Yeah, sorry, I should have led with "I agree with you".

No part of this post inspired confidence with me. I use Dropbox to store and share screenshots on Twitter, and try not to use it for anything else.

<comment><USER17> _I use Dropbox to store and share screenshots on Twitter_

Wildly off-topic, but I just upload screenshots to twitter directly and let them figure out the hosting. Is your usage of dropbox for this simply a legacy of when twitter didn't have support for uploading images?

<comment><USER5> Probably, yes.

</comment></comment></comment></comment></comment></comment><comment><USER18> Here's how facebook does it: [http://chunk.io/f/72f9c680ac2a4777b6dbf33c532e1d3c.jpg](http://chunk.io/f/72f9c680ac2a4777b6dbf33c532e1d3c.jpg) (Alec Moffat talking at RealWorldCrypto)

Seems like the combination of strong hash + encryption on a HSM is the way to go these days. Dropbox's scheme looks good to me.

</comment><comment><USER19> One concern I have here, is that people are going to perceive this post as "this is what you should do and it's easy!", because the post doesn't really address the complexities of implementing this kind of thing.

As a result, we're probably going to have a bunch more issues like this one: [http://blog.ircmaxell.com/2015/03/security-issue- combining-b...](http://blog.ircmaxell.com/2015/03/security-issue-combining- bcrypt-with.html)

I'm not looking forward to having to talk people off that particular ledge for the next several months...

</comment><comment><USER20> Cool approach, you need to compromise two separate servers just to have a usable password database you could run tools against. A key compromise can be fixed quickly and a password compromise is useless without the key.

<comment><USER21> Of the last 10 or so security engagements I have done, I can only recall one where I wasn't able to compromise _all_ servers. Once you get the first few, the incremental work to get everything is relatively small.

When breaking in, your end goal isn't the database server... it's the domain controller or the configuration management server.

</comment></comment><comment><USER22> Does anyone know what is a good practice to create a "vault" \- the kind that is used for the Pepper in this case?

I have heard of it being a separate, ip restricted server with daily changing ip address, etc. A simpler use case would be to store oauth2 tokens or some kind of PII

<comment><USER23> I've heard before of it just being stored in the codebase. Doesn't add much security but it does mean both the database server and at least 1 of the app servers or the codebase have to be breached

<comment><USER24> > I've heard before of it just being stored in the codebase.

Yuck! At the very least put it in an environment variable. Best case is loaded once at server boot from an HSM, kept only in memory, and rotated on a regular basis.

Having it in the code, like all other config, is a terrible idea.

<comment><USER22> If we don't have a hsm? Apparently,neither does Dropbox. So how do you engineer one ?

<comment><USER25> Use something like Vault: [https://www.vaultproject.io/](https://www.vaultproject.io/)

</comment></comment></comment></comment></comment><comment><USER26> > Some implementations of bcrypt truncate the input to 72 bytes, which reduces > the entropy of the passwords.... By applying [SHA512], we can quickly > convert really long passwords into a fixed length 512 bit value, solving > [that problem].

This part confused me. How can truncating to 72 bytes be a more severe reduction in entropy than generating a 64-byte hash?

<comment><USER27> Password lengths are variable. With passwords longer than 72 ASCII characters, you will lose entropy after that.

Let A be a 72 character long string, and B be A + X. Regardless of what X is, when bcrypted the result for A and B will be the same.

<comment><USER28> A random X does not reduce the entropy of bcrypt(B), just fails to add any additional entropy beyond bcrypt(A)'s.

Assuming C, where len(C) < 72, I don't know if it is possible at all to chose some value of Y such that:

D:= C+Y

Entropy(bcrypt(D))< Entropy(bcrypt(C))

<comment><USER29> "Lose" meaning it is thrown away. Not "lose" meaning it subtracts.

</comment></comment></comment></comment><comment><USER30> _If we use the global pepper for hashing, we can't easily rotate it._

I don't get this point. Why is it harder to rotate pepper for a hash compared to an encryption key?

<comment><USER31> Because encryption can be decrypted with the correct key. A hash can't be reversed once you've hashed something with the pepper.

<comment><USER30> Hmm. Ok. That's true. Thanks!

</comment></comment></comment><comment><USER32> A bit of an overkill, no? Doesn't bcrypt suffice?

<comment><USER30> Assume their salt+hash database leaks. It's true that salting the passwords and using bcrypt would prevent mass cracking of the database (i.e. you would have to crack each password individually, not the entire database at once), it would still be feasible to crack a single user's password if it was weak enough (which would be worth it, if the user is, say, firstname.lastname@example.com).

Using pepper prevents that from happening, and storing it separately from your database makes it much harder to get both.

<comment><USER33> > it would still be feasible to crack a single user's password if it was weak > enough

I'm sorry, but how? You'd need data centers worth of effort for many years.

<comment><USER30> They say that the bcrypt hashing takes 100 ms on their servers. If we take that as our limitations, it means we can try 10 passwords per second. So if you had a dictionary of common passwords, plus the salt, you could try 36000 passwords per hour. If the password is "password12345" (hence "weak enough"), you could feasibly crack that.

(I should say I'm not a professional security engineer so I'm probably wrong about everything and would appreciate corrections. )

<comment><USER3> Great analysis (that in my experience is largely accurate).

Just want to tack on one footnote: 100 ms is based on a single CPU core. Most CPUs have 4-8 cores. So instead of 10 passwords a second you could argue 40-80 passwords a second is believable with concurrent operations (which all popular hash cracking software supports).

It is definitely viable to break a single user's hashed password no matter what the scheme or work factor. Strong hashing algorithms with high work factors just stop you breaking multiple user's passwords quickly (and gives you 1-3 days of delay). It is a stop gap, not impenetrable defence like some believe.

All technical people need to take a day and learn how to break password hashes. Not just the theory, but go download the software and actually do it.

<comment><USER34> > Not just the theory, but go download the software and actually do it.

And now that hashcat and oclhashcat have merged[1] and are open-source[2] it's never been easier to install hashcat and start cracking passwords!

[1] [https://hashcat.net/forum/thread-5559.html](https://hashcat.net/forum/thread-5559.html) [2] [https://hashcat.net/forum/thread-4880.html](https://hashcat.net/forum/thread-4880.html)

</comment></comment></comment></comment></comment></comment><comment><USER35> I would be interested in the details of the storage mechanism of the global pepper. Is this in an HSM? For AWS customers, something like KMS? There are then huge operational and redundancy issues to think about. Failovers for your HSM. Handling the possibility that AWS might not be available or corrupt the key, other cases. These things are easy to whiteboard, but when the rubber hits the road and you need to think about all the operational edge cases, things get hard quick.

<comment><USER4> It's not in an HSM. Dropbox states towards the end of the article that they're exploring HSM applications for pepper storage, which I think is a great idea. If I recall correctly, Facebook is also exploring (or has already implemented) an HSM for password database secret key storage.

You raise good points though. This system is significantly safer than best practices (bcrypt(password, 10)), but it has significantly more overhead. There's also diminishing returns here. For a company of Dropbox's size - sure, invest in this. For a company that came out of YC S16, no, don't bother. Just properly bcrypt/PBKDF2/scrypt/argon2 the thing and revisit much later.

I love it, but I would not recommend this system to my clients for password storage unless they had a very mature operations/reliability team.

</comment></comment><comment><USER36> Realistically, how much better is this than the standard bcrypt recommendation? I don't mean for a company the size of Dropbox/Facebook/etc., I mean in general, will this really be much more useful than just using bcrypt? Using an encryption key means that if the database is compromised, as long as the OS isn't (or wherever the key is being stored), the passwords are encrypted in a way that's effectively impossible to decrypt, which is nice. However, are they sure that hashing the password first before hashing it in bcrypt won't cause issues?

Unless Dropbox employs or contracted someone to verify that this is okay (not an engineer, a mathematician/cryptographer who can understand the math behind the algorithms) I'd be hesitant about it. Same goes for other companies that do some complex sequences of hashing e.g. Facebook. Implementing the idea is engineering related, but verifying it is not, and I don't trust engineers (including myself) to verify that a specific algorithm or sequence of algorithms is valid.

</comment><comment><USER37> From the diagram, Dropbox stores no passwords: it stores an encrypted hash (hasing in two steps, SHA512 and then "bcrypt") of the password. I.e. stored = AES256(bcrypt(SHA512(password), per_user_salt, 10), global_key).

I would like to know if "salted-bcrypt"+SHA512 hashing is really safer than using just SHA512 (e.g. because of the risk of making locating hash collisions easier, etc.).

<comment><USER38> I posted a separate comment, but will post here again...

Dropbox do store one password AFAIK. It appears that they store OSX users administrator password... I am keen to see if they address this somewhere.

See discussion: [https://news.ycombinator.com/item?id=12457067](https://news.ycombinator.com/item?id=12457067)

<comment><USER39> No, they use the admin access they obtain from you to modify the system so they can obtain admin access again in the future. They do not ever have your admin password.

</comment></comment></comment><comment><USER40> Their solution is very similar to the mode prescribed by [1] and implemented in [2].

There are actually two problems with bcrypt:

      - It truncates after 72 characters
      - It truncates after a NUL byte

If anyone is dead set on following Dropbox's example, make sure you aren't passing raw binary to bcrypt. You're playing with fire.

Additionally, if you're going to use AES-256, don't implement it yourself. Use a well tested library that either uses AEAD or an Encrypt then MAC construction.

[1]: [https://paragonie.com/blog/2016/02/how-safely-store- password...](https://paragonie.com/blog/2016/02/how-safely-store-password- in-2016#why-bcrypt)

[2]: [https://github.com/paragonie/password_lock](https://github.com/paragonie/password_lock)

<comment><USER41> nitpick: encrypt-then-MAC _is_ an AEAD construction.

<comment><USER40> Nitpick to counter your nitpick: Not necessarily.

It's an AE construction, but not necessarily an AEAD construction. You can have IND-CCA3 security without additional data.

The converse is also not necessarily true (i.e. an AEAD scheme could be based on MAC-then-Encrypt and IIRC there are some that _are_ built that way), but that's a less useful counterpoint. The recommended AEAD constructions (AES-GCM and ChaCha20-Poly1305) _are_ EtM.

</comment></comment></comment><comment><USER42> How dropbox "NOW" securely stores your passwords

</comment><comment><USER43> While this is very impressive, it feels like trying to solve the wrong problem. The real problem is getting rid of passwords (Persona, anyone?).

Don't get me wrong, what's described there is super-important to secure the authentication of today, but what about a word for the authentication of tomorrow?

There already are various solutions. Passwordless[0] is a familiar one for nodejs, and I recently bumped into the promising Portier[1], which is, according to its authors, a "spiritual successor to Mozilla Persona".

[0] [https://passwordless.net/](https://passwordless.net/)

[1] [https://portier.github.io/](https://portier.github.io/)

<comment><USER44> So I have to login to my email every time I want to login to something using passwordless? Doesn't this just move the problem and create a few others?

<comment><USER45> For most companies offloading your password management onto an email provider is the right way to go. Suddenly, for free, you get MFA, a dedicated security team, and you'll never need to do one of those "Our password database has been hacked. Here's what we're doing..." press releases.

You've eliminated one point of failure (your company), and haven't added any because you are already doing email based password resets.

You can delete all your password related stories from trello or whatever.

You eliminate all the bike shedding around how to store passwords.

You've improved your initial user experience by an order of magnitude. Everyone dreads setting up yet another account password. Don't underestimate the joy a user feels when the signup form is just "click one of these buttons or fill in the email field". (The buttons are 'Connect with Facebook' and 'Connect with Twitter').

Users would much rather flip over to email (which is always logged in anyway) and click a link (especially on a mobile device) than enter a login/password.

</comment></comment></comment><comment><USER46> The blog mentions, "We're considering argon2 for our next upgrade". I suppose they could do in-line upgrades: as users are signing in, the SHA512 is piped through the old pipeline for verification and through the new pipeline for migration. As far as I can tell, there's no way for them to swap bcrypt out for argon2 using just their cold store.

</comment><comment><USER47> > Some implementations of bcrypt truncate the input to 72 bytes, which reduces > the entropy of the passwords. Other implementations don't truncate the input > and are therefore vulnerable to DoS attacks because they allow the input of > arbitrarily long passwords.

Huh? BCrypt works by stuffing the password into a 72 byte Blowfish key and using it to recursively encrypt a 24 byte payload. Either it's truncating, or it's pre-hashing the password to fit much like they are.

The link they use to justify it is funny: [http://arstechnica.com/security/2013/09/long-passwords- are-g...](http://arstechnica.com/security/2013/09/long-passwords-are-good-but- too-much-length-can-be-bad-for-security/)

That's just a naive PBKDF2 implementation that's pointlessly reinitializing the HMAC context each iteration instead of just doing it once at the start. The difference between storing a 1 byte and a 1MB password with PBKDF2 should be on the order of a couple of milliseconds.

<comment><USER41> Having the SHA-512 hash at the beginning simplifies the implementation because the "security" code only needs to handle 64-byte random strings (which are truncated to 54-byte strings for `bcrypt`, but still...). That removes all sorts of stupid edge cases that come with variable-length strings.

<comment><USER47> > Having the SHA-512 hash at the beginning simplifies the implementation

The hash is there to ensure very long passwords contribute entropy to the final hash instead of being truncated. It also ensures the entropy is evenly distributed - every bit of the password affects every bit of the hash.

> the "security" code only needs to handle 64-byte random strings

You can't feed any typical BCrypt implementation a raw SHA-512 hash because it's not binary safe - it truncates at the first NULL byte.

Well, you can, and it'll appear to work, but it'll be laughably easy to break. It's a pretty stupid sharp edge IMO.

> which are truncated to 54-byte strings for `bcrypt`

72 bytes, because that's the size of the key array. _56_ bytes is just where extra entropy helps less, because the last 16 bytes don't affect every bit of the output.

> That removes all sorts of stupid edge cases that come with variable-length > strings.

It's just treated as a circular buffer. And what are we doing here, implementing our own version of BCrypt? Yeah, that certainly simplifies things :P

</comment></comment></comment><comment><USER48> So just to reiterate, taking the sha256 of the password before running bcrypt on it is recommended? Funny, this is the first I've heard of this. You'd think bcrypt would have just implemented the sha256 step into the algorithm?

</comment><comment><USER49> Really surprised to see they are not using a HSM yet for the global pepper. What kind of physical controls put in place for global pepper currently?

</comment><comment><USER50> pepper = sha256(global_randnum + user_start_dt + username + global_randnum)

the above creates a per user pepper, and largely obviates the need for pepper rotation.

if only password db is stolen, all users are impervious to dictionary style attacks.

if only codebase is stolen, then they only know how to calculate each pepper per user.

if both are stolen, you are in no worse shape than just using a salted bcbrypt, scrypt, or PBKDF2.

</comment><comment><USER38> I am wondering how they store OSX users administrator password, since it isn't being hashed - they actually store the password somewhere... it would be nice if that were addressed somewhere.

See discussion: [https://news.ycombinator.com/item?id=12457067](https://news.ycombinator.com/item?id=12457067)

<comment><USER51> I thought they created a setuid executable on first run (when they asked for your password), and then use that later?

<comment><USER38> Admittedly I don't know what that is and I don't know a lot about how applications on OSX get permissions. From reading that thread I posted I was under the impression that Dropbox stored the password because it was able to reinstate itself as an accessibility service as many times as it liked without having to ask for the admin password.

From reading, that wasn't supposed to be allowed. The only way that could work would be if Dropbox kept your password on file. In effect meaning that the dialogue you entered your admin password for wasn't a system modal - <USER2> rather a dropbox modal imitating the system one.

<comment><USER52> _I was under the impression that Dropbox stored the password because it was able to reinstate itself as an accessibility service as many times as it liked without having to ask for the admin password_

That is not necessary. A SUID binary owned by root runs with root's privileges. So, they only need the administrator's password once to install the SUID binaries. Afterwards, they have their own 'backdoor' to reinstate the accessibility settings, without needing an administrator password.

So, when they deny storing your password, it's probably true, they don't need it.

(If you are not convinced, write a small C program that executes a shell, compile it, make root the owner, set the SUID bit. You can be in a root shell without ever typing a password. This is why it is a good practice to have as few root-owned SUID binaries as possible.)

</comment></comment></comment></comment><comment><USER53> I wonder why Dropbox didn't mention its robust support for 2nd factor authentication?

</comment><comment><USER54> Now if only their customer service had been 'quietly' improved over the years.

</comment><comment><USER55> In 2016, shouldn't we be using public/private key pairs instead of passwords?

<comment><USER56> The problem is, in my opinion, that the average person will have no concept of how to manage a public/private key setup.

</comment></comment><comment><USER57> What is wrong with bcrypt with cost 13. It makes 8194 cycles on CPU...

</comment><comment><USER58> May I ask a potentially dumb question? Why store my password at all?

<comment><USER59> How else are they supposed to verify your password when you attempt to log in?

<comment><USER60> Email a link with token to log in.

<comment><USER61> That is a good way to get endless token links spammed at your inbox...

<comment><USER60> Gmail tokens tab.

</comment></comment></comment></comment></comment><comment><USER62> too little, too late. Once you loose trust, you never gain it again.

</comment><comment><USER63> So when is Dropbox going to allow users to encrypt files client-side before getting synced to its servers? It should be relatively trivial from both a technical point of view and a UX one.

</comment><comment><USER64> This "promise" model of security needs to end. I "promise" I'll encrypt your password, honest.

<comment><USER65> Forgive my ignorance, but are there any companies that provide proof that they encrypt users' passwords?

<comment><USER64> No. This is impossible. The whole model is broken. A less broken model is the use of asymetric keys for authentication that doesn't require the service provider to "promise" they'll keep your password secret.

<comment><USER66> This model, of course, is broken in its own way, in that if a user loses their private key, all their data is lost; there's possible recourse or password reset. It's also broken in that, if the company believes that a user's private key was compromised by a third party, they can't completely destroy that key until the user manually logs in and changes it.

<comment><USER64> Well, one could always leave one's private key with a trusted third party. So the public key model can accomodate those who are uncomfortable with the responsibility of keeping their private key secure. The password model however does not allow for allodial title to a secret.

</comment></comment></comment></comment></comment><comment><USER55> Why won't people stop using passwords? It is 2016!

</comment><comment><USER67> Is this before, or after, Condoleezza Rice uploads them to the NSA?

</comment><comment><USER68> Isn't this the company that authenticated all production users without checking passwords for a few hours, a couple years back?</comment>


How WePay Failed Eden Alexander - <AUTHOR> http://kittystryker.com/2014/05/not-waving-but-drowning-how-wepay-failed-eden-alexander/ <comment><USER2> This is one of the areas where bitcoin actually has value as a payment system (vs. as an investment asset or other more sophisticated things) -- industries which are suffering from gatekeepers.

If WePay could process bitcoin payments, they wouldn't be beholden to essentially the lowest common denominator of bank policies at the card associations. This would disproportionately help marginalized groups, and would be awesome.

So, rather than hating WePay, I'd rather just ask them to allow bitcoin. It's ironic that WePay started in response to PayPal (who are worse than the banks by some factor), but has now gotten to the point where they banks are preventing them from getting too much better.

<comment><USER3> Or you could just follow the rules with whatever payment processor you choose. Adult content is not a problem, you just have to pay more for it because the fraud rates are much higher than for non-adult content.

<comment><USER2> If no-adult is what you built your business model around, you're probably not going to be able to do adult rates due to 1% of your business being adult and make it work.

<comment><USER4> Surely there are ways to set up paperwork to separate " FooBar Prodictions" from "FooBarXXX Productions"

</comment></comment></comment></comment><comment><USER5> To anyone that is considering using WePay, I STRONGLY urge you to select Balanced Payments instead. I personally had a $2,000+ order cancelled by WePay, even though my account was 2 years old and had done many thousands of dollars worth of previous transactions. The "stated reason" by their support team was that I included a line item for an "electronic physical item" in the invoice displayed on the checkout page, which was for consulting purposes. If you actually check their prohibited businesses/items, electronic items are not included.

Overall, WePay's support is generally horrendous. You should expect to have your orders randomly cancelled, funds frozen, and no explanation given... Unless of course you're a major figure and raise a big deal about it.

</comment><comment><USER6> Our response: [http://blog.wepay.com/post/86048891401/wepays-terms-of- servi...](http://blog.wepay.com/post/86048891401/wepays-terms-of-service-as- it-relates-to-the-adult)

<comment><USER7> Feel free to check out Eden's twitter and take some time to appreciate how your approach has affected her (ie she's now in hospital), then consider changing the way you deal with issues like this.

<comment><USER8> WePay has the same terms as PayPal, Stripe, etc. It's not out of some moral choice on their parts, it's because all of the major banks in the US deem adult content as risky and forbid it.

<comment><USER7> this is ridiculous.

1) this wasn't about porn, it was about a seriously ill woman who needed help

2) she retweeted some stuff from third parties who weren't part of her campaign, probably out of ignorance, not realizing it might be a problem

3) wepay didn't do the sensible thing and just talk to her and let her know she might be breaking the ToS, they closed down her campaign, her situation escalated and now she's in ER

4) It's amazing that people just say banks deem adult content as risky, and just accept that this is ok. It only just recently broke that banks are actively closing down porn actors bank accounts just because of their profession.

<comment><USER9> > _4) It 's amazing that people just say banks deem adult content as risky, > and just accept that this is ok._

That's because it _is_ risky. No question. Adult services have a higher rate of charge-backs and fraudulently obtained cards than many other sectors.

<comment><USER10> Statistics?

</comment></comment></comment></comment></comment></comment><comment><USER11> This is first and foremost a failure of the United States and its dysfunctional health care system. WePay made it worse, but she shouldn'tve had to collect donations to pay for essential medical care in the first place!

</comment><comment><USER12> Related material: Cindy Gallop's struggle to get ANYTHING for MakeLoveNotPorn.com - which has a fantastic, social mission. She can't even get a bank. We've been working our banking partners to see if we can find a fit, but no takers.

Here's a really great talk on it if you're interested: [http://www.swiss- miss.com/2014/05/cindy-gallop-make-love-not...](http://www.swiss- miss.com/2014/05/cindy-gallop-make-love-not-porn.html)

</comment><comment><USER13> So... Who is the "stripe/wepay/paypal" of the adult payments industry?

<comment><USER14> ccbill

</comment></comment><comment><USER15> Is there a legal reason for being so conservative about financing porn? Or is this purely a moral stance that WePay is taking.

<comment><USER3> It's traditionally a very high fraud industry (tons of stolen credit cards, charge backs, etc). There are high-risk processors that specialize in it, but with higher fees than your typical processor. WePay's agreements with who they send payments through require that the transactions meet a certain standard and to get the rates that they get one of those standards is no adult content.

<comment><USER16> But why did the owner simply retweeting other tweets makes it higher risk?

<comment><USER17> For WePay it's not about risk, it's about being compliant with the rules laid down by the bank they use. The banks don't want to be involved with payment processors who process adult-related transactions, and if companies like WePay don't play ball with the banks rules then the banks walk away and WePay has to close.

There are probably situations where a company like WePay has to take the stance of better safe than sorry and not piss off the bank, because if the bank gets pissed and walks then bye-bye WePay.

So when WePay sees the owner retweets the tweets encouraging violating the TOS, they can't risk losing their relationship with the bank that processes their payments. On the one hand they might alienate some customers or put someone in an unfortunate situation, on the other hand their business collapses because they can no longer process any payments. Shutting down some accounts like this one is sad, but for payment processors it's their only real choice.

<comment><USER10> Why was WePay monitoring the owner's twitter account at all?</comment></comment></comment></comment></comment>


Google Ventures - <AUTHOR> http://www.googleventures.com/

<comment><USER2> It's interesting that while Google is well established worldwide, Google Ventures has 4 offices in the US and none abroad.</comment>


How to Build a Growth Team – Lessons from Uber, Hubspot, and Others (50 Slides) - <AUTHOR> https://andrewchen.co/how-to-build-a-growth-team/ <comment><USER2> Great article, thank you for sharing! Favorite quote:

"Too many startups are beginning with "I need a growth team!" and accepting a random org configuration, without thinking it through from the fundamentals. Ultimately, You have to start with the problems you are trying to solve. Begin with the KPIs, the insights you've generated, and then move onto execution. You staff the problem area and the type of execution you want. The organizational structure follows from there."</comment>


Tell HN: I'm going to stop using Twitter if it doesn't stop cryptocurrency ads - <AUTHOR> The relentless flow of garbage, lying ads trying to get me to buy bitcoin or floobcoin or whatever.
Always disguised as credible tech news. Somehow I found myself reading some article about how Bill Gates and Richard Branson together had decided to back some bitcoin thing which I wasted time reading until I realised it was garbage.
The thing Twitter does not understand is that straight out false advertising dressed up as valuable content diminishes the trust in the Twitter platform.  After I read the stupid article above I felt less inclined to go back to Twitter, but seemingly Twitter is willing to pay this price for that juicy cryptocurrency advertising cash.  It makes Twitter feel spammy and trashy.
Twitter, please stop it. <comment><USER2> They banned them a few weeks ago. [https://techcrunch.com/2018/03/26/twitter- moves-to-ban-crypt...](https://techcrunch.com/2018/03/26/twitter-moves-to-ban- crypto-ads/)

<comment><AUTHOR> Not on the Twitter I'm using.

</comment></comment><comment><USER3> They are getting banned on other platforms: FB, Instagram, etc.</comment>


Why Cloud Based Load Testing Is A Killer App - <AUTHOR> http://johngannonblog.com/2009/03/04/why-cloud-based-load-testing-is-a-killer-app/

<comment><USER2> "Virtual Stress-free Testing in the Cloud" [http://aws.typepad.com/aws/2009/02/virtual-stressfree- testin...](http://aws.typepad.com/aws/2009/02/virtual-stressfree-testing-in- the-cloud.html)

Has links to several load testing products using EC2.

</comment><comment><USER3> Did he link to Apple's Keynote software as a "load testing managed service provider"?

<comment><USER4> He must have confused Keynote with Keynote. <http://www.keynote.com/>

<comment><USER5> Thanks for the catch - I use Zemanta to speed up the linking and tagging of posts and I didn't check to make sure that the proper Keynote was linked :) In any case, it should be fixed now.

</comment></comment></comment><comment><USER6> What a nonsense article.

Makes it sound as if the script-playback was the difficult or expensive part of a load-test. In reality that is the _trivial_ part.

<comment><USER7> That's not how I read it, I think this actually makes a lot of sense.

In a regular (non-cloud) deployment, there are unavoidable differences between the staging and real production configurations. No one can afford to double their hardware costs for the sake of uber-realistic load testing results. So people usually opt to create a closely matched staging environment where testing is performed. Every once in a while something slips by that's only reproducible on production, and you're stuck debugging a critical and hard to reproduce bug. Not the best combination.

With cloud computing, you're supposed to have lots of redundant computing capacity. So it's trivial to test in an environment that's identical to production by just asking for more processing power for the purposes of testing. As an added bonus you also have the distributed resources to generate considerable parallel load, which also helps.

That last part like you said is not that hard. But that's only the icing. The cake is being able to test on a 1-1 replica of production. That's precious.

<comment><USER6> Hmm indeed, from that angle it makes more sense. I guess I skimmed it a bit too lightly. But I'm still not overly impressed by the article as this is really just stating the obvious - basically a rehash of the standard cloud- propaganda that has been raining down on us for years.

Nonetheless I take back my "nonsense" statement. That was too quick a judgement...</comment></comment></comment>


PSD is not a good format. PSD is not even a bad format. - <AUTHOR> https://code.google.com/p/xee/source/browse/XeePhotoshopLoader.m?r=f16763d221dfca6253983824b470adf553a19e06#108

<comment><USER2> Awesome that he understood the detailed history and requirements of the whole format after spending a few weeks with it, huh?

<comment><USER3> Awesome that you could read his mind after reading a 4-paragraph rant, huh?

</comment></comment><comment><USER4> It's by design, I guess. No software should be able to properly im-/export that format. A little bit like Microsoft formats.

<comment><USER5> A case where security by obscurity ensures the future of the product, for now.

I guess this is similar to Facebook / g+ / etc refusing to have easily imported / exported data - if you can export your data to a useful format, you can move it to a rival product.

<comment><USER6> Facebook isn't about data, it's about friends. Most people move to a rival product once many of their friends have done so.

<comment><USER5> Facebook doesn't have, and won't ever add, an 'export my photo albums, messages and friends list as a series of easily parsable xml files' button, because that would be building redundancy into the product. One of the ways that it locks you in is to to secure all this data, all this work that you've done. The fact that your friends all use it is obviously another significant factor.

Similarly, I CBA to switch from Spotify, because I've got all of my playlists saved in it. If I moved to google play or one of the other up and coming music services, I'd have to rebuild these manually. There are some scraping tools to get this data out but they tend to be _painful_ \- "Just right click on each song and click 'copy HTML link'" etc ...

<comment><USER7> Actually Facebook has, and has had for the past few years, a "Download your data" option within the "Account Settings" page. That system collects your posts, messages, friend list, events, photos, and albums into one zip file. The structure isn't terrible (HTML), but it changed fairly often.

</comment></comment></comment></comment></comment><comment><USER8> Some previous discussion: [https://news.ycombinator.com/item?id=4746787](https://news.ycombinator.com/item?id=4746787)</comment>