How Amazon Plans To Use Whole Foods to Dominate the Retail Industry - <AUTHOR> http://fortune.com/longform/amazon-groceries-fortune-500/ <comment><USER2> I'm amazed at how little discussion there is of Amazon using WF as a beachhead to go after the grocery warehousing and distribution industry.

Retail is just one touchpoint for administering a tax on the industry. Warehousing and distribution is a far more centralized and efficient infrastructure play for a company like Amazon than trying to go after all the individually managed and differentiated retail stores.

Does Amazon understand how to do cold warehouse storage and shipping of perishable goods? Not yet. Can they? Certainly.

<comment><USER3> In 1992, I worked in a precast/prestressed concrete plant. One of the projects was a 500' fire wall for a Walmart distribution center near Brooksville, Florida. One side was dry goods, the other side was cold goods. There was 1000 linear feat of wall because it was double panel (as fire walls often are). That cold/perishable side was up and running 25 years ago. Before Walmart was even "in the grocery business". Of course before Amazon...Bezos hadn't even left Wall Street.

The hype about Amazon and Whole Foods ignores the real logistical challenge of the grocery business at scale. Catching up with Walmart means delivering fresh food everywhere 24 hours a day, not just "in select markets during business hours." I'd put it this way, Walmart's infrastructure can move upscale in the grocery market more easily than Amazon/WF's infrastructure can scale up for grocery sales. A significant amount of Amazon's existing inventory sits as stock in a warehouse. Amazon is not nearly as driven by just in time contracts with suppliers as Walmart.

<comment><USER4> >Walmart's infrastructure can move upscale in the grocery market more easily than Amazon/WF's infrastructure can scale up for grocery sales.

Can Walmart move their brand upscale? That may be more of a challenge than infrastructure.

As purchase trends continue to shift, Walmart needs to effectively service low-income customers in-store and middle- and high-income customers online (and perhaps occasionally in-store for pickups). Their core brand needs to represent different things to different customer segments. Not easy to do, but I'm glad they are trying.

<comment><USER3> Walmart carries organic produce, free range eggs, Dave's Power Seed, hormone free milk and whatnot. A shopper can put them in the same cart as their eighty cent Great Value Mac and Cheese and a box of Strawberry Pop Tarts. And Walmart will gladly take their money without judgment.

Walmart's business wasn't built on selling to low income people. It is built on selling to everyone in places like Fort Stockton, Texas. The person renting a dilapidated trailer and the rancher with 150,000 acres of cattle and oil leases both shop there because there aren't a lot of other practical options short of driving to El Paso. Not saying that there's not a significant difference in branding, but a lot of that difference is that Whole Foods brand images for people who are offended by Pop Tarts and HFCS on "their" store's shelves. On the other hand, Walmart's potential customer base is anyone who is not offended by shopping at Walmart for the vastest majority of grocery purchases.

<comment><USER5> I just finished the Sam Walton biography. I'm not sure of the author off hand (in case there was more than one) but you are spot on. Walmart started off in small towns where other stores couldn't justify a presence. It was a great day when a Walmart came to your town. The current evil mom-and-pop killing mega- corp is a more recent rhetoric.

<comment><USER6> I read it as part of my business degree. It's a great book, and a fascinating story: "Sam Walton: Made In America" by Sam Walton and John Huey.

</comment></comment></comment></comment></comment></comment><comment><USER7> Why is this implied assumption that Amazon will win, where Walmart seems much better positioned to win groceries ?

One possible scenario, for example: convenience is key. So people shift from cooking at home(only ~10% love to cook ) to buying fresh, restaurant quality, heat-and-eat meals.

Key parameters to compete here are quality, selection and price. Walmart could offer good quality via help of external chefs, if they get critical mass they could offer enough selection, and since they buy much more food than Amazon ingredients will come cheaper for them, supply chain will be cheaper for them, and and the only place Amazon may have an advantage is more efficient food making process, so there some chance Amazon could win where(altougj most like the innovation will come from a startup, which also Walmart can acquire) , but even so ,will it negate all of Walmart's advantages, and won't be copyable in some form ?

And remember, even for wealthy people that care less about price, better cost efficiencies could mean better food, better reviews or better paid marketing.

So What about other Amazon differentiators: Amazon Ecbo ? Google will offer that. Instead of just delivering stuff, putting them is people's fridges. Is that something that people do? Is that something so hard to copy ? And is that really a meaningful differentiation ?

<comment><USER8> > positioned to win groceries

Retail grocery sales != meal kits. In one instance, you simply procure the product and sell it on, and the other you procure, kit and sell via online. Huge difference.

> Key parameters to compete here are quality, selection and price. Walmart > could offer good quality via help of external chefs, if they get critical > mass they could offer enough selection,

Have you ever actually stepped into a Walmart? Walmart does not compete on any of those parameters, except for price.

> to buying fresh, restaurant quality, heat-and-eat meals.

Except this is not what is happening. Those companies have ridiculously high churn rates, which means the trend is fairly benign to certain sectors of the market (mainly large cities).

<comment><USER7> // churn:

Most meal kits companies will die because they are inconvenient. Too expensive, too much messing around in the kitchen.

But heat-and-eat is easy(just putting something in the oven), somewhat close to takeout easy, and Walmart may take it to really interesting price points.

// Quality, selection at Walmart

Online ordering greatly increased selection. Will do for Walmart.

Walmart order food at the quality and price point fitting customers. Assuming grocery pickup/delivery will attract wealthy urbanists(see my other comment), Walmart will offer also higher quality products

</comment></comment></comment><comment><USER9> > _The Seattle giant believes selling you groceries is the key to selling you > everything else._

I mean...they're not wrong here. The grocery store is primarily for selling you groceries, but it's also the first place most people stop for cookware, trashbags, deodorant, etc because it's right there. I am a guy who tries to buy "quality" equipment for anything and everything, and usually I do meticulous research before buying anything online (read the Amazon reviews, cross reference with America's Test Kitchen and Serious Eats guides, etc). Nonetheless, I've bought my fair share of food containers or occasional kitchen tool from my grocery store because I had some cookie dough in my basket and I needed a cookie sheet _right now_ or those cookies weren't going to happen.

</comment><comment><USER10> They really choose to start somewhat from the bottom then, doesn't Whole Foods have something like ~3%-5% of the grocery market?

Compare that to companies with comparatively massive market shares like Kroger and Walmart. I can see an argument being made that groceries is a good way to go because people have to eat, but the topology of the industry doesn't really have a medium sized company that you could pour gasoline onto to gain a lot of market share. Something in a different industry might have been a better play to pilot before moving into other retail spaces.

<comment><USER11> They're at the top in brand reputation and at the bottom in market share. People who care a lot about food and are willing to spend on it accordingly shop at Whole Foods. There are relatively few of these people, hence their small market share. (A similar dynamic happens in most consumer industries: Apple famously has only 18% of the smartphone market but makes 87% of the profits, because they market to people who are willing to spend a lot on a cell phone.)

This is a pretty rational choice for Amazon, because Amazon's core competency is making things cheaper and more widely available but they have little experience in making things better - and indeed, their whole value proposition is largely in making things commoditized, helping you buy a "good enough" product more conveniently for less money. With Whole Foods, they're buying the brand and the expertise in quality, and then they can apply their existing strengths in logistics and supplier management to wrangle down the price and expand market share.

<comment><USER12> People that like to spend a lot of money for the same product shop at Whole Foods. Smart people, and price conscious people, buy the same product for a fraction of the price at their local franchise grocer, whether that is Hannafords, Market Basket, Price Chopper, Publix, or what have you, wherever you are.

Walmart does have typically inferior grocery items, still; it's been 15 or so years since the super centers were built in this area with groceries, but the meat and produce is still well below par.

</comment></comment></comment><comment><USER13> My wife is literally at Whole Foods right now, picking up an item she bought from their Treasure Box today. She will most likely walk inside since she's "already there" and pick up some stuff at Whole Foods.

</comment><comment><USER14> I'm curious if Amazon will ever attempt something equivalent to a weekly neighborhood-wide prime grocery shipment receiving time.

Basically, every Tuesday at 10am-11am, my neighborhood will receive its weekly grocery shipments. If my other non-perishable prime packages happen to be in their holding warehouse by that time, I will get my non-perishable packages delivered to me at that time as well.

It would seem by bunching up the shipments it would save gas costs and last mile delivery costs.

<comment><USER15> Along those same lines, I'd totally stick a crappy IoT digital scale in my kitchen so that Walmart, Amazon, the NSA. etc can track my $regularly_purchased_item consumption habits so long as said crappy little digital scale orders me more of whatever that item is before I run out.

<comment><USER14> Yeah - I'm envisioning a refillable Amazon box for sugar, salts, cereal etc. that re-ordered themselves when low. Maybe using that Dash button technology.

Also a nice, paintable residential Amazon locker for home delivery to the front porch would be ideal.

I hear Amazon is working with Lennar these days but it seems to be all about Alexa.

</comment></comment></comment><comment><USER16> They should've bought Kroger if that was their intent. WF is too expensive for like 80% of the U.S population.

<comment><USER17> I disagree. The nice thing about WF is they have relatively fat margins, so that "margin space" gives Amazon a lot of room to operate. For example, they could provide cheaper delivery while still being profitable, just less profitable. Margins on traditional grocers like Kroger are so thin there is very little room to operate. With WF, Amazon could get a model that works and _then_ work on making it more broadly applicable (like WF has already started doing with their cheaper WF 360 stores).

<comment><USER16> I see what you're saying but I still think a more accessible store with more economically diverse location (most WF were built where could afford to shop there) would've been a better move. It would've put Amazon in the heart of more communities. Remember Walmart started by catering to poor people and it turned out to be the right decision. I'm not so sure starting out catering to rich people is going to be the right decision in the long run.

<comment><USER18> It worked for iPhone.

<comment><USER16> you mean, carrier subsidization worked for the iphone.

<comment><USER18> No, I don't. The iphone was $600 with subsidy. Other phones were free or less than $100 w subsidy.

</comment></comment></comment></comment></comment></comment><comment><USER19> Where does Instacart fit into this equation?

</comment><comment><USER20> A bit off topic but I wish the Costco membership had something like "Amazon Prime Video" for watching movies.</comment>


Why is encryption so hard? - <AUTHOR> So I see a lot of posts on HN that alude to difficulties when developing apps with encrypted comms. Do most languages not already have encrypt/decrypt libraries to leverage. E.g I would expect to be able to find a public-private (RSA) implementation in most languages, where I could do somthing like:
String encryptedMessage = RSAEncryptor.encrypt(publicKey, message)
Is this not the case? Do such libs have bugs? <comment><USER2> You can certainly find libraries with interfaces like that. For example, OpenSSL has extensive libraries for all sorts of cryptographic primitives and protocols.

If you take a narrow focus on a particular cryptographic event (such as your encryption of a string with an RSA public key) then you miss the greater story about encryption: it's not just the individual cryptographic primitive that needs to be implemented correctly, it's everything else.

An RSA encryption like that does not stand alone. Keys must be generated, secured and distributed. The RSA library itself must be validated to ensure that it works correctly. The actual primitive must be used correctly (in the case of RSA don't use a stupid exponent as some have done). And the environment within which the encryption is used must be understood and secured (just look at the CRIME and BREACH attacks against TLS to see how something 'secure' can be broken because of something apparently irrelevant, in this case, compression).

The overriding reason that encryption is 'hard' is that secure computer systems have enemies and those enemies (attackers) will do _anything_ to attack the system. They will attack it based on timing, compression problems, flaws in the protocol, freezing the RAM to extract a private key, etc. etc. There's really no end to the variety of things you can try to attack a cryptosystem.

So, building a secure system may have encryption as a necessary condition, but it's not sufficient. So much else can and will go horribly wrong.

If you are interested in this hit the books and understand the history of cryptography. For example, look at how Vigenere was broken by Babbage, or the Venona ciphers, or Lorenz. These 'old' ciphers can tell you a lot about how people actually attack things. Then read about modern ciphers and attacks on them. Wikipedia has much. Read about TEMPEST and imagine other attacks possible in that way.

<comment><USER3> "They will attack it based on timing, compression problems, flaws in the protocol, freezing the RAM to extract a private key, etc. etc. There's really no end to the variety of things you can try to attack a cryptosystem."

I will say that in my experience 95% of the attacks are going to be social engineering, less about sophisticated things when the social is way easier.

<comment><USER4> The problem is that while probably FEWER than 5% of attacks are technical, they generate industrial scale issues -- e.g. tens of thousands of stolen identities.

Even so, while there have been high profile examples of encryption algorithms being shown to be flawed (e.g. RSA had to reissue all its dongles a couple years back because they were using a flawed RNG) I do not know of any actual successful dark-hat attacks along those lines (of course they may have occurred undetected or not been disclosed or I may simply be ill-informed).

High profile security breaches are generally a result of poor or no cryptographic practices, negligence (e.g. IEEE keeping its member records in a plain text file on an FTP server), or (as you say) social engineering.

In short, while really good cryptography may be hard, halfway decent is not hard, so it becomes a case of "assumed hard and left untried" rather than "tried and found hard".

Finally: there's also the problem of security theater, such as forcing people to change their passwords at a ridiculous rate.

</comment></comment></comment><comment><USER5> Lots of good answers here. NaCl (salt) is one (relatively) recent effort to be just such a library, see eg under the sub-heading "High Level Primitives" on the features page:

      http://nacl.cr.yp.to/features.html

      High-level primitives

      A typical cryptographic library requires several
      steps to authenticate and encrypt a message.
      Consider, for example, the following typical
      combination of RSA, AES, etc.:

      * Generate a random AES key.
      * Use the AES key to encrypt the message.
      * Hash the encrypted message using SHA-256.
      * Read the sender's RSA secret key from
        "wire format."
      * Use the sender's RSA secret key to sign the
        hash.
      * Read the recipient's RSA public key from wire format.
      * Use the recipient's public key to encrypt the
        AES key, hash, and signature.
      * Convert the encrypted key, hash, and signature to wire
        format.
      * Concatenate with the encrypted message.

      Sometimes even more steps are required for storage
      allocation, error handling, etc.

      NaCl provides a simple crypto_box function that
      does everything in one step. The function takes the
      sender's secret key, the recipient's public
      key, and a message, and produces an authenticated
      ciphertext. All objects are represented in wire
      format, as sequences of bytes suitable for
      transmission; the crypto_box function
      automatically handles all necessary conversions,
      initializations, etc.

Of course, "such libs have bugs" \-- it is software after all. But bugs can (and will be) fixed.

Somewhat unique to security and cryptography are the number of subtle bugs possible. There are both problems of actual "normal" bugs (like the Debian entropy bug) and system level design errors (like CRIME).

NaCl/Salt tries to reduce the number of errors possible by using the library wrong (as opposed to eg: openssl that has a very (some say too) rich interface). But you could still end up writing the secret key to swap. Or doing something silly with the plain text. Or expose yourself to a buffer overflow in the part of the code that renders those cute avatar-images for your chat application.

edit: formating

<comment><USER6> Encryption is basically a tool against wiretappers and men in the middle. But what if you don't fully trust your users, and fear they'll try to add exploits to your software ?

Another high level effort in security is meredith paterson's "language theoretic security"[1] can help you code secure protocols, to fight against this problem.

There's also a tool to help implement it called hammer[2].Not sure if fully developed yet.

[1][http://boingboing.net/2011/12/28/linguistics-turing- complete...](http://boingboing.net/2011/12/28/linguistics-turing- completene.html)

[2][https://github.com/abiggerhammer/hammer](https://github.com/abiggerhammer/hammer)

</comment></comment><comment><USER7> Simply invoking some sort of "encrypt" library is easy, it's everything else that's hard, and you have to get it perfect.

\- Simply encrypting your message as indicated will not protect you from replay attacks. Someone could record your message and re-transmit.

\- Simply encrypting your message will not assure that the contents haven't been modified, someone could patiently sit in the middle poking bits to see what happens.

\- Most encryption schemes will require you to choose a block cypher, doing so requires some knowledge of the options and the data you're sending. Some handle large amounts of data poorly, others fail when you send identical messages.

\- Most encryption schemes will require you to initialize them with truly random data, both an early version of Netscape, and Debian messed something up and provided far less entropy than they appeared. Relying on /dev/urandom on a machine that's just booted, or otherwise faulty entropy providers is fatal.

\- Attackers can record your data and play with it forever, so even if a mistake or attack isn't revealed for years, they can still go back and decrypt your data. I believe the NSA broke the Russian's use of a One Time Pad because they re-used pages years later.

\- Simply encrypting data doesn't provide assurances that you're communicating with the system you think you are, the initial contact is still tricky.

So there's more to it than a single function call.

</comment><comment><USER8> A few reasons.

The overarching problem is that you don't really get any feedback about whether what you're doing is right or wrong. For example, no cryptographer would use RSA like that, but that's not obvious just from studying the wiki article. Or from looking at the function output - it does turn ASCII into gibberish, as advertised, and that's where most developers will call it a day.

The moving parts are also treacherous. You're not _just_ going to encrypt a string - someone is meant to decrypt it. Have you authenticated the ciphertext? Are you exposing a padding oracle? Or timing attacks? Are messages susceptible to replay? In crypto systems, these things are equivalent to locking the front door and leaving the window wide open.

In practice, most insecure crypto constructions aren't due to bugs in the implementation of RSA or AES. They're because of developers choosing inappropriate primitives, gluing them together incorrectly, or inadvertently exposing dangerous side channels.

Fortunately, there are libraries that can help. As mentioned elsewhere, NaCl/Sodium and KeyCzar provide higher-level interfaces that can abstract away many of these issues.

</comment><comment><USER9> To answer the "why is it hard?" question, I tried to collect my own experiences at [http://www.acooke.org/cute/WhyandHowW0.html](http://www.acooke.org/cute/WhyandHowW0.html) \- not sure I did a good job, but the main conclusion was that you underestimate how important experience is in avoiding errors.

To repeat what others have said in answer to your more general question - solutions to "real world" problems include more than a single call to a primitive. So you need to find libraries that provide a higher level API, like parts of NaCL [http://nacl.cr.yp.to/](http://nacl.cr.yp.to/), Google's keyczar [http://www.keyczar.org/](http://www.keyczar.org/), etc.

Even for simply encrypting a string with a password - [https://pypi.python.org/pypi/simple- crypt](https://pypi.python.org/pypi/simple-crypt) which is what I talk about in the first link - I needed three things: key strengthening, the encryption itself, and an HMAC. Making those work well together was harder than I expected (at least 5 bugs harder...)

</comment><comment><USER10> Lots of good answers on this thread. I think the fundamental underlying reason is that programming is difficult and so poorly understood.

A given: all software has bugs. Usually, that doesn't matter — a CRUD app will eventually get debugged enough to the point of usability. (Sometimes even maintainability.) We do not understand enough about programming to guarantee perfect execution in all cases, but no one gains any value by causing an obscure input case to cause a null pointer exception.

Whenever we use crypto, however, we inherently have code which protects something valuable: from forum passwords to credit card numbers to state secrets. This means that all the subtleties which break in ordinary code, but no one cares about, suddenly become important. Every interaction of input to memory to processing to storage (to network) must be scrutinized for places where a crucial piece of data may leak an encryption key, or perhaps just enough known plaintext of known cyphertext to mount an attack.

</comment><comment><USER11> Also, encryption is all about maths, so there are hundreds of ways to do just about anything, different parameters, different algorithms with different tradeoffs about speed, performance, resistance to attacks, data bandwidth, etc. etc. So I don't think a library with the kind of interface you describe would be very useful. But I do think it would be great to have a library that allows us to configure encryption based on requirements instead of technicalities.

</comment><comment><USER12> There are two facts about crypto that often get mixed up in these discussions:

1\. For a high value target like Edward Snowden, there is a broad spectrum of attacks, and any operational weakness is fatal. There are many examples of these attacks described on this thread. Unless you know what Snowden knows, odds are you will not get it right.

2\. BUT, if everyone had easy encrypted email and real time communication, the mass surveillance machine would be blinded, because the kinds of attacks that are used against high value targets do not scale up well.

</comment><comment><USER13> I guess the main problem is that encryption is foreign to most of us (myself included). It's hard to understand what is safe from what isn't.

It's also very hard to figure out if your encryption is bugged or not. I guess that for most us, once your method returns a hash, you expect that everything is secure.

On a side note, I wonder how many people on HN would claim to know the inside out of encryptions. (Not the difference between SHA1/MD5/bcrypt but the actual math behind derivations and how they work)

</comment><comment><USER14> Encryption being painfully and needlessly difficult is one reason why it isn't widespread on both the business end and the consumer end. GPG, which _everybody_ should use for email, has one of the most terrible interfaces conceived. It is absolutely no surprise that people would rather be spied on than spend a week getting that POS working.

There's a massive market for easy-to-use encryption. Easy-to-use does not imply insecure in any way at all.

<comment><USER15> There are two orthogonal aspects, and you touch both of them.

1) All software has bugs. The problem with crypto software is that bugs are far more dangerous, even when they may appear to be insignificant. To use an utterly broken analogy: a faulty seal in a pressure cooker does not cause all the locks and hinges in your house to magically unscrew themselves. But even a slightly broken crypto implementation in software _can_ cause a complete breach. (No pun intended.)

2) Developing good and intuitive UI's is hard. When the UI has to hide the complexity of secure key management, it's even harder. Humans are by nature lazy and inventive; if the UI allows any way to achieve convenience over security, conveniene _will_ be what most of the users choose.

In my mind, there is one particular implementation where adding security allowed more convenience. The humble ssh-agent. When used properly, you don't _need_ to know any of the passwords for remote systems. And unfortunately, the most convenient way to achieve this is, of course, to leave the private key either unencrypted or protected with an empty passphrase...

</comment></comment><comment><USER16> AFAIK there's no "encryption for humans" library (at least no widely known, widely used, widely tested one) - they all rely on the developer to specify the right parameters into the function, with no sanity checking asserts.

The results of this is things like the developer who used "1" as the multiplication factor, so to decrypt the data, you need to divide each block by 1...

</comment><comment><USER17> Sometimes crypto libraries have bugs, but it's also easy to use them incorrectly, especially if you don't have an good understanding of cryptography.

For example, a common mistake is to assume that by encrypting something, attackers can no longer change it. Or perhaps you'll use your standard equality operation to check whether a decrypted string matches some value, without thinking about timing attacks. Or maybe you'll just use AES in ECB mode.

</comment><comment><USER18> Just in your example there's already a problem. If you don't use something like OAEP padding (PKCS1 v1.5 padding has been proven to have issues) then you're vulnerable to attack (see: Bleichenbacher [http://www.bell- labs.com/user/bleichen/papers/pkcs.ps](http://www.bell- labs.com/user/bleichen/papers/pkcs.ps)).

</comment><comment><USER19> For anyone interested, we have encryption examples in Python (PyCrypto & M2Crypto library), Ruby (OpenSSL), PHP (phpseclib), Java (Spongy Castle) and Objective-C (CommonCrypto) here: [https://launchkey.com/docs/api/encryption](https://launchkey.com/docs/api/encryption)

disclosure: I'm a co-founder of LaunchKey

</comment><comment><USER20> I know it's cool to hate Microsoft and .NET here on [Y], but .NET framework actually comes with a ton of encryption classes & methods - [http://msdn.microsoft.com/en- us/library/system.security.cryp...](http://msdn.microsoft.com/en- us/library/system.security.cryptography.aspx)

</comment><comment><USER21> There is a crypto challenge that explains many of the flaws in crypto done not exactly right, by giving real examples / puzzles on how to break the crypto. See [http://www.matasano.com/articles/crypto- challenges/](http://www.matasano.com/articles/crypto-challenges/)

</comment><comment><USER22> This is exactly what the NSA wants you to think! Encryption is only a tiny part of the problem space, and yet still gets broken in fun ways (padding oracles, bad RNGs, etc). The more difficult mart is key management and distribution. This is where crypto rubs up against the human. Humans suck.

</comment><comment><USER23> As a different, less technical response: crypto is so hard because it's natural to assume that cryptanalysts are so persistent.

A good crypto library should keep your data safe for decades. We don't make the same demands (no bugs, due to no updates possible) of other software that often.

</comment><comment><USER24> Encryption is easy, security is hard. Every time you increase the security of a product, you decrease usability.

e.g. easiest to use :: SSH with password <<>> SSH with passphraseless keys <<>> SSH with passphrase-protected keys :: most secure

<comment><USER25> This is not entirely correct.

For your SSH example, pubkey authentication has a bit of a learning to _to set up_ but then makes _ongoing_ use far easier.

For networked file systems, at work we have a common shared drive (H:) and a private shared drive (J:). The private drive is clearly more secure as people who shouldn't have access can't even _see_ the files, but it's also much easier to use as you don't have to sort thru tons of other people's old crap.

</comment></comment><comment><USER26> [https://code.google.com/p/keyczar/](https://code.google.com/p/keyczar/)

From their homepage:

Crypter crypter = new Crypter("/path/to/your/keys");

String ciphertext = crypter.encrypt("Secret message");

</comment><comment><USER27> Encryption is hard because computers are constrained by (but exceptionally good at) discrete maths. All encryption does is slow cryptographic attacks down (a lot).

Also, proper key management is out of reach for most of us.

</comment><comment><USER28> More simply put, and to answer the main (title) question, the reason encryption is so hard is because it's usually economically rewarding to break.

</comment><comment><USER29> To step back a bit from the tech problems -- encryption is hard because not everybody uses it.

I think what we need, for email at least, is a completely new protocol that's end to end secure (as hard as that is). The problem though is that I don't think something like this can be done anymore, without "interested" corporations co-opting or talking it to death. The golden age of the internet is gone.

<comment><USER30> So, reading many of the replies in this thread, they all cover good points, but I have a slightly different point: The current encryption libraries make the "easy" stuff hard. If we assume that all the hard work of actually doing things is taken care of, and just looking at the API of the command line tools (the C api's are generally worse).

The NSS command line tools require you to provide a provide a entropy file of "sufficient size". "Oh!" you think, "I'm on linux, I can just use /dev/random". No, it tries to read the entire file, and thus hangs infinitely after using up all your entropy. "Ok, I'll do something like: dd if=/dev/random of=- bs=1024 count=1 | .... --nonce-file /proc/self/fd/0", nope, doesn't work, because it tries to read from this file multiple times. (Also how big is "sufficient"? 8196 bits of entropy seems overkill worth of entropy I just threw down the toilet, but the opposite case is even worse.)

The openssl command line tool has a simple "ca" command, which appears to be the "State of the art", but doesn't support revoking keys or even concurrent access. The openssl command line in general is confusing and difficult to remember.

PKCS#11 is a nice C api, but there's no standardised config to tell the machine to use PKCS#11 for everything, so you have to specify it on every command line. Forget it just once and you've possibly just created a key thats not in your HSM and you might not notice that.

A little bit of UI TLC would make the world much more secure. SSH sees high levels of uptake because while it has strong crypto, it's relatively easy to use.

Exploits with people using TLS and then not properly verifying the far ends cert is usually down to the fact that library that they're using doesn't provide a "just verify the cert for me" but expects every developer to implement it themselves. Most libraries will default to ridiculously stupid ciphers, even if the other end will let you negotiate stronger ciphers, again requiring every application developer to make sure that they have overridden the defaults to something sane, and perhaps just offloading the problem onto the system administrator.

djb's NaCl has a nice API, but it's incompatible with everything, so you can't use it to interoperate with the rest of the world. (Also djb has a history of abandonware, and licenses that prevent other people from continuing to maintain the software as the realities around it evolve, so it's probably not useful for something that's going to last >3-5 years.).

Crypto is notoriously difficult to get right, but the API of the current tools (both the language bindings and CLI tools) makes it far harder than it needs to be, inviting a slew of additional errors.</comment></comment>


Groupon To Sell 30M Shares At $16-$18 A Pop - <AUTHOR> http://techcrunch.com/2011/10/21/groupon-to-sell-30m-shares-at-16-18-a-pop-valuation-as-much-as-11-4b/ <comment><USER2> Anyone thinking about buying Groupon stock - check LinkedIn's after November when the lock-up agreement expires.

LinkedIn, like many IPOs, had a lock-up agreement. That means that insiders and venture capitalists (or investors who bought the stock pre-IPO on a secondary marketplace like SharesPost) can't sell stock for six months following an IPO. This helps stabilize the stock in its early days. It also leads to possible dips in stock value around the time of expiration. The agreements for LinkedIn will expire in November.

[http://www.investmentu.com/2011/August/linkedin-good- company...](http://www.investmentu.com/2011/August/linkedin-good-company-bad- stock.html)

In Matt Taibbi's book, Griftopia, most of these stocks are little more than pump & dump operations by finance industry, who collect some fees on the side.

 _Here's how it works: Say you're Goldman Sachs and Worthless.com comes to you and asks you to take their company public. You agree on the usual terms: you'll price the stock, determine how many shares should be released, and take the Worthless.com CEO on a "road tour" to meet and schmooze investors, in exchange for a substantial fee (typically 6–7 percent of the amount raised, which added up to enormous sums in the tens if not hundreds of millions).

You then promise your best clients the right to buy big chunks of the IPO at the low offering price—let's say Worthless.com's starting share price is 15—in exchange for a promise to reenter the bidding later, buying the shares on the open market. Now you've got inside knowledge of the IPO's future, knowledge that wasn't disclosed to the day-trader schmucks who only had the prospectus to go by: you know that certain of your clients who bought X amount of shares at 15 are also going to buy Y more shares at 20 or 25, virtually guaranteeing that the price is going to go past 25 and beyond. In this way the bank could artificially jack up the new company's price, which of course was to the bank's benefit—a 6 percent fee of a $500 million or $750 million IPO was serious money._

Taibbi, Matt (2010-11-02). Griftopia: Bubble Machines, Vampire Squids, and the Long Con That Is Breaking America (pp. 213-214). Spiegel & Grau. Kindle Edition.

<comment><USER3> That 6-7% is 6-7% of the offering price, not the day's closing price. So the statement "In this way the bank could artificially jack up the new company's price," is the opposite of the truth: in this way, _the bank lowers its fee in exchange for a first-day price increase_. That's marketing: Goldman wants to be able to sell the next IPO, so they try to pay off their customers. And that means they _can_ promise e.g. Groupon that they'll have enough buyers.

As usually, Taibbi is vaguely correct (investment banking is a middleman between two groups that are individually worse-informed than the bank is). But he either doesn't understand how this actually works, or deliberately obscures the facts in order to make his point slightly stronger.

<comment><USER2> >So the statement "In this way the bank could artificially jack up the new company's price," is the opposite of the truth

You're taking issue with at what point Goldman Sachs takes their 6-7%, while totally ignoring the main point: Goldman Sachs has engaged in laddering which is considered fraud.

</comment></comment></comment><comment><USER4> I'd like to suggest that is should read "Groupon to TRY to sell 30M shares at $18 - $18 a pop" as I really don't think anyone in their right mind should buy them.

Sadly much as I'd like to believe that this IPO will fall on stony ground I suspect that there are enough people out there who'll snap it up that they'll get their money.

Odd how we can be in a global economic crisis and a bubble at the same time. Strange times.

<comment><USER5> It will definitely sell for that much. After 4 days it will reach its all time high and then plummet. Pump and dump!

</comment></comment><comment><USER6> I'm not going to buy it until it goes on sale for 90% off.

</comment><comment><USER7> i would never buy groupon stock. this is a doomed biz model because it's not based on any real technology...the entire company is mostly comprised of only salespeople, which is pretty scary.

<comment><USER8> >this is a doomed biz model because it's not based on any real technology

Neither is Dairy Queen, but everyone knows if they want delicious vanilla soft server where they need to go. Likewise, millions of discount seeking customers know that a place to go to find discounts is Groupon.

Groupon is doomed if they can't get revenue above their operating costs. It has nothing to do with the technology they are based on.

<comment><USER9> No one makes Dairy Queen soft serve but Dairy Queen. Everyone makes 50% time based group coupons.

<comment><USER8> The point was that Dairy Queen isn't based off some spectacular technology.

> Everyone makes 50% time based group coupons.

Everyone? That's silly. Do you mean the coupon market is saturated? I would agree. But Groupon is at the very top. How does that doom them?

<comment><USER7> Dairy Queen has physical locations, serving that specific location and market. It takes time for the company to do research on specific market segment and why there needs to be a physical location there. It also takes time to build, train, and hire employees in order to service that market.

Groupon: user is only sustainable based on type of deals. business model easily replaceable by any upcoming player.

plus there's so many things going inside the company right now...Groupon's own sales team is suing the company? come on, what does that speak to the company itself? the whole company is made up of salespeople!!

<comment><USER7> jemka, my point is that Dairy Queen has more real technology than Groupon. Their icecream is their product. Groupon has none. Groupon has coupons as a product and that's not a technology. Icecream is!!

<comment><USER8> The point you're painfully missing is that success of a company (including Groupon) isn't dictated by how much or little technology they use. If you feel Dairy Queen was a bad metaphor for that, then I apologize for your confusion.

<comment><USER7> Sure, Groupon is successful, but is it sustainable? To me, it's not a good business model because it could be easily copied, easily replaceable, and anyone could do it. Short-term success doesn't translate into a sustainable business model.</comment></comment></comment></comment></comment></comment></comment></comment>


Home in a Can: When Trailers Offered a Compact Version of the American Dream - <AUTHOR> https://www.collectorsweekly.com/articles/home-in-a-can/ <comment><USER3> I built my own trailer, a "tiny home": [http://tinyhousemansion.com/](http://tinyhousemansion.com/)

I filmed a several how-to videos and put them up on youtube. Not the most economical (cost/resale-wise), but a great experiment and real-world break from full-time programing.

My Tiny House is super insulated and energy efficient. The only leaks are from windows. I wrapped the whole thing in rigid foam panels and lifted the metal roof up an inch to keep the roof from ever getting hotter than the ambient air temperature.

<comment><USER4> Just watched the video on your site and this is by far my favorite Tiny House project I've seen yet. It's the first one I've seen that I felt like I could actually live in that with two kids. Great job! Thanks for taking the time to document the experience and share it with all of us!

<comment><USER3> Thanks, I couldn't have done it without others who shared their ideas.

Building energy-efficient, non-toxic homes is important. Many of the ideas I used for this Tiny House I plan on implementing on more traditional housing structures.

Regular homes are supper-sloppy when it comes to conserving energy and there is a lot of room for improvement (with little added cost) if the ideas are implemented during the initial construction.

</comment></comment></comment><comment><USER5> Trailer living is stigmatized because it's what poor people can afford. With poverty, come all it's correlated afflictions: substance abuse, limited employment, low education and family planning, lack of role models, each determining the other and trapping multiple generation into the same cycle. I.e "trash" people, those who just don't seem to get ahead.

Trailer trash is a largely american stereotype because US had a car-centric culture and early economic development that allowed even the poor to buy an RV, while locking them out of the traditional property market. But the stereotyping is against poor people and it's practiced in any society that values material gain.

</comment><comment><USER6> What a $300k trailer looks like in Mountain View:

[https://www.trulia.com/property/5033711929-325-Sylvan- Ave-28...](https://www.trulia.com/property/5033711929-325-Sylvan- Ave-28-Mountain-View-CA-94041)

<comment><USER7> You'll have to be 55+ to buy it though (I think.. it's in a 55+ park).

</comment></comment><comment><USER8> There's an odd moment in Steinbeck's Travels with Charley, his book about a 1960 roadtrip across the US, where he encounters a couple living in a mobile home and sings the praises of it. Modern, mobile, allows the inhabitants the freedom to follow their dreams and the job market. Truly another time.

[https://schlemielintheory.com/2014/06/18/the-restless- ones-a...](https://schlemielintheory.com/2014/06/18/the-restless-ones-and- uneasy-permanence-steinbeck-on-america-mobile-homes-and-rootlessness/)

<comment><USER9> Trailer living used to be a lot less stigmatized than it is now. Even as recently as the late 70's, the officers in "CHiPs" lived in trailers; something you never see heroic television leads do anymore.

<comment><USER10> Also Mel Gibson's character in Lethal Weapon (1987)...

<comment><USER11> Pretty sure Mel Gibson's character lived in a trailer _because_ of the stigma: "man, he used to be a good cop, then his wife got killed. Now he lives in a _trailer_! He's just given up!"

</comment></comment></comment></comment><comment><USER12> When I was young living in South Carolina there were the trashy trailer parks, but there were also nice tidy trailer parks for working class families.

It was probably a bit nicer than this 30 years ago

[https://firstname.lastname@example.com,-82.4379351,3a,68y,28...](https://email@example.com,-82.4379351,3a,68y,283.7h,84.57t/data=!3m6!1e1!3m4!1sfIfcXKJ22LpGy30RcRlt2w!2e0!7i13312!8i6656)

<comment><USER13> I didn't really do the math when I replied before, but if you lived in or near that mobile home park and were a kid ~30 years ago, we almost certainly went to the same schools at around the same time, which is pretty surreal. I don't really run into many folks from my hometown in nerd circles (or anywhere, really).

<comment><USER12> Lakeview, probably in '88 I didnt go there long, we moved around a lot when I was a kid.

<comment><USER13> OK, nope, mine was Berea; same era, though. I knew a girl who lived on Longforest Drive who went to Lakeview, and everyone who lived on Plano Drive went to Berea, so I guess the line was somewhere in between the two. That's wild as hell. Howdy, distant-past-teenage-neighbor.

<comment><USER12> I did go to Berea high school for a short time. 9ths grade I guess. I hung out with the nerds :D

<comment><USER13> I only went to Berea for middle school, but lots of my friends from middle and elementary school went to Berea for high school (as did my older sister...the line moved in the interim). A dude named Mark...whose last name I now cannot remember, is the only nerd I can think of who went to Berea. Most of my friends who went there were ag kids in 4A ansd the horticulture program. We both had C64 BBSes and ran warez and did various illegal shit with phones. And, later, I think we both had Amigas (I know I did, but I think Mark got one, too...but we'd drifted apart a bit by that time).

</comment></comment></comment></comment></comment></comment><comment><USER14> I think a lot of people who would be satisfied with a small, cheap home such as a trailer. I mean, I felt like I had the space of trailer in my first apartment, but it sure as hell wasn't as cheap. It's just a shame they have such a negative connotation (with pretty good reason).

<comment><USER15> They've been rebranded "tiny homes" and now they're all the rage.

Granted, you are expected to build your own if you want it to be cool, but it has all the hallmarks of a traditional trailer.

I'm not knocking tiny homes nor trailers, to be clear. I think they're a great solution for folks wanting to own a home on a budget.

<comment><USER16> I'm sure there are a large range of mobile home building quality but the many that I have ever been in have had thin walls and cheap finishes inside. Maybe it's because of the price point they need to hit for their target market. Most tiny homes I've seen have had high quality materials and finishes. Maybe people convert the cost savings of building themselves to building with higher quality materials (for tiny homes).

It's also worth mentioning that most tiny homes I've looked at were far lower square footage than a typical mobile home.

All this to say tiny homes are not imho rebranded mobile homes. They are a different beast altogether.

<comment><USER17> > Maybe people convert the cost savings of building themselves to building > with higher quality materials (for tiny homes).

Building your own construction project is generally not a cost savings, in the same way that growing your own food is not a cost savings.

<comment><USER18> DIY House building can save a lot of money and a higher quality build at least in the uk

</comment></comment></comment></comment></comment><comment><USER19> Technology needs to be refocused into making homes more affordable to the working class instead of creating more profitable McMansions.

</comment><comment><USER20> These days trailer parks are disappearing, because the land is too valuable. I've thought about renting a small trailer in a trailer park, if it saved enough cash, but buying one risks not having anywhere to park it. Times change.

<comment><USER21> Land isn't too valuable, rather there's very little zoned to allow for trailer parks, which can be very dense and thus a great way to use land that's valuable but not so valuable as to support skyscrapers [https://oldurbanist.blogspot.com/2014/05/mobile-home- impedim...](https://oldurbanist.blogspot.com/2014/05/mobile-home-impediments- and.html)

</comment></comment><comment><USER22> Many of the active outdoors people I know (mostly mountain bikers) are superkeen on converted Sprinter vans. I think as more active boomers retire, there will be a resurgence of small RVs.

<comment><USER23> Winnebago just introduced a new model - the Revel - for people looking to travel/live like this. I'm not sure how Winnebago is getting enough 4x4 Sprinters to build them, as Mercedes apparently wasn't prepared for how many of them Americans are buying.

[https://www.youtube.com/watch?v=oOQzZo00kYw](https://www.youtube.com/watch?v=oOQzZo00kYw)

I don't really care for the floorplan, but the concept is attractive - <USER2> able to travel off the beaten path, irregardless of the weather conditions.

</comment></comment><comment><USER24> Trailer, RV, and mobile home living is experiencing a huge resurgence, largely because of the 10k baby boomers _a day_ entering retirement.

Unfortunately, while many of those boomers are just wanting to check out the experience, many have found that their "retirement plans" aren't able to include a regular home. This is partly the subject of this book:

Nomadland: Surviving America in the 21st Century

[http://a.co/htj0AfR](http://a.co/htj0AfR)

<comment><USER20> I took one for the team, and clicked. It goes here:

[https://www.amazon.com/dp/039324931X/ref=cm_sw_r_cp_ep_dp_t7...](https://www.amazon.com/dp/039324931X/ref=cm_sw_r_cp_ep_dp_t7NrAbC0JXDVZ)

Interesting that the shortened link is http, while the destination is https. Is that usual?

Ah, and I just noticed that a.co goes directly to amazon.com, so I suppose it's their own shortner. Of course I didn't know that before I clicked.

<comment><USER25> it's shortner that adds the poster's affiliate link.

<comment><USER11> Or a shortener that _hides_ the referral link, depending on one's perspective.

</comment></comment></comment></comment><comment><USER26> Can't have a conversation about trailers without mentioning James Carville's famous double-wammy.

"Drag a hundred-dollar bill through a trailer park, you never know what you'll find."

With that one quote, he smeared one of the victims of Bill Clinton's predatory sexual attacks AND insulted low-income Americans.

[https://en.wikiquote.org/wiki/James_Carville](https://en.wikiquote.org/wiki/James_Carville)

</comment><comment><USER27> [http://kipcaravans.nl/en/model/kip- shelter/](http://kipcaravans.nl/en/model/kip-shelter/) This has been a staple of Dutch stereotypes for decades.

</comment><comment><USER13> I think about RV/trailer parks a lot, as I live in an RV and travel a lot. It's a really fascinating part of the American experience, and it is pretty uniquely American, as well. There are, of course RVs and trailers and mobile homes in other places, but America's reverance for the "freedom of the road" or whatever gives the whole community around RVs and trailers a different vibe (even compared to Mexico, where a _lot_ of the RV parks are owned by US ex- pats, and Canada).

The part I find maybe most interesting, and most troubling, is the way classism plays out even among the set of people that live in RVs. I've noticed a distinct set of phrases rich/white people who travel in RVs use in reviews to describe parks that are inhabited by poor people or people of color, which is a subtle bit of red-lining among RVers. And, this reinforces the division; parks that want good reviews from the wealthy travelers take steps to prevent poor people from staying in their parks. Imposing a trailer age restriction is the most common and most effective; poor folks can't afford a new RV. It's maybe even _more_ pronounced among RVers because the rich ones don't want to be associated with "trailer trash", even though they have a lot in common with them.

But, I think the explosion of housing costs have led to an interesting shakeup in the market. RV manufacturers that historically targeted the high end, where retirees would sell their house (or use the equity in their house) to buy a monster luxury motorhome, a few years ago started making lower end trailers again. Winnebago even makes trailers again, which they didn't do for a couple of decades.

There's also a strong bit of nostalgia in the market that's somewhat new. Manufacturers are looking backward for design ideas; partly to make low-cost options again, and partly because the industry got lost in the woods for a long time just making the biggest and most expensive thing they could because that's where the money was. I'm pleased to see the change because old RVs were often beautiful or charming in ways that modern fiberglass rectangle boxes distinctly are not.

One thing I've also noticed: RV park prices in major cities have gone through the roof, while availability has gone way down, in the ~9 years I've been traveling. I used to be able to arrive in a city and get a spot in a park within a reasonable distance, without a reservation. No longer is that true. Austin is maybe the most pronounced difference. It was always a favorite landing place for me because it's a major city with a couple of RV parks in easy biking distance to downtown. Those parks have doubled their rent and have months-long waiting lists now. It's not as pleasant to travel in an RV as it once was, even just a few years ago, if you like visiting bigger cities. And, I think that's because so many people have been pushed out of houses/apartments and into RVs. That's an alarming trend, I think.

<comment><USER24> Trailer age restrictions, I haven't heard of that.

We bought a pretty old (1986), small (15') travel trailer that we've been working on in order to explore some bits of California and hopefully other parts of the US we haven't seen. And yes, there is a _huge_ bit of nostalgia in the market: We were shocked at the prices of 'classic', partially-or-fully restored trailers like Shastas and Aristocrats. You can buy used "fiberglass rectangles" with more amenities and generally better construction for far less (but they are, as you say, not in any way charming).

Check out the 5-year history of the stock of Winnebago Industries (NYSE: WGO). I think that's a lot of baby boomers hitting the market.

<comment><USER13> "Trailer age restrictions, I haven't heard of that."

You will. California is _awful_. Particularly the bay area. I ended up having to park over an hour and a half away from San Francisco when I visited. I'm in an old Avion travel trailer these days. When I first moved into an RV it was a newish motorhome, and I never ran into the trailer age restriction. But, I think there's actually two things at play: Parks that never had such restrictions in the past have added them in recent years. All the parks I used to stay in in Austin had no such restrictions in the past, and all but one does now. This is, I think, a reflection of the growing class divide among RVers and the growing desire among wealthy RVers, who do it by choice, to not be associated with or share space with poor trailer residents who do it out of necessity.

It is most pronounced in major cities. It's rare to find this restriction at parks that are out in the country. Also, obviously, city, county, state, and national parks have no such restrictions. So, it's usually possible to find a good place to park it, but it's surprisingly difficult to get a spot somewhere close to a major city.

It's been such a frustrating problem for me that come spring, I plan to buy a newer rig just so I don't have this problem anymore. I _like_ to be able to visit major cities with my RV. It pisses me off that I'll have to give my business to parks that participate in ugly classism in order to do it, though. I've also considered going with a big old bus and just putting a gazillion solar panels on the roof and huge water tanks, so I don't need to be in a park at all.

</comment></comment></comment><comment><USER28> 800 through 1000 square foot Aladdin or Sears Roebuck-style kit homes would do far more to affordably beautify the typical America neighbourhood than 1960s style trailer homes replete with turn signals up top and flattened tires chocked below. A visit to the outskirts of the average mid-sized American city will soon cure anyone of nostalgia for trailer-laden communities.

IMHO, homes should be smaller and designed by architects rather than efficiency experts and with an eye toward past stylistic precedents (gothic, craftsman, prairie, stick, etc.)</comment>


Dick Smith receiver puts customer databases up for sale - <AUTHOR> http://www.itnews.com.au/news/dick-smith-receiver-puts-customer-databases-up-for-sale-416422 <comment><AUTHOR> Dick Smith Electronics receiver Ferrier Hodgson is trying to sell the retailer's customer databases in an effort to recoup revenue from the failed business. What do you think? Should everything including privacy be sold off?</comment>


Ask HN (Azure users): SQL Database or SQL VM - <AUTHOR>

Hi guys, we are considering using Azure to host a medium-sized (in terms of traffic and DB/IO read/write) Web app, and wondering if anyone can share their opinion and experience.
Azure offers two main options for SQL Server databases: SQL Database (which seems like a managed high-availability database on a cluster) and SQL VM (which seems like an unmanaged/self-managed instance on a VM).
In terms of cost and basic needs, SQL Database should be sufficient for us for now, but I'm not convinced about it's performance (plenty of posts about slowness, even for simple queries & small datasets). SQL VM will obviously be faster and more efficient, but the relatively higher cost is a prohibition.
What do you guys suggest? How has your experience been with either (or both) SQL Database and/or SQL VM? <comment><USER2> Consider you also trade high availability away when you build on a single vm.. I can't say I have used either on Azure, but the hosted db service seems more attractive on a budget.</comment>


Tus - an Open Source File Upload Protocol - pow-tac http://www.tus.io/ <comment><USER2> This is a pretty cool idea. I absolutely detest how much apps don't properly handle network degradation correctly, so anything resumable is great in my opinion.

I have a question though... regarding the resuming you do the HEAD request to see what has been uploaded:

> HTTP/1.1 200 Ok

> Content-Length: 100

> Content-Type: image/jpg

> Content-Disposition: attachment; filename="cat.jpg"'

> Range: bytes=0-69

Is it possible that the data that is already there could be corrupt?

I'm also wondering how things like proxies deal with this. A lot of mobile networks have nasty transparent caching proxies in their network. Also when uploading a file through Nginx (when the upload works correctly) it won't send anything to the backend until it has the complete data, is this the same if the connection cut half way through?

<comment><USER3> > Is it possible that the data that is already there could be corrupt?

TCP error correction should take care of this for most parts, but we're actively discussing adding support for stronger guarantees: [https://github.com/tus/tus-resumable-upload- protocol/issues/...](https://github.com/tus/tus-resumable-upload- protocol/issues/7)

> I'm also wondering how things like proxies deal with this. A lot of mobile > networks have nasty transparent caching proxies in their network.

That's a good question. A http proxy could always cause issues, but most proxies should leave POST/PUT/HEAD requests untouched. That being said, we won't freeze the protocol until we had a chance to try it against a variety of mobile networks, which is why we're already starting to implement an initial iOS client over here: <https://github.com/tus/tus-ios-client> (not ready yet, but keep an eye on it).

> Also when uploading a file through Nginx (when the upload works correctly) > it won't send anything to the backend until it has the complete data, is > this the same if the connection cut half way through?

NGINX is somewhat unsuitable as a proxy for file uploads due to the buffering you mentioned. Ideally people will implement the tus protocol as an NGINX Addon like this one: <http://www.grid.net.ru/nginx/resumable_uploads.en.html>

Meanwhile clients are free to choose a small chunk size for individual PUT requests (e.g. 1 MB), which will allow them to still have resumability (in 1 MB intervals) without changing their architecture.

Last but not least, we'll implement the tus protocol for our commercial uploading service, transloadit.com.

So I'm reasonably optimistic that NGINX won't be a major hurdle for the adoption of the protocol.

<comment><USER4> > TCP error correction should take care of this for most parts ...

Oh no it doesn't! We have an analytics service receiving HTTP posts from browsers all over the world as JSON. There is an astonishing amount of single bit errors going on. Usually the initial 20 bytes are okay, but after that we see all sorts of patterns including a bit flip every 8 bytes or so. Note that these will have been received at Google's appengine servers with the correct checksum. I believe that much of the cause is intermediary devices (eg performing NAT or routers) that are responsible for the corruption and recalculate the checksum putting a good checksum on what is now corrupted data.

For that service we have to use HTTP (grumble grumble IE grumble). For our regular stuff we use HTTPS where we do still see the problem but it is considerably rarer. In that case the cause is most likely the client device having problems (eg RAM bit flips, cosmic rays, overclocked/overheated CPUs etc)

All else being equal I'd recommend you add a layer of checksums as a helpful sanity check. Using SSL also does that for you, but it sees the data late.

</comment></comment></comment><comment><USER5> I've been a user of their product <https://transloadit.com/> for about a year now for video encoding and am very happy with it. The API is elegant, the whole service is reliable and fast.

These guys know what they're doing.

<comment><USER3> Thanks for the kind words! Tus.io the beginning of a larger commitment to open source here at Transloadit, which will also translate into many new features (resumable uploads being the next one).

<comment><USER6> Felix, given that the rest of your stack is on Node, what made you choose Go?

<comment><USER3> Node.js was a good choice for Transloadit when we started (2009). By good I mean better than PHP which we tried using for transloadit before.

I started playing with Go late last year, so far I'm under the impression that it's easier to write reliable software with it than node.js. Callbacks and exception handling are a huge PITA in node.js, and the community has chosen to refuse improvements that would help with some of the issues (promises).

Go is also an incredible joy to work with given the modern nature of the standard library, static typing, gofmt, built-in testing, and many other small things that the Go team has done right.

That being said, tus.io is not a Go project. Our first server implementation (tusd) is written in Go, but we're working on support for other plattforms like node.js as well.

Generally speaking node.js will continue to be part of our toolkit at Transloadit (for the quick & dirty), but I suspect that we'll use Go for the more criticial parts we work on going forward.

</comment></comment></comment></comment><comment><USER7> Shameless plug, but HTML5 + CORS + S3 can enable resumable file uploads. I've written a library that uploads to S3, and can resume uploads (think internet going down for a while, force-closing the tab, etc.): <https://github.com/cinely/mule-uploader> . There's a demo available, I suggest you test it with bigger files (>100MB)

<comment><USER3> Your library looks great - thanks for releasing it!

S3 is an incredible offering, but since I'm working on tus.io, I'll focus on what's wrong with it : )

\- Multipart chunks need to be 5 MB at least. An interrupted part cannot be resumed. This kills the mobile use case.

\- Throughput to S3 is bad from outside of EC2, uploads often start at very slow speeds and won't reach the capacity of the connection in many cases.

\- S3 does not let you stream/access an upload in progress easily, so you can't start to transcode a video while it's still uploading.

\- The S3 API is the opposite of RESTful.

\- S3 is a proprietary service, their protocols are not intendent/documented for adoption, and IMO they don't deserve great people like you making free contributions to their ecosystem.

edit: I'm not trying to say S3 isn't a good choice for many people. But our goal is to bring resumable file uploads to every iOS, jQuery, Wordpress, Drupal, Rails, etc. application in the world - S3 is not the right starting point for that.

<comment><USER7> Thanks for your insight, you make good points; it seems to me you don't really like S3, can you elaborate why?

<comment><USER3> I love S3, and we use it all over the place at transloadit.

I realized my comment sounded overly negative, so I added a clarification to my comment: Our goal is to bring resumable file uploads to the entire planet, S3 or any other proprietary protocol should not be the base for that.

<comment><USER8> What's your opinion on Swift (<http://www.openstack.org/software/openstack- storage/>)? It's basically an open standard for "an object store with compatibility to, and the same guarantees of, S3." Used in, for example, Rackspace Cloud Files.

</comment></comment></comment></comment></comment><comment><USER9> I think some of your responses aren't quite right. For example, in response to the first PUT, you have:

      HTTP/1.1 200 Ok
      Range: bytes=0-99
      Content-Length: 0

But the Range header surely can't be used here, since it's a request header and this is a response. A Content-Range header wouldn't be any more appropriate, since you're not actually returning any content (of any amount). Do you really need this info in the response anyway? The sender knows what they sent, and either it was entirely successful (a 2xx response) or it wasn't.

Also, if you're going to return a zero-length 200 response, you might as well use 204 No Content instead.

Then, when resuming an upload, you send a HEAD that returns the following:

      HTTP/1.1 200 Ok
      Content-Length: 100
      Content-Type: image/jpg
      Content-Disposition: attachment; filename="cat.jpg"'
      Range: bytes=0-69

Again, you can't use the Range request header in a response. And the Content- Length should surely be 70, since that's how much content would be returned if this was a GET request. You could possibly include a Content-Range of 0-69/100 if the server wanted to communicate the expected file size, but I'm not convinced that's necessary and seems something of an abuse of that header.

Finally, the response to the resumed PUT has the same problems as the first PUT response. It should probably just be a 204 No Content response - <AUTHOR> Content-Length or Range headers required.

<comment><USER3> > But the Range header surely can't be used here, since it's a request header > and this is a response.

We're currently discussing how to interpret RFC 2616 (http 1.1) for this here: [https://github.com/tus/tus-resumable-upload- protocol/issues/...](https://github.com/tus/tus-resumable-upload- protocol/issues/2)

If you have a better suggestions than using the Range header that will still allow clients to send multiple file chunks in parallel, I'd be very interested in it!

> Do you really need this info in the response anyway? The sender knows what > they sent, and either it was entirely successful (a 2xx response) or it > wasn't.

We don't need it for the PUT request, but we do need it for HEAD. Adding it to PUT is redundant, but simplifies the logic for clients who choose to upload multiple chunks in parallel.

> Also, if you're going to return a zero-length 200 response, you might as > well use 204 No Content instead.

Good point, I'll investigate on that: [https://github.com/tus/tus-resumable- upload-protocol/issues/...](https://github.com/tus/tus-resumable-upload- protocol/issues/12)

> And the Content-Length should surely be 70, since that's how much content > would be returned if this was a GET request.

It's 100. We haven't specified GET requests yet, but a server could stream an upload in this case until all bytes have been received.

Anyway, this is awesome feedback - thank you so much!

<comment><USER9> > We're currently discussing how to interpret RFC 2616 (http 1.1) for this > here: [https://github.com/tus/tus-resumable-upload- > protocol/issues/...](https://github.com/tus/tus-resumable-upload- > protocol/issues/2)

I've followed up with further comments there.

> If you have a better suggestions than using the Range header that will still > allow clients to send multiple file chunks in parallel, I'd be very > interested in it!

I don't see a way to support parallel transfers using only existing HTTP headers (without violating the HTTP spec). I would suggest maybe proposing a new header in the HTTPbis WG. For example, something like Available-Ranges that returns a ranges-specifier indicating the set of ranges that are avaiable.

This could possibly be returned as part of a 416 response when attempting to GET a file that isn't entirely available yet. A HEAD request would thus return the same thing.

> It's 100. We haven't specified GET requests yet, but a server could stream > an upload in this case until all bytes have been received.

The reason I brought up a GET request is because "the metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request." (section 9.4 of RFC2616)

If you haven't got all 100 bytes yet, your GET request can't return a Content- Length of 100, thus your HEAD request shouldn't be returning 100 either.

I would have thought you would return whatever content you had available (hence the 70 bytes), but if you want to support parallel transfers, then a 416 error response indicating the available ranges might make more sense.

</comment></comment></comment><comment><USER10> Call me a prude, but I saw the F-word and hit the back button.

<comment><USER3> Point taken. We'll try to hit a better balance between emotionalizing the subject without being too exclusive when iterating the text for this again.

<https://github.com/tus/tus.io/issues/14>

<comment><USER11> I don't think it's an issue with "emotionalizing the subject", nor one of people getting offended, but rather one of professionalism.

That sort of language, in a context like that, reminds me of a Zed-style rant. It makes it harder for me to take it seriously, you know? The whole project ends up coming off as an amateur effort, even if that may not be the case.

<comment><USER3> I don't consider professionalism and the word "fuck" to be mutually exclusive, but at the end of the day we'll focus on what attracts people. Our current choice of words clearly fails at this goal, so we'll consider replacing it.

<comment><USER12> It clearly fails but you'll only consider changing it? That's more worrying than choosing the word in the first place...

</comment></comment></comment></comment></comment><comment><USER13> Seems fine as a best practice for using HTTP for file uploads. I feel the requirement for the server to have fixed URLs for uploading to be limiting, but then again, I'm one of those HATEOAS freaks.

<comment><USER3> The urls don't have to be fixed. I'll clarify this in the docs: [https://github.com/tus/tus-resumable-upload- protocol/issues/...](https://github.com/tus/tus-resumable-upload- protocol/issues/11)

Also: The protocol is still under heavy development, so please post any additional ideas, issues, patches or feedback you may have!

</comment></comment><comment><USER14> I'm confused on what information the HEAD request gives after a chunk has failed. Suppose a client concurrently uploads chunks 1, 2, 3, 4 and 5; chunks 2 and 4 fail, and the rest work. What information does the HEAD give to tell the client that it needs to re-send the data that was in chunks 2 and 4? Wouldn't it make more sense for the client to store the success of each of its chunk uploads?

I'm also not seeing how the client indicates that the upload is complete. It could be done server-side, by just detecting when a file has no more holes in it, but that seems hacky. Holes can also be useful; suppose I make a 32GB .vmdk file (non-sparse) and put 2GB of data on it. If the server can support holes, then I can upload (and the server only has to store) about 2GB of data; if the server can't support holes, then I'll have to upload a bit more data (assuming compression), and the server will have to store a lot more data. If there were some final message the client could submit to the resource saying "I'm done, commit it!", I think the protocol would be a bit more complete.

<comment><USER9> Lets say the whole file is 1000 bytes and each of the five chunks is 200 bytes. If 2 and 4 have failed, then the HEAD would return with a Range header like this:

      Range: bytes=0-199,400-599,800-999

The client would then know it had to resend bytes 200-399 and 600-799 (namely parts 2 and 4). If the chunks only partially failed (say 100 bytes of each chunk was received), the HEAD might even return something like this:

      Range: bytes=0-299,400-699,800-999

So now the client knows it only has to resend bytes 300-399 and 700-799 (only the last 100 bytes of chunks 2 and 4).

Technically the Range header isn't valid in an HTTP response (something they are aware of), but conceptually I think the idea works fairly well.

</comment></comment><comment><USER15> As usual, relevant xkcd:

<http://xkcd.com/927/>

<comment><USER3> : ) - hopefully that's not the case here, afaik our protocol is the first proposal in this space meant for general adoption. All prior art is tied to specific libraries and services.

<comment><USER15> I hope so too =-)

It looked nice from my quick skim, it just brought that classic xkcd to mind!

If this can be put into any page/service, it'd be a huge contribution.

</comment></comment></comment><comment><USER16> It seems like the handling of concurrent access has been neglected in this protocol. What if multiple clients try to resume uploading the same file?

<comment><USER17> Some of hte ideas from <http://www.w3.org/1999/04/Editing/> might come in handy there. What realistic usage scenarios are there for multiple clients concurrently uploading (parts of) the same file?</comment></comment>


Add a status LED to your Raspberry Pi - <AUTHOR> https://levelup.gitconnected.com/add-a-status-led-to-your-raspberry-pi-d3718846d66b <comment><USER2> Assuming that /sys/class/leds/led{0,1} are already used for something ;)

<comment><AUTHOR> Aw those tiny little LEDs? Can you even see them from 10 ft away?

<comment><USER2> They're surprisingly bright. Been using them for status for ages...but yeah, they're single-color, and I don't think you can actually control the brightness. For anything beyond "blink this when X", other solutions are needed. You can never have too many blinkenlights :)</comment></comment></comment>


Ask HN: A Git-like diff server for web assets - <AUTHOR> This idea popped into my head and I wanted to get the HN community's opinion:
Would serving diffs of static assets on the web improve loading speed and reduce server load?
Instead of serving an entirely new file to be downloaded every time changes are made, compile diffs for the last 10 or so releases and then serve them based on a visitors' current version, supplied as a release code in request url. Then, make the changes and cache the new file(s).
Does this exist? If not, why? <comment><USER2> It's not quite what you described but static asset fingerprinting or asset digest fingerprinting is a good place to start.

In theory, if you're using this behind a cache or CDN, then you've already heavily optimized your server load for serving static assets.

I haven't seen any sort of a diff-based approach to caching in the browser. I don't think there is a notion of cache "layers" for an arbitrary binary file in a browser to easily support this, but that doesn't mean you couldn't build it. Maybe you could use localStorage / sessionStorage here?

Another research term that might be useful is incremental caching.

This seems relatively new but cache digests could also be useful here.

[https://calendar.perfplanet.com/2016/cache-digests- http2-ser...](https://calendar.perfplanet.com/2016/cache-digests-http2-server- push/)</comment>


Freemium business model pricing - <AUTHOR>

When you are making up a freemium business model pricing plan, what is a good way to come up with the difference in prices between each tier?  It obviously has to include features that entice a buyer to pay more for the product, but what level of price increase is too much? <comment><USER2> Pricing is something that you're never going to get perfect right off the bat. Start high, then figure things out with your customers. Remember that it's easier to lower prices than it is to raise them.

I will say that it's quite common to have a super-high price tier that hardly anyone buys, that is there to make the other tiers look cheaper by comparison. Not saying that it's a tactic that works 100% of the time, but anecdotally speaking, I see it around on SaaS websites.

e.g.

Basic - $30

Premium - $50 _most popular!_

Corporate - $200 (comes with super-duper feature that 99% of people don't need)

<comment><AUTHOR> I agree with everything you say. It is the usual best practices with this type of pricing model.

Let's just compare these two services and the somewhat different take they have between the tiers of pricing.

<http://www.chompstack.com/pricing/>

-only differentiation is if you need 11 restaurant. The upsell is only $15.

<http://letsfreckle.com/plans/>

-Freckle obviously plays better with the "idiot package" by upselling 10 more accounts instead of 1 restaurant like chompstack

They both had different pricing strategies I see when sorting through the tiers: -Chompstack just went up $15 each tier -Freckle doubled each time instead of sticking with a fixed rate

Any feedback on those thoughts?

<comment><USER3> Yep, hi, I'm the chief everything of Freckle. ;) (Except chief code monkey.)

So, pricing wise, we had a particular strategy and we add very particular, measurable value. The value for a team is honestly more than double the value for an individual because value increases directly with the number of people who track time.

I've actually hired a pricing consultant to talk about whether our pricing strategy (the chunking of user seats, the size of price increases) is the best for us.

Til that process is wrapped up, it's a bit hard to weigh in!

I will say that we priced our base plan higher than our "competitors", based on the difference between Basecamp and Basecamp With Time Tracking, although it looks now like our "competitors" have increased their prices as well.

(I say "competitors" because we really serve different markets.)

Also: we have as many Solo account members as all other paid accounts combined.

We're not hurting for money either ;) and growing about 20% month over month since I've started serious promotion strategies.

Hope this helps!

<comment><USER4> I noticed that you guys have a free plan. Is that a significant percentage of your user base?

It's a little bit hidden and I didn't see it at first (which I guess is the point).

<comment><USER3> You're asking the wrong questions :) Is it a significant portion of accounts? Yes. There are thousands of free accounts and about 600 paying. But do people who actually use their free accounts stay free? No, at an overwhelming rate, they upgrade. Something like 30-50% of all free accounts with more than 5 time entries upgrade.

Most free accounts are drive-bys or people who are merely curious. I'm also looking into ways to capture some of them, convince them to get over the hump to actually trying it, but I don't worry about how many accounts are free vs paid any more.

<comment><AUTHOR> ahoyhere, thanks for the insight. I am working on pricing for our startup right now. We serve different markets, but I found your pricing structure interesting, that is why I used it as an example. Is there something to be said for offering a very cheap package instead of free (similar to the Chompstack pricing)? Have you guys tried that at Freckle?

<comment><USER3> Hi Matt, you're very welcome :)

Here's how I see the difference between us and Chompstack: there are no for- fun restaurants.

We have a free plan because...

1\. some people are just drive-by gawkers, who will rarely sign up at all, and maybe one day if they actually need time tracking they'll come back.

2\. some people refuse to give CC details until they've checked it out

3\. because of the nature of the app/data, it costs us nothing in terms of resources to have a lot of free users - it's not like social network data or file storage - and we do not offer support to free users

4\. for people who use time tracking for their business, the premium features (invoicing, many projects, multiple people) are huge benefits

That was the free part... for the "very cheap package" idea, we're working on a paid version of Freckle that has some premium features (multiple projects, etc) but not the ones to do with earning money (invoicing), that'll offer to non-hourly-billing productivity nuts for a lower price.

That's about as far as we'll go, though, I think :)

<comment><AUTHOR> Ahoyhere,

I have learned more about pricing stuff from you than I hoped! Thanks for the valuable insight. I have one more question that does not apply to you and Freckle directly now, but I was wondering if you could share some thoughts on it anyway. Before launching a product, how important is it to have a marketing page visible on our website?

<comment><USER3> Definitely important. You want to start reaching out to your audience well before you launch, and collect their email addresses. That was part of how we got about $1000 of paying business right out of the gate. We're doing it again:

<http://charmde.sk>

I'm Amy, by the way. And I'm starting to write more about my biz/approach to biz @ <http://unicornfree.com> if you're interested :)

<comment><AUTHOR> Love the design of both Freckle and Charde.sk. Thanks for the help. As a young and first time entrepreneur, I like to think I know the answers, but most of the time it is not the case. Finding out what is needed from a strategic standpoint is something that I have found others are willing to share, especially bootstrapped, product based startups. I will definitely be checking out your blog at unicornfree.com! My name is Matt :)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> I recommend reading Don't Just Roll The Dice, by Neil Davidson. <http://scr.bi/c9n8Gu> (via Scribd) Great content, and gives insight to the understanding of customer perceptions in pricing decisions. I found it useful for non-software products as well. Also, can you share what type of business the freemium model would be used for?

</comment><comment><USER6> One thing to consider. Some people will be relatively price-insensitive and will buy the most expensive option regardless just to ensure they get all the features. So, make your top price point higher than you might initially think.</comment>


The iPad is [priced like] a game console - <AUTHOR> http://semiaccurate.com/2011/02/22/ipad-game-console/

<comment><USER2> The BoM for the iPad is around $259. ([http://www.electronicsadvocate.com/2010/04/08/ipad-bom- is-25...](http://www.electronicsadvocate.com/2010/04/08/ipad-bom- is-25960-says-isuppli/)) Apple buys a majority of all touch panels and flash memory manufactured as well as highly optimizing the design and integration of the hardware. The game console model would mean Apple loses money on the hardware, hoping to make it up in sw sales/fees. Obviously this is NOT the case. Your comment is awaiting moderation.</comment>


AWS Security Book - <AUTHOR> https://www.manning.com/books/aws-security <comment><USER2> A book on AWS security penned by a current GCP engineer is certainly a bold decision! Looking forward to reading it either way…</comment>


Indie bookstores sue Amazon, publishers for using DRM to create a monopoly - <AUTHOR> http://paidcontent.org/2013/02/20/indie-bookstores-sue-amazon-big-6-publishers-for-using-drm-to-create-monopoly-on-ebooks/

<comment><USER2> This seems a lawsuit hoping for a payoff without having much merit. If you can purchase the item in several different ways, either on other devices or print, then how is it a monopoly? Just because a company enjoys a large market share doesn't necessarily make it a monopoly.

Also, publishers have the ability to specify no DRM on their ebooks for the Kindle. So the fact the DRM exists at all is for the benefit of the publisher, not Amazon. They might as well sue the publishers for exercising their ability to use DRM on the Kindle that creates a monopoly for Amazon.

The open source DRM thing is just for PR and is rather silly. They brought it up in an attempt to make themselves appear reasonable. If such a thing existed then you might as well not use it. If it's open source then it would be trivial to crack it, so what would be the point?

This lawsuit should be tossed, but alas, how much money shall be wasted fighting it that benefits no one but the lawyers for both sides?

Another thing, if the claim is that Amazon has a monopoly on publishing ebooks, does that mean the bookstores are saying that Amazon's monopoly is preventing them from publishing ebooks on the Kindle?

<comment><USER3> "If it's open source then it would be trivial to crack it, so what would be the point?"

You are subscribing to the security-through-obscurity myth and I claim my five dollars.

<comment><USER4> Unfortunately, I don't think you'll get your payout in this case.

DRM is inherently insecure. All DRM is actually employing obscuring tricks to make it as difficult as possible to access the plaintext content. At _some point_ though, you have the plaintext sitting somewhere in memory or in registers on the attacker's machine.

Since DRM _is_ security-through-obscurity, an open source module would make it significantly easier to break.

<comment><USER5> With ebooks it's crazy easy because you can READ the plaintext unlike with video etc. where it take a tad more work to read the plaintext.

</comment></comment></comment></comment><comment><USER6>
      > the filing takes issue with Amazon's proprietary DRM,
      > AZW: "Ebooks with the AZW DRM can only be read on a
      > Kindle device or on another device enabled with a Kindle
      > application…the Kindle app works solely with ebooks sold
      > by Amazon."

Being able to read books from Amazon only on a Kindle or via the Kindle app isn't a big deal in practice, because there are Kindle apps available for every major platform and on the web.

The claim that the Kindle app only works with Amazon books is flat-out false. I know for a fact that it supports DRM-free Mobipocket files, and I assume that the Amazon document conversion service works for the app just as it does for Kindle devices.

<comment><USER5> It's a HUGE deal if you want to sell books. Yes the Kindle reads DRM free books, but 80% of the books being sold in terms of dollar volume MUST be sold with DRM. Thus anyone with a Kindle can't buy and read the majority of books being sold if they buy those books from a 3rd party.

I know you can buy good books without DRM, but if you want to read Stephen King, Nora Roberts, John Grisham they are ONLY sold with DRM. Selling those ebooks when they can't work on the eBook reader system that has sold more units than every other system becomes a problem.

Note, I happen to have a solution for getting ebooks with DRM onto a kindle in a readable form that I'm working on.

<comment><USER6> If the ebook reader market is dominated by one manufacturer that supports both DRM'd and open formats, and some author refuses to sell books in an open format while complaining about DRM, then that's not the manufacturer's fault. It's not reasonable to expect Amazon to support every hair-brained DRM scheme that a publisher can come up with.

</comment></comment></comment><comment><USER7> "Open source DRM"? No, they should push for removing DRM entirely.

<comment><USER8> There are many good reasons we all know that DRM is evil, but here's another:

I wanted to buy a book recently, but for various reasons I don't want to have that purchase permanently associated with me. The only ebook options were DRM'd, so that was impossible. They all necessarily bind the book to your account.

My only option for maintaining some semblance of privacy is to buy a hard copy from an independent retailer.

</comment></comment><comment><USER9> I can understand why they're frustrated by the big 6, but why Amazon?

Amazon _does not_ mandate DRM on ebooks. All O'Reilly ebooks are DRM-free on Amazon, for example...

</comment><comment><USER10> Fuck DRM, get DRM free ebooks at ebookoid.com

<comment><USER2> You can buy DRM free books on Amazon as well, it's the publisher's choice. I don't know the percentages, there doesn't seem to be a huge number of them but they are there.</comment></comment>