I ended up paying $150 for a single 60GB download from Amazon Glacier - <AUTHOR> https://medium.com/@karppinen/how-i-ended-up-paying-150-for-a-single-60gb-download-from-amazon-glacier-6cb77b288c3e#.w582hjycu <comment><USER2> I'm the first one to admit that Glacier pricing is neither clear nor competetive regarding retreival fees. I do think that a lot of people use it the wrong way: as a cheap backup. I use:

1\. My Time Machine backup (primary backup)

2\. BackBlaze (secondary, offsite backup)

3\. Amazon Glacier (tertiary, Amazon Ireland region)

I only store stuff that I can't afford to miss on Glacier: photos, family videos and some important documents. Glacier isn't my backup, it's the backup of my backup of my backup: it's my end-of-the-world-scenario backup. When my physical harddrive fails AND my backblaze account is compromised for some reason, only then will I need to retrieve files from Glacier. I chose the Ireland region so my most important files aren't even on the same physical contintent.

When things get so dire that I need to retrieve stuff from Glacier, I'd be happy to pony up 150 dollars. For the rest of it, the 90 cents a month fee is just a cheap insurance.

<comment><USER3> Similarly, I have a NAS at my house pull stuff from all the other computers and back it up to its drives (snapshotted with ZFS to avoid deleting files and realizing it days later), and borg to upload to rsync.net. I paid $54 per year for 150 GB, which is pretty cheap.

<comment><USER4> fyi, there's also Amazon Cloud Drive which includes unlimited storage for $59.99/year.

<comment><USER3> That's a great price, but I don't think it works with borg, which is just fantastic. Hands down, the best of the programs I tried (it's a fork of attic: [http://www.stavros.io/posts/holy-grail- backups/](http://www.stavros.io/posts/holy-grail-backups/))

</comment></comment></comment></comment><comment><USER5> Glacier pricing has to be the most convoluted AWS pricing structure and can really screw you.

Google Nearline is a much better option IMO. Seconds of retrieval time and still the same low price, and much easier to calculate your costs when looking into large downloads.

[https://cloud.google.com/storage/docs/nearline?hl=en](https://cloud.google.com/storage/docs/nearline?hl=en)

<comment><USER6> There's also Backblaze B2 (public beta): [https://www.backblaze.com/b2/cloud- storage.html](https://www.backblaze.com/b2/cloud-storage.html)

Their pricing is great (0.5c/month) but I'm a little worried about their single DC.

<comment><USER7> 3 days ago they sent a newsletter about their new (alpha quality) b2sync tool which essentially is a "rsync to backblaze" utility.

This makes their offer very interesting.

<comment><USER8> Can this tool offer deduplication (so that changing a folder name does not re- upload thousands of files) or is that something I would have to code into my own backup solution?

<comment><USER7> No, it is even less than rsync. It can only upload a folder recursively and skip files with common modification time on local and remote.

Considering though that before that you couldn't even upload a directory, only files, this is a huge step. :)

<comment><USER9> I currently need some help whipping up a solution for this.

<comment><USER10> git with lfs

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> OP here. Some updates and clarifications are in order!

First of all, I just woke up (it's morning here in Helsinki) and found a nice email from Amazon letting me know that they had refunded the retrieval cost to my account. They also acknowledged the need to clarify the charges on their product pages.

This obviously makes me happy, but I would caution against taking this as a signal that Amazon will bail you out in case you mess up like I did. It continues to be up to us to fully understand the products and associated liabilities we sign up for.

I didn't request a refund because I frankly didn't think I had a case. The only angle I considered pursuing was the boto bug. Even though it didn't increase my bill, it stopped me from getting my files quickly. And getting them quickly was what I was paying the huge premium for.

That said, here are some comments on specific issues raised in this thread:

\- Using Arq or S3's lifecycle policies would have made a huge difference in my retrieval experience. Unfortunately for me, those options didn't exist when I first uploaded the archives, and switching to them would have involved the same sort of retrieval process I described in the post.

\- During my investigation and even my visits to the AWS console, I saw plenty of tools and options for limiting retrieval rates and costs. The problem was that since my mental model had the maximum cost at less than a dollar, I didn't pay attention. I imagined that the tools were there for people with terabytes or petabytes of archives, not for me with just 60GB.

\- I continue to believe that "starting at $0.011 per gigabyte" is not a honest way of describing the data retrieval costs of Glacier, especially when the actual cost is detailed, of all things, as an answer to a FAQ question. I hammer on this point because I don't think other AWS products have this problem.

\- I obviously don't think it's against the law here in Finland to migrate content off your legally bought CDs and then throw the CDs out. Selling the originals, or even giving them away to friend, might have been a different story. But as pointed out in the thread, your mileage will vary.

\- I am a very happy AWS customer, and my business will continue to spend tens of thousands a year on AWS services. That goes to something boulos said in the thread: "I think the reality is that most cloud customers are approximately consumers". You'd hope my due diligence is better on the business side of things, as a 185X mistake there would easily bankrupt the whole company. But the consumer me and the business owner me are, at the end, the same person.

</comment><comment><USER11> Glacier's pricing structure is complicated, but fortunately it's now fairly straightforward to set up a policy to cap your data retrieval rate and limit your costs. This was only introduced a year ago, so if like Marko you started using Glacier before that it could be easy to miss, but it's probably something that anyone using Glacier should do.

[http://docs.aws.amazon.com/amazonglacier/latest/dev/data- ret...](http://docs.aws.amazon.com/amazonglacier/latest/dev/data-retrieval- policy.html#data-retrieval-policy-using-console) [https://aws.amazon.com/blogs/aws/data-retrieval-policies- aud...](https://aws.amazon.com/blogs/aws/data-retrieval-policies-audit- logging-glacier/)

<comment><USER12> > fortunately it is fairly strait forward to cap your data retreival rates.

Amazon has done a great job with this feature. By doing a poor job implementing something for an extremely narrow usecase, in a technology that is outdated and then providing the most complicated pricing structure surrounding every aspect of the product one can't helpbut use the feature: any other provider or service.

Like, wtf would be the usecase for amazon glacier in 2016? I dont think I would put hubdreds of petabytes of sata into 20 year cold storage, and the author of this post certainly wouldnt use it again. The fact that i need to read 2 pages of pricing docs and then the 2 pages you linked to control them because I cant estimate them myself, is a sure sign this is absurd

<comment><USER13> Tape storage is still the most optimal form of long-term storage. If you need to store things for an exceptionally long time, such as financial data, scientific data, etc, then you're going to get the most bang for your buck on tape.

<comment><USER12> The post states he paid $150 yo retrieve 60gb of data. For $150 you can buy a 5tb hdd drive.

What usecase could the price delta make sense to have a 4 hour feedback loop and all of your important data locked in someone elses data center.

the usecase where your data is so properly massive that this makes sense && you don't have the storage infrastructure in place, is so narrow that it doesnt make sense.

It is basically one research student's crawl data

Edit: also, s3 is pretty cheap. So again, i dont really see the usecase here. How much room is in the market between your own physical or digital system and amazon s3 or an equivalent. you would have to have a massive amount of data you dont care about and be very price sensitive.

<comment><USER14> Durability is one reason. A physical hard drive in my desk is so much more susceptible to destruction or loss or theft.

<comment><USER15> This.

The data I care about is already backed up on two different multi-TB drives at home, and another one at work.

Glacier is the contingency for "something took out the original data and all three backups in two different locations 7 or 8km apart - if I'm still alive after whatever just happened, I'll consider whether or not to pay Amazon a grand or so to retrieve it quickly from Glacier, or wait ~20 months to get it all for single-digit-dollars".

<comment><USER16> If you're talking about personal data, why not just use backblaze and amazon's consumer unlimited cloud storage?

That gives you 2 backup providers that can durably store everything and it's free and quick to access. Why deal with all the harddrives and glacier?

<comment><USER14> Neither of those seem free - am I looking in the wrong place? Also there's no Linux client for either AFAICT.

<comment><USER16> Right the services cost money but retrieval is free. Going by the cost of the harddrives amortized out, it'll probably be the same or less. You get far more durability and less complexity with universal web access.

I believe there are other similar services for Linux or you can just use browser to upload files with Amazon.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Arq has a fantastic Glacier restore mechanism. You select a transfer rate with a slider, and it informs you how much it will cost and how long it will take to retrieve. It optimizes this with an every-four-hours sequencing as well. See [https://www.arqbackup.com/documentation/pages/restoring_from...](https://www.arqbackup.com/documentation/pages/restoring_from_glacier.html) for reference.

<comment><USER18> It's unfortunate that Arq forces you to archive in their proprietary format. That locks you in to the tool.

<comment><USER19> I was also concerned about this when I was looking into Arq, so I wrote a cross-platform restoration tool that'll also work on Windows and Linux (not just Mac): [https://github.com/asimihsan/arqinator](https://github.com/asimihsan/arqinator)

This is purely based on the author's excellent description of the format in his arq_restore tool: [https://www.arqbackup.com/s3_data_format.txt](https://www.arqbackup.com/s3_data_format.txt)

</comment></comment></comment><comment><USER20> The only use case I would be willing to commit to glacier would be legal-hold or similar compliance requirement.

The idea would be that the data would either never be restored or you could compel someone else to foot the bill or using cost sharing as a negotiation lever. (Oh, you want all of our email for the last 10 years? Sure, you pick up the $X retrieval and processing costs)

Few if any individuals have any business using the service. Nerds should use standard object storage or something like rsync.net. Normal people should use Backblaze/etc and be done with it.

<comment><USER21> Back when I worked in banking we had requirements like that (though we didn't use glacier)

We had a legal requirement to be able to product up to 7 years worth of bank statements upon receipt of a subpoena.

Not "reproduce the statements from your transactions records" but "give us a copy of the statement that you sent to this person 6.5 years ago"

We had operational data stores that could generate a new statement for that time period, but if we received the subpoena then we needed to be able to produce the original, that included the (printed) address that we sent it to, etc.

We had (online) records of "for account 12345, on 27th October 2011, we sent out a statement with id XYZ", we'd just need a way to pull up statement XYZ.

There's no way(^) we'd _ever_ get subpoenaed for more than 5% of our total statement records in a single month, so something like Glacier would have been a great fit.

We had other imaging+workflow processes where we'd receive a fax/letter from a client requesting certain work be undertaken (e.g a change of address form). 90 days after the task was completed, you could be _pretty sure_ that you wouldn't need to look at the imaged form again, but not 100% sure. We could have used glacier for that.

We use case that would have cost us (rare, but we needed to plan for it) was "We just found that employee ABC was committing fraud. Pull up the original copies of all the work they did for the 3 years they worked here, and have someone check that they performed the actions as requested." Depending on circumstances & volume that might trigger some retrieval costs, but the net saving would almost certainly still be worth it.

(^) Unless there was some sort of class action against us, but that's not a scenario we optimised for.

</comment></comment><comment><USER22> Glacier is not a cheap/viable backup

its even less suited to disaster recovery (unless you have insurance)

Think about it. For a primary backup, you need speed and easy of retrieval. Local media is best suited to that. Unless you have a internet pipe big enough for your dataset (at a very minimum 100meg per terabyte.)

4/8hour time for recovery is pretty poor for small company, so you'll need something quicker for primary backup.

Then we get into the realms of disaster recovery. However getting your data out is neither fast nor cheap. at ~$2000 per terabyte for just retrieval, plus the inherent lack of speed, its really not compelling.

Previous $work had two tape robots. one was 2.5 pb, the other 7(ish). They cost about $200-400k each. Yes they were reasonably slow at random access, but once you got the tapes you wanted (about 15 minutes for all 24 drives) you could stream data in or out as 2400 megabytes a second.

Yes there is the cost of power and cooling, but its fairly cold, and unless you are on full tilt.

We had a reciprocal arrangement where we hosted another company's robot in exchange for hosting ours. we then had DWDM fibre to get a 40 gig link between the two server rooms

</comment><comment><USER23> The post is a useful cautionary tale, and he's not alone in getting burned by Glacier pricing. Unfortunately it was OP not reading the docs properly.

Yes, the docs are imperfect (and were likely worse back in the day). And it was compounded by the bug, apparently. But it's what everyone on HN has learned in one way or another... RTFM.

Was it mentioned in the article that the retrieval pricing is spread over four hours, and you can request partial chunks of a file? Heck, you can retrieve always all your data from Glacier for free if you're willing to wait long enough.

And if it's a LOT of data, you can even pay and they'll ship it on a hardware storage device (Amazon Snowball).

Anyone can screw up, I'm sure we all have done, goodness knows I have. But at the very least, pay attention to the _pricing_ section, especially if it links to an FAQ.

<comment><USER24> I would say the FM in this case was unreadable. "starting at $0.011 per gigabyte", "learn more" \- no one would expect to pay $150 here.

<comment><USER25> It's definitely sleazy of Amazon to hide the pricing info like that, but I can't imagine seeing that sentence while making a purchase and not clicking through to see how the pricing actually works. But I guess that's just me.

<comment><USER24> Not just you. But read the page referred by that "Learn more" link. It's very unclear. I mean, or course, it has unambiguous meaning. But to understand it you need to read a lengthy sheet of prose, and only deep in it, after many definitions, you must notice a phrase "we multiply your peak hourly billable retrieval rate" ... "by the number of hours in a month". What concentration of attention, and how much time a reader need to spend, to understand this important nuance?

As far as I see, in all other places they specify their pricing "per GB", and only this small phrase uncovers real meaning, which is not per actual GBs you transferred, but your peak rate multiplied by number of hours in month. IMHO this should be one of the first phrases describing the pricing model.

</comment></comment></comment></comment><comment><USER26> This sounds a lot like demand-billing [1] [2] that's common with electric utilities, particularly commercial, and increasingly, people with grid-tied solar installations. [citation needed]

You pay a lower per-kilowatt-hour rate, but your demand rate for the entire month is based on the highest 15-minute average in the entire month, then applied to the entire month.

You can easily double or triple your electric bill with only 15 minutes of full-power usage.

I once got a demand bill from the power company that indicated a load that was 3 times the capacity of my circuit (1800 amps on a 600 amp service). It took me several days to get through to a representative that understood why that was not possible.

[1] [http://www.stem.com/resources/learning](http://www.stem.com/resources/learning)

[2] [http://www.askoncor.com/EN/Pages/FAQs/Billing-and- Rates-8.as...](http://www.askoncor.com/EN/Pages/FAQs/Billing-and-Rates-8.aspx)

</comment><comment><USER27> You don't have a backup until you test its restore.

<comment><USER28> I've never heard that and I'm stealing it.

<comment><USER29> Come on, don't downvote someone for learning something (you think is) well known for the first time. Especially when their response is "that's great and I'm going to use it". [https://xkcd.com/1053/](https://xkcd.com/1053/)

<comment><USER30> The comment is noise - no different from "+1" or "me too". If you found a comment helpful and want to thank someone for it, the way to do that is an upvote.

<comment><USER28> It's a bit different, I added that I had never heard of the phrase and that I would be using it.

</comment></comment></comment></comment></comment><comment><USER31> Even with the large data retrieval bill he still saves ~$100 vs the price of keeping that data in S3 over the same time period. Reading this honestly makes me think glacier could be great for a catastrophic failure backup.

<comment><USER32> Compare to Google Drive at 100 Gigs for2 bucks and no bandwidth charges that I'm aware of.

<comment><USER31> I looked up the Google drive pricing and the costs increase quickly over that 100GB level. I'm going to use 3TB of data for my example because that is approximately the amount of data I would be backing up at work. The cost for Google Drive would be the $99 a month 10TB plan, Amazon Glacier is $21.51 a month. This is before you get into things like having the enterprise AWS ecosystem with IAM versus a single user Gmail account. Remember I am only talking about retrieval in the case of a catastrophic failure, the data is already backed up elsewhere. As long as I can manage to go a year without destroying all my backups Glacier comes out on top over Drive even taking into account the retrieval fees. In the best case scenario I never ever retrieve that data.

</comment></comment></comment><comment><USER33> If one can download a percentage for free each month - 5% in this case, and the price of storage is dirt-cheap, then couldn't one just dump empty blocks in until the amount desired for retrieval falls under the 5% limit? In this case, if one wants to retrieve 63.3 GB, uploading 1202.7 GB more for a total of 1266 GB, 63.3 GB of which represents just under 5%. There's no cost for data transfer in and the monthly cost at $0.007/GB would be just $8.87. And that's just for the one month because everything wanted would be coming out the same month.

Has anyone tried this or know of a gotcha that would exclude this?

And I realize that for the OP's situation, it wouldn't have mattered since he thought he was going to get charged a fraction of this.

<comment><USER34> That's possibly a way to trick the system, but storing 63.3 GB in S3 would cost OP less than 2$ for standard availability and less than 1$ for reduced availability, not counting request costs (which are not as surprising as the hidden costs in question here). At this scale you should just store it in S3 and be done with it.

<comment><USER33> sure, I wasn't intending to suggest this as a good premeditated maneuver, but there are probably other individuals out there who have found themselves in a similar position to the OP and considering their predicament.

If you've gotten into Glacier for the wrong reason, you may already be in the trap, and you can quickly rip yourself free and take a bunch of skin, spend almost 2 years ever so gently prying yourself free, or maybe a third way. That's my angle here. Also, traps don't have to be laid for someone to feel like he's in one, so I'm not putting that on AWS.

The cheapest way out seems to be to just grab 5%/month over 20 months, but that's a lot of sustained effort and contact with the service. So I could see a trick like this as a potential middle ground, at three months and ~$30 according to previous comment's details.

</comment></comment></comment><comment><USER35> Glacier is more comfortable to use through S3, where you upload and download files with the regular S3 console, and just set their storage class to Glacier with a lifecycle rule. I've used the instructions in here to do it: [https://aws.amazon.com/blogs/aws/archive-s3-to- glacier/](https://aws.amazon.com/blogs/aws/archive-s3-to-glacier/)

<comment><USER36> That can bite you if you have a lot of small files[1], which with automatic archiving of S3 can happen.

[1] [https://therub.org/2015/11/18/glacier-costlier- than-s3-for-s...](https://therub.org/2015/11/18/glacier-costlier-than-s3-for- small-files/)

</comment></comment><comment><USER37> If downloading more than 5% of stored data is so expensive, wouldn't it have been cheaper to upload a file 19 times the size of the stored data (containing /dev/urandom)? After that, downloading just 5% of total data would have been free.

<comment><USER38> It still wouldn't have been free. The free download allowance is spread daily across the entire month. That is, you can download 5% of your data per month, and only 0.16% of it per day. For your optimization to work, you'd have to retrieve your data over 30 days for it to be free.

</comment></comment><comment><USER39> I've had some big Glacier bills in the past, even the upload pricing has gotchas[1]

These days the infrequent access storage method is probably better for most people. It is about 50% more than Glacier (but still 40% of normal S3 cost) but is a lot closer in pricing structure to standard S3.

Only use glacier if you spend a lot of time working out your numbers and are really sure your use case won't change.

[1] - 5 cents per 1000 requests adds with with a lot of little files.

<comment><USER40> I use Infrequent Access Storage for backups, through a tool called duplicity (or more aptly, I use a GUI front-end for that tool called Deja Dup). Instead of uploading every individual file, it gathers them into 25 mb .tar files & uploads those along with an index describing where each actual file is. That has made the requests negligible for me.

<comment><USER39> We combined the files too.

And then a customer wanted all their files rather than just one or two. Although that was billed back to the them.

</comment></comment></comment><comment><USER41> Pricing should always be made straight forward, easy to understand, and that pricing plan is dodgy as hell

<comment><USER42> Pricing plans FOR CONSUMERS should always be straightforward and easy to understand. This is not a consumer product.

Pricing plans FOR B2B should model, as effectively as possible, the underlying costs -- this allows the provider to offer the lowest possible pricing for the services that cost them the least to provide, with expensive services priced accordingly. As others have mentioned on this thread, utilities are really, really good at this -- they come up with extremely complex rate plans for their largest customers that help them achieve whatever economies they are aiming for, for example incentivizing customers to provide level-loading (which is effectively what Amazon is doing in this retrieval scheme).

<comment><USER43> > This is not a consumer product.

That's a manufactured excuse for a fundamentally bad product api and pricing structure. When dropbox is more useful than AWS, amazon has screwed the pooch (which they do pretty often). Segmenting users by arbitrary circuitous logic into "consumers" (can't find a good use for it) and "enterprise" (can find a good use for it) isn't constructive. Both classes should avoid it, because it's not even an inexpensive choice, for what you get.

<comment><USER16> How is Dropbox more useful? Dropbox and AWS Glacier are vastly different products and show exactly the divide between consumer and enterprise that you say doesn't exist.

</comment></comment></comment></comment><comment><USER44> Seems like "precise prediction and execution of Amazon Glacier operations" might be a niche product people would pay for (and probably already exists for enterprise use cases?)

That's something that generally keeps me from using AWS and many other cloud services in many cases: the inability to enforce cost limits. For private/side project use I can live with losing performance/uptime due to a cost breaker kicking in. I can't live with accidentally generating massive bills without knowingly raising a limit.

<comment><USER45> You can do it pretty easily with the AWS APIs, and in the process scram-switch only the stuff you really want to kill.

<comment><USER44> How would you script a protecting against the issue described in the article? Unless you make sure to check the cost before each single request you can't stop it (and if you accidentally send one massive retrieval request even that isn't enough)

For other services it is easier, but even then, setting up and managing my own cost control mechanism is a level of complexity (and risk of failure) I'd really want to avoid, esp. since I probably use AWS to _avoid_ management overhead.

<comment><USER45> You can't in this particular use case, but I can't envisage AWS providing a cost-control system that would stop this, either. It doesn't make sense-- they're not calculating costs as they go. What I'm saying is that what Amazon would provide you is not functionally different from what you can do yourself right now.

I would be a lot more worried about a risk of over-charging myself if AWS wasn't incredibly good about refunding accidental overages.

<comment><USER44> as user _re_ pointed out above, apparently for Glacier that functionality actually exists: [https://docs.aws.amazon.com/amazonglacier/latest/dev/data- re...](https://docs.aws.amazon.com/amazonglacier/latest/dev/data-retrieval- policy.html#data-retrieval-policy-using-console)

<comment><USER45> Huh, that's...surprising. Glad it exists, though, as Glacier definitely is opaque. Thanks for pointing that post out (to you and to `re).

</comment></comment></comment></comment></comment></comment><comment><USER46> "The problem turned out to be a bug in AWS official Python SDK, boto."

My only experience of using boto was _not_ good. Between point versions they would move the API all over the place, and being amazon some requests take ages to complete.

After that worked with google APIs which were a better, but still not what I'd describe as fantastic (hopefully things are better over last 2 years).

</comment><comment><USER47> Wouldn't it be better for the OP to simply upload 20 * 60GB (= 1.8TB) of random data, wait a month (paying less than 20 USD), and then download the initial 60GB within his 5% monthly limit?

<comment><USER11> No, because he can download the 60GB at a slower, cheaper rate, paying less than the storage for the 1.8TB and getting it sooner than a month.

<comment><USER48> How?

<comment><USER44> By requesting the CDs one after another (or small numbers in parallel), keeping the data rate low.

</comment></comment></comment></comment><comment><USER49> This article claims that glacier uses custom low-RPM hard disks, kept offline, to store data.

Does s/he substantiate this claim in any way? AFAIK glacier's precise functioning is a trade secret and has never been publicly confirmed.

<comment><USER50> I think this is a good guess, [http://storagemojo.com/2014/04/25/amazons- glacier-secret-bdx...](http://storagemojo.com/2014/04/25/amazons-glacier- secret-bdxl/)

<comment><USER50> Oh, there also a hacker news thread about this

[https://news.ycombinator.com/item?id=4416065](https://news.ycombinator.com/item?id=4416065)

</comment></comment></comment><comment><USER51> Considering the fact that bugs in the official APIs resulted in multiple retry attempts, he should demand some of his money back.

<comment><USER44> The retries don't cost extra. Sending off all 150 retrieval requests for all the data at once set the retrieval rate and the price, they should have been staggered over a lot of time to keep the rate low.

</comment></comment><comment><USER52> They should rename this service to Amazon Iceberg

</comment><comment><USER53> About a year ago NetApp bought Riverbed's old SteelStore (nee Whitewater) product -- it's an enterprise-grade front-end to using Glacier (and other nearline storage systems). It provide a nice cached index via a web GUI that let you queue up restores in a fairly painless way. It even had smarts in there to let you throttle your restores to stay under the magical 5% free retrieval quota. It's not a cheap product, and obviously overkill for a one- off throw of 60GB of non-critical data ... but point being there are some good interfaces to Glacier, and roll-your-own shell scripts probably aren't.

As noted by others here, if you treat glacier as a restore-of-absolute-last- resort, you'll have a happier time of it.

Perhaps I'm being churlish, but I railed at a few things in this article:

If you're concerned about music quality / longevity / (future) portability - why convert your audio collection AAC?

Assuming ~650MB per CD, and the 150 CD's quoted, and ~50% reduction using FLAC, I get just shy of 50GB total storage requirements -- compared to the 63GB 'apple lossless' quoted. (Again, why the appeal of proprietary formats for long term storage and future re-encoding?)

I know 2012 was an awfully long time ago, but were external mag disks really that onerous back then, in terms of price and management of redundant copies? How was the OP's other critical data being stored (presumably not on glacier). F.e. my photo collection has been larger than 60GB since way before 2012.

Why not just keep the box of CD's in the garage / under the bed / in the attic? SPOF, understood. But world+dog is ditching their physical CD's, so replacements are now easy and inexpensive to re-acquire.

If you can't tell the difference between high-quality audio and originals now - why would you think your hearing is going to improve over the next decade such that you can discern a difference?

And if you're going to buy a service, why forego exploring and understanding the costs of using same?

<comment><USER54> > Assuming ~650MB per CD, and the 150 CD's quoted, and ~50% reduction using > FLAC, I get just shy of 50GB total storage requirements -- compared to the > 63GB 'apple lossless' quoted. (Again, why the appeal of proprietary formats > for long term storage and future re-encoding?)

I did a comparison between FLAC and ALAC (a.k.a. Apple Lossless) on my CD library a few years ago (plus a few 48kHz tracks taken from DVDs), and the difference in total filesize was less than 10% so I doubt that is a major factor. I personally went for ALAC, as it has equal (EAC, VLC) or better support (OS X Finder, iTunes, Windows Explorer, Windows 10 media player, some tagging scripts, iOS) in stuff I currently use. Providing I keep a decoder with the files, its proprietary nature doesn't really bother me - I can always convert to xLAC if desired.

<comment><USER53> Interesting, thanks.

I wouldn't use a proprietary format because I could neverbe sure when in the future I'd want to read / re-encode, and what type of systems I'd have available at that timer, other than knowing I'd always have access to free software.

I have some FLAC archives, but I don't _use_ them - so support to play that format hasn't been something I've taken much notice of. Do you normally play your ALACs, or keep an mp3 / ogg / aac version around to actually listen to?

<comment><USER55> It's not exactly proprietary since Apple open sourced the ALAC code in 2011.

[https://alac.macosforge.org](https://alac.macosforge.org)

Also, someone has wrapped it to build with different tool chains.

[https://github.com/TimothyGu/alac](https://github.com/TimothyGu/alac)

</comment></comment></comment></comment><comment><USER56> Does anyone have a success story for this type of backup and retrieval on another service?

</comment><comment><USER57> I briefly used Glacier for daily backups as a failsafe if our internal tape backups failed when we needed them. The 4 hour inventory retrieval when I went to test the strategy and the bizarre pricing quickly make me look at other options.

</comment><comment><USER58> I have a strong feeling that he would get a refund if he contacted Amazon support, considering it was caused by a bug in the official SDK and he didn't ACTUALLY use the capacity he's being asked to pay for.

<comment><USER59> Multiple requests didn't cause the issue. It was asking for all of his data to be queued in such a small window of time. The same thing would have happened without the bug.

<comment><USER60> That said, this is Amazon, who will refund you for a product if you ship it to the wrong address accidentally...I'm sure OP could get a refund.

</comment></comment></comment><comment><USER61> This is why I break my large files uploaded to Glacier into 100MB chunks before uploading. If I ever need them, I have the option of getting them in a slow trickle.

<comment><USER23> This is no longer necessary, they now allow you to specify ranges of partial files. So you can split a single large file into multiple requests to keep you within allowance or within budget.

<comment><USER61> Nice. I also upload par2 checksum files for each chunk so changing things now would involve a bit of script rewriting, but that's good to know for the future.

<comment><USER62> Is that necessary? They report a checksum for every file when you request an inventory.

<comment><USER61> No, it's not necessary at all, I hope. I probably should have called them just "Par2" files not Par2 checksum files. They aren't just checksums. They allow reconstruction of the original file even after fairly extensive damage. Which isn't likely, so, yes, not strictly necessary, but who knows, could be useful in the what-if scenario. The checksums you mention only help with checking, not with repair.

[https://en.wikipedia.org/wiki/Parchive](https://en.wikipedia.org/wiki/Parchive)

</comment></comment></comment></comment></comment><comment><USER63> Perhaps a naive question but why would glacier try to discourage bulk retrieval? Is it because the data is fragmented physically?

<comment><USER64> We don't actually know how glacier data is stored (though there are several theories from regular s3 to tape robots to experimental optical media).

<comment><USER65> I suppose part of the neat trick of it is that because we don't have to know, Amazon can switch it out for something else anytime it's convenient for them, or some new tech comes up. Or split thing up among several methods and compare costs. As long as they structure their operations such that nothing is ever lost and everything can be retrieved with a few hours notice at any time, they can try anything they want.

<comment><USER66> How would us knowing prevent them from doing this?

<comment><USER30> Whatever information became public, people would write scripts and the like with those assumptions baked in. Even if it wasn't officially documented, it'd be bad PR for Amazon to break things people were relying on.

</comment></comment></comment></comment></comment><comment><USER67> For cheap storage there is also Oracle Archive Storage with 0.1c/GB ($0.001/GB). They have horrible cloud management system though.

[https://cloud.oracle.com/en_US/storage?tabID=1406491833493](https://cloud.oracle.com/en_US/storage?tabID=1406491833493)

</comment><comment><USER68> > I'd need more than one drive, preferably not using HFS+, and a maintenance > regimen to keep them in working order.

I'm really doubting the need for a maintenance regimen on a drive which is almost entirely unused. Could have spent $50 on a magnetic-disk-drive and saved yourself hours worth of trouble.

<comment><USER69> The problem with physical drives is that they can (and do!) fail. The authors point is surely that the backup would need to be checked periodically, and drive failures dealt with.

Does magnetic media like this (especially spinning disk) suffer from bit-rot? What about the possibility of mechanical failure?

I'd never rely on mechanical disks as the one and only backup of any data critical to me - a two tier approach of mechanical for fast retrieval, and cloud/online backup seems to be the safest bet.

</comment></comment><comment><USER70> For unique data you want super robust storage options, both local and remote. But for something as generic as ripped CDs? Why bother? Just use an external drive or two if you are super worried about one dying. Even if you lose both drives the data on them isn't impossible to replace.

</comment><comment><USER71> Wow, thanks for this!

I currently have 100gb of photos on Glacier. I am going to be finding another hosting provider now.

</comment><comment><USER72> So depending on how the "average monthly storage" is computed you could get 20x more data in one month and then retrieve the 5% (previously 100%) that you care about for free, and then delete the additional data?

<comment><USER73> There's a 90 day minimum storage cost for data.

</comment></comment><comment><USER74> You will ALWAYS pay more that you expect when you use AWS (and probably other cloud services). This case is quite extreme, but the way costs are assigned, is quite complicated not to miss something at some point...

</comment><comment><USER75> I was looking at Glacier for my backups, but it seemed to complicated ... glad I didn't use it.

I ended up using some cheap VPS, two of them located in two different countries. And it's still cheaper then say Dropbox.

</comment><comment><USER76> Curious: if you use a "general storage provider" (like glacier) for backup, rather than a "pure backup provider" (like Backblaze, CrashPlan) why is that?

<comment><USER61> I control the encryption algorithms, the compression scheme, the file chunking strategy, the encryption keys, the encryption of archive/file names, the file naming scheme, the addition of Par2 files, everything.

And I don't pay the overhead of an add-on service.

Also I back up stuff that's not on my hard drive (only on external USB drives) and I'm not sure how the services handle that.

If the services give me some of these points, that's not sufficient; they would have to give me all of these points. Only then would I consider them. All things being equal I'd be willing to pay for some convenience but my current solution is all scripted so it's pretty darn convenient.

<comment><USER76> > I control the encryption algorithms, the compression scheme, the file > chunking strategy, the encryption keys, the encryption of archive/file > names, the file naming scheme, the addition of Par2 files, everything.

I can see why detailed control would be one reason, but you could still just have a very controlled backup to your own storage location(s) as a first step and just let a backup service bulk store your already named and encrypted files? It's only the last-resort you need to go to so if it's a huge blob of encrypted data that shouldn't matter too much -- you only need to access that in case of a total disaster where you lost all your own backup endpoints first.

> And I don't pay the overhead of an add-on service.

The reason I'm asking is because I was under the impression that backup services are much _cheaper_ than pure storage, while still offering some conveniences such as versioning/backup apps. Glacier charges $0.007 per GB per month, that's $7/month just for a single 1TB machine, just for a single version (If my math is correct, it's early)! If you have dozens of versions it quickly adds up.

I do 10 machines at around 1TB on average, unlimited storage in unlimited versions, at $1.25 per machine per month (flat rate, regardless of storage volume). I have tried building my own machines, tried looking at storage providers etc., but can't get near.

Even if I did only 1-2 machines, the cost in Glacier would break the backup service cost already at a couple of TB total storage.

<comment><USER61> Great analysis, but why "per machine"?

I back up data, not machines. I would prefer that third party backup software not have access to the unencrypted files on my machines.

The pricing you mention is compelling. Which service is that? Does your model work if the data is on multiple external drives?

Getting down to a specific example, say I have one laptop and three external drives. Do the backup services you have in mind work with this setup? How would they charge?

<comment><USER76> The service I use is the CrashPlan "family" plan. If I would just gather all the data to one machine instead it would work too (could just keep it on a file server with a directory per machine or whatever). The only reason "machines" are useful is for having separate easy restores -- basically my parents or mother in law can restore their machines from the other side of the country.

As long as you just keep the external drives connected to the same machine during backup, the backup application doesn't care where the file set to backup is mounted. its just a directory list per machine.

I used to backup "data" too, by using a first step of backup of multiple machines to a NAS and then only backup that data to the cloud. However, I sacrificed that extra security for the added convenience of direct restore of individual files and machines. It also reduced the risk of me having done a mistake in the backup config of the first step (which I estimated to be a way higher risk than hardware failure, fire or a data breach at the cloud provider, since I have basically never configured anything right). Being able to easily fetch an individual file from 1 day or 1 week ago can really save time. Edit: also remember that the backup client on Linux uses ram proportional to the backup size so on my cheapo NAS I also outgrew its ram and would have had to get a faster one or a file server, that was also partly why I left it.

The good thing about the backup pricing is that it doesn't increase with aggregated backups like normal storage. It's 150/year so to be competitive you need a few TB, but you can very easily backup your parents machines too and save some time around Christmas... Note though that people sharing the same plan can see each others data (or so I presume)

</comment></comment></comment></comment></comment><comment><USER77> So ... why not just upload and additional 60GB / 0.05 and then download the entire 60GB which is now 5% of the total storage for free ?

</comment><comment><USER78> Does anyone have a backup script for backblaze or a similar windows app like SimpleGlacier Uploader?

</comment><comment><USER79> So, what's the most cost effective way to download all your files from Glacier then?

<comment><USER80> Spread the download over time. Over 20 months would be free. Over 1 month would be the quoted per-GB charge based on your peak download rate.

</comment></comment><comment><USER81> If there's a bug in Amazon's libraries, can't you ask for a refund?

<comment><USER82> Once he was charged the $150, it didn't cost him any more to try again to download it, because of how the pricing works. So, he would have been charged that much no matter what if he'd downloaded the data at that speed, and there was nothing to refund once he successfully got his data out.

</comment></comment><comment><USER83> I don't get Glacier. It's painfully slow, painful to use and insanely expensive. [https://hubic.com/en](https://hubic.com/en) is $5/month for 10 Tb, with unmetered bandwidth. A far better option for backups.

<comment><USER84> I'm interested in Hubic, but where do you read that bandwith is unmetered?

</comment></comment><comment><USER85> Impressive that Amazon can choose to serve a request at 2x the bandwidth you need, with no advance notice, and charge you double the price for the privilege.

<comment><USER62> Can you explain what you mean here? I'm fairly certain it isn't true.

</comment></comment><comment><USER86> This is a simple case of spending more than you should have because you didn't understand the service you were using. It's impacted a little worse by how silly the whole endeavor is, given the preponderance of music streaming services.

<comment><USER87> Uh no, actually this is really a case of a bug in OFFICIAL SDK that caused a higher bill than expected.

<comment><USER44> No, requesting all his data at once alone determined the rate. The retries didn't cost extra.

</comment></comment></comment><comment><USER48> I'm surprised that the author had 150GB of Creative Commons audio CDs to begin with!

<comment><USER88> Are you trying to say that the OP was not allowed to back up their own CDs? I have several boxes of space wasting CDs of my own that have all been ripped in lossless, and feel somewhat insulted by that notion. Note: I live in Australia, a somewhat less insane country for DRM than the US (AFAIK).

<comment><USER89> Also in Australia, and its not that black-and-white in my understanding:

This is known as "Format Shifting" — taking one copyrighted medium and converting it to another. In Australia, you are explicitly not allowed to do this with CDs, DVDs and Blu-rays.

You are only allowed to keep a digital copy if you continue to retain the original — a backup. If the original is lost or destroyed, your digital copies must be discarded.

For example, you can rip a CD and put it on your iPod, or computer, as long as you continue to own the CD. The issue here is that in both cases you also control the device you are copying it to. You don't control but rather lease space on Amazon's servers — so it introduces a grey area on whether you are allowed to backup to such places and whether putting data on those servers constitutes distribution of the copyright material.

Realistically, none of this is black-and-white and Amazon could flag it as infringing content and remove it just to cover themselves against DMCA complaints anyway. This is true in both Australia and the US, regardless of the differences in copyright law (Australian copyright law offers far fewer protections than the US, incidentally) because both having similar DMCA laws.

<comment><USER88> Saw this too late to edit my other reply; an interesting overview of what you may or may not do:

[http://www.lifehacker.com.au/2013/01/format- shifting-101-wha...](http://www.lifehacker.com.au/2013/01/format- shifting-101-what-are-your-legal-rights-in-australia/)

Also backed up by: [http://copyright.com.au/about- copyright/exceptions/](http://copyright.com.au/about-copyright/exceptions/) \- which states we're allowed to "space shift" music.

</comment></comment></comment></comment><comment><USER28> I don't like how the title and article reads like a hit piece on Amazon Glacier. It's great at what it is intended for. In addition it seems he still saved money because over 3 years because the $9 a month savings added up to more than the $150 bill for retrieval.

I'm surprised that this aspect has not been mentioned here in the comments yet:

> I was initiating the same 150 retrievals, over and over again, in the same > order.

This was the actual problem that resulted in the large cost.

At my old job we would get a lot of complaints about overage charges based on usage to our paid API. It wasn't as complicated of pricing as a lot of AWS services, just x req / month and $0.0x per req after that, but every billing cycle someone would complain that we overcharged them. We would then look through our logs to confirm they had indeed made the requests and provide the client with these logs.

<comment><USER44> > _> I was initiating the same 150 retrievals, over and over again, in the > same order.

This was the actual problem that resulted in the large cost._

Except that it wasn't. The repeated requests were free, because he already set the maximum rate with the first wave of requests. Surprising?

Also, it really is not a hit piece. It's an honest report of what he did (and what he did wrong) and that he thinks the docs aren't as clear as they could be.

<comment><USER28> Thanks, I guess I glossed over that part of the article but when I got to the end and saw the original quote I assumed the worst. The title alone is pretty inflammatory in my opinion.</comment></comment></comment>


Want to Find a Misinformed Public? Facebook's Already Done It - <AUTHOR> https://themarkup.org/coronavirus/2020/04/23/want-to-find-a-misinformed-public-facebooks-already-done-it <comment><USER2> It seems that these topics are created by scraping Wikipedia article titles. For instance, try typing "List of people".</comment>


Grumpy Website – a blog about everything wrong with modern web & tech - <AUTHOR> https://grumpy.website <comment><USER3> If you open more than one draw at a time you would unbalance the drawer and it would tip over, falling on you and possibly injuring your feet and damaging your stuff and the floor.

This safety mechanism is not there if the drawer is embedded in a larger frame where tipping over is not possible.

Sometimes it's not bad UX, it's just the thing trying to protect you from your stupidity.

<comment><USER4> That makes sense for a 4 drawer, 5' tall filing cabinet -- which generally all have the interlock, and little to none for a 3 drawer under desk unit with drawers half the length or less of a filing cabinet. Even if you wanted to load them as heavily as a file cabinet, the drawers are generally far less substantial or well made. Doubly pointless considering the drawer unit shown has no filing drawer. Most of those solve the possible tip with a castor or slide under the front of the sole bottom filing drawer.

<comment><USER5> Those drawers can be used to hang folders and those can be _very_ heavy and easily tip over the whole thing if you would slide out all of them at once.

<comment><USER4> That would be a 12" deep file drawer, which either do have an interlock, or in a desk side with single file drawer the castor or slide underneath, as mentioned.

The one illustrated isn't deep enough for hanging files -- maybe the odd ream of A4, pens and pencils. Will need an awful lot of carefully placed paperclips and pencils to tip. Which is why you rarely see those locking each other out. If you use it for storing your collection of lead weights you bring it on yourself (the base of the drawer would probably fall through anyway)... :)

</comment></comment></comment></comment><comment><USER6> When trying to find more about the website on mobile, I scroll down and I only see a few links at the bottom, but within less than a second new content loads and the links are gone, I will probably need to load all the content before being able to see the credits / about links.

That's grumpy-worthy I find ;-).

<comment><USER7> I have JS default to off (saves big on load times, bypasses some "please buy a subscription" things), and thought this was just a weirdly short list of eccentric things until I saw this comment.

So, add "JS is required for crucial, rather than merely useful, features."

</comment></comment><comment><USER8> [https://grumpy.website/post/0Sqvx1C9H](https://grumpy.website/post/0Sqvx1C9H)

> Five other people are reading this doc, right? Wrong! The user with a white > avatar is actually a button to start a chat! Why does it look like a user > and is placed where all other users are, we'll never know.

> But wait, isn't there a chat button next to Share? No, that's comments! > What's the difference? Comments are square, while chat bubbles are oval!

> Also, if you ever wanted to check how your stocks are doing, there's a > button for that too.

Nice :-)

</comment><comment><USER9> My all-time anti-favorite UI thing like this is Mac dialogs where two different buttons are focused.

E.g. when closing a file with unsaved edits, you typically get a dialog with three buttons:

           [Don't save]       Cancel        [Save]

"Don't save" and "Save" are both highlighted, but in two different ways - <USER2> for added excitement, spacebar activates one of them and Enter activates the other. Feeling lucky?

<comment><USER10> Fully highlighted button responds to Enter, the most powerful action. The button with a border only is associated with the secondary action, space bar. Makes perfect sense.

<comment><USER11> It makes sense because you're already familiar with the platform convention. It's not intuitive. Space bar is a larger key and easier to reach, why isn't it associated with the more "powerful" action?

<comment><USER12> It's not about "power", it's about the default key to perform an action being enter. So dialogs will run the default action, signified by the highlighted button, when you press enter. Space is intended for selecting the retargetable focused item.

<comment><USER11> > Space is intended for selecting the retargetable focused item.

Sure, but how is that intuitive?

<comment><USER12> It's standard UI for keyboard focus across most operating systems. At some point, you're just going to have to know that space selects the item that the focus ring is around, just like you know that tab switches between fields. (By the way, this is off by default so you don't invoke it accidentally without knowing how it works.)

</comment></comment></comment></comment></comment></comment><comment><USER13> A Chrome UI idiocy: The new tab page in Chrome contains a text entry field that says to search Google or type a URL. But as soon as you start to type in it, it switches the focus to the URL bar. Why the heck do they put in the text field if they aren't going to let you use it!?

On a desktop that is mildly annoying. On a Windows tablet it is extremely annoying, because the onscreen keyboard closes when the focus leaves the text entry field.

For Chrome startup you can fix this by changing the startup page from the new tab page to a specific URL, and simply give the URL for the Google front page as that specific URL.

But Chrome doesn't seem to have a way to set the new tab page itself to start at a specific URL (without installing a third party extension) so you still have this focus changing idiocy on subsequent tabs.

A further Chrome annoyance on Windows tablets: if you tap the address bar to bring up the virtual keyboard and then try to paste something by typing CTRL and then V (CTRL on the virtual keyboard acts as a one-shot CTRL lock, so you can invoke control keys with one finger), Chrome somehow loses the CTRL part, and you get a "V" in the address bar. If you then backspace, and try CTRL V again it works.

</comment><comment><USER14> The biggest problem with modern tech for me is that it almost never works as advertised. A lot of times it's just there to tick a box on the marketing material. The most recent example, got myself a new Fuji camera and it has bluetooth and wireless and a smartphone app and theoretically you can transfer your pictures wirelessly (but not over a physical usb cable). I'd rather it didn't have these features, because it's not reliable, sometimes it won't work at all, other times it takes too long to connect, it's a mess and I have no time in my life to deal with this crap anymore.

<comment><USER15> I totally agree. For file transfer on my Fuji camera, I just take out the SD card and grab the files on a SD card reader (+ usb-c adapter) on my phone. (The Fuji wireless app won't let me transfer regular JPEGs or RAW files. Only compressed JPEGS less than 1MB large even though the original JPEGs are ~2.5 MB big.)

</comment></comment><comment><USER16> The "doesn't add up to 6.5gb" is because storage is shared with other services and doesn't pertain to just Google Drive.

I agree with Google, that it is more important to show how much overall storage you have left, than how much is being used in this particular service, but there needs to be an overview that can show where storage is used. And clicking around on Google Drive it appears there isn't such an overview.

So yes, bad UI, but for different reason than what grumpy website says.

<comment><USER17> If the fix is relatively easy, the UI is probably bad. A good UI could show "Other services x.y GB", but didn't.

Good UX/UI should provide intuition to the user, especially in more complex circumstances.

</comment></comment><comment><USER18> I've got one. Sometimes when you start typing on Google, the keyboard drops in the middle of typing causing you to tap a link where the key had been.

[https://imgur.com/a/v1CGba5](https://imgur.com/a/v1CGba5)

<comment><USER19> This issue happens in so many places. About to press a button on a page and suddenly something loads up top and the page shifts 50px.

<comment><USER20> Also when you try to dismiss a notification on the phone, and just as you're doing it a new one appears and you end up dismissing that instead. Boom! Now you don't know what it was.

This is on Android BTW, not sure how iOS does it. But instead of the newest notification appearing at the top of the list, maybe it should appear at the bottom. Problem solved.

</comment></comment></comment><comment><USER21> The git status issue he lists (that it will say it's up to date with origin/master) is really confusing to folks. Hard to wrap your head that your local "pointer" to that branch is potentially not up to date. Well that really it's an out of date copy of a remote pointer, known as a reference... That when you "fetch" it's really updating local copies of remote references.

In a world of devs that has to wrap their head around pointers or references, having copies of references, in a distributed system, doesn't exactly come naturally to most, especially new, developers. This is pretty bad for what should be baseline dev tooling. It's like having to learn regular expressions just to be able to edit your first hello world program!

That being said, git is complex, and trying to idiot proof a complex system with more abstractions only makes it more confusing when the abstraction leaks (which it will)...

<comment><USER22> Abstractions only leak if the system design doesn't match the abstractions properly. Other tools like Mercurial do a better (but not perfect) job because they get more right than git does.

</comment></comment><comment><USER23> This site makes me happy because it reminds me that other people are just as frustrated with stupid UI decisions everywhere. And that most apps are actually terrible by default. Including ones I've built. UX is hard.

<comment><USER24> If you enjoyed that there is [https://thedailywtf.com/series/errord](https://thedailywtf.com/series/errord)

Stupid UX / programming decisions documented since 2004

<comment><USER25> Are they making their own decisions based on those documentation efforts? _Scrolls to bottom of page, no prev /next buttons. Scrolls to top, "ah, there they are!" :(_

</comment></comment></comment><comment><USER26> It's Don Norman meets the Muppets!

Actually I really appreciate someone collecting collective stupidity in one place. I am constantly frustrated by terrible design and product decisions every day (both at work and in general) that I can't do anything about. Maybe shame will work.

<comment><AUTHOR> The problem is that I'm not sure it's stupidity. I bet a lot of this BS is caused by "data driven" decisions that maybe improve ad engagement by 0.1% at the expense of everyone else's productivity, but the company considers the 0.1% increase a win.

What makes me sad is that it seems like us who appreciate & demand good quality, no-nonsense software seem to be a minority, and the masses don't seem to care. I've noticed most non-tech people just accept their fate, either they've already came to terms with the idea that "tech sucks and will always suck", or they legitimately don't know how the experience can be improved (a lot of things may look absurd to us as developers, but might not to someone not familiar with how those things are built) or that they think software is delivered by gods or equivalent superior entities and that they, the "peasants" have no way to call for change even though voicing discontent in feedback sometimes works and is at least worth a try.

<comment><USER27> I agree with everything, except with the part that tech people appreciate and demand non-nonsense software.

A lot of people in tech enjoy being the "ultimate power user" and are proud of using things that look or feel complicated.

You just have to look at HN itself to read trough rationalisations of why software is bloated and why bigger software is probably better. It's never driven by data or by facts, it's purely by anecdotal experience.

<comment><USER28> We, collectively as the software industry, have the power to fix this. The fact that awful software still exists demonstrates we don't care or want to fix it. It's not a popular opinion here, but software developers/designers are responsible for bad software.

</comment></comment></comment></comment><comment><USER29> My new favorite default behavior annoyance was FF for Android deciding that everytime I clicked an open tab after some short time interval, it would reload the entire page making it extra annoying when you have the massive HN who is hiring thread open and want to scroll through it while on a subway with no connection.

<comment><USER30> Mobile OS periodically have to reclaim allocated memory and this happens.

</comment></comment><comment><USER31> [https://grumpy.website/post/0SzBFLZy9](https://grumpy.website/post/0SzBFLZy9)

Not being able to open all drawers at once is actually a safety feature preventing it from tipping over (see Ikea MALM fiasco). I suppose the UX could still be better.

<comment><USER5> [https://www.ikea.com/us/en/customer-service/product- support/...](https://www.ikea.com/us/en/customer-service/product- support/recalls/following-an-additional-child-fatality-ikea- recalls-29-million-malm-and-other-models-of-chests-pub4128a7af)

</comment></comment><comment><USER32> I noticed that this site is missing the modern classic "giant modal covering the entire screen". Usually reserved for "log in pls", "subscribe to site/mailing list", etc. And then sometimes the modal doesn't render correctly and instead I just see the whole screen go white/gray. News sites, reddit, twitter, facebook, and tons of blogs are guilty of this.

</comment><comment><USER33> The one about complaining about git status "Your branch is up to date with origin/master" Not hitting the network just belies a complete misunderstanding of both the message and distributed versioning systems in general. This message is telling you that your branch is up to date with your LOCAL tracking branch and is absolutely accurate and helpful in this regard. If you fetch and pull new diffs then it will tell you a different story. I wish people would stop complaining about git and just read the very easy to understand git book [https://git-scm.com/book/en/v2](https://git-scm.com/book/en/v2) Learn your tools people!

<comment><USER34> The point is that this behavior is confusing, not that the behavior isn't written down somewhere in some documentation.

<comment><USER33> My point is the message is not at all confusing if have a fundamental understating of a distributed system as opposed to a centralized one. Just because most people want to use git just like you would have with SVN doesn't mean it isn't a completely different architecture that is (admittedly) a bit difficult to grok, but far more flexible and powerful

<comment><USER35> It is confusing, as evidenced by people frequently being confused by it.

<comment><USER33> Learn. Your. Tools.

<comment><USER34> Rationalize. Away. Any. Criticism.

</comment></comment></comment></comment></comment></comment><comment><USER36> What are these "Hyperlink" that links to a page that do not contain anything more to read. This makes me grumpy ;-)

<comment><USER27> They're permalinks so you can share the posts with friends and family :)

<comment><AUTHOR> That is unacceptable, URLs and copy/paste are unnecessarily complicated. Better make those social media sharing buttons instead. ;)

<comment><USER37> Pro tips:

\- on a phone, long press on that hyperlink. Bingo, you've got your sharing screen to the post

\- on a PC: right-click, copy. Bingo, you've got your shareable link to the post

(Yes, I realise your post is /s :) )

<comment><AUTHOR> But long-press and right click are too complicated! /s

</comment></comment></comment></comment></comment><comment><USER38> Hehe at our new office in the kitchen there's a cupboard above the sink so that any person going to the sink will not be able to see what they're doing cause the cupboard is in the way unless you double yourself over so that you can stick your head under the cupboard or are a maximum of 140çm tall. Maybe the design was done by a person who has 2m long arms so that he/she can operate the sink from afar and thus avoid this problem.

</comment><comment><USER39> Hm I was doing something similar, but in the end there was just too much stuff that's broken, all the time: [https://twitter.com/everythingisbr2](https://twitter.com/everythingisbr2)

In the same vein, I've been thinking of having some universal bug/annoyance reporting tool where you'd tag each annoyance with the software/hardware/website and optionally its version.

<comment><USER40> Assuming you collected those yourself and events happening close to upload. If you have such a high amount of breakages, it means an average user is experiencing _several breakages per day_. Comparing with my own anecdotal experience I'm not surprised but seeing it all in one place just makes one depressed.

</comment></comment><comment><USER41> Title edit suggestion: the authors often cover non-web related technology, and even everyday objects like microwave ovens or furniture.

<comment><AUTHOR> Done!

</comment></comment><comment><USER41> There's also a relatively new sister project [https://annoying.technology/](https://annoying.technology/)

</comment><comment><USER42> This reminds me of the classic 'this is broken', which has now been moved to:

[http://goodexperience.com/tib/](http://goodexperience.com/tib/)

Some real gems in there.

</comment><comment><USER43> Wonderful website! Can't wait to regularly follow this.

<comment><USER44> That was my first thought. My second thought is this can't be good for my blood pressure.

</comment></comment><comment><USER4> In the spiwit (just watched Life of Brian) of grumpy Christmas, what's with a grumpy _single page, lazy loading_ blog? Paginate the thing.

Happy holidays.

<comment><USER45> I would, happily, if you could explain me why?

</comment></comment><comment><USER46> If you're paying attention and not too desensitized, you can usually find at least one software bug per day (often a new one).

</comment><comment><USER47> `user-select: none` - From MDN docs: CSS property controls whether the user can select text.

When the developer uses that on the body tag...

</comment><comment><USER48> I like the copyright pun at the bottom.

<comment><USER49> I can't see it because of infinite scrolling, which is pretty ironic given the website's subject.

<comment><USER41> Simply click the "refresh" button in the header to easily access the footer.

<comment><AUTHOR> Is there an easy way out of it after? The first time I used that button I had to turn off my Wi-Fi so I could down to the "top" and click the button again without infinite scroll kicking in.

<comment><USER41> Simply restart your computer and clear cookies.

</comment></comment></comment></comment></comment><comment><USER50> To the Google drive one that storage also includes Gmail and some other Google services too.

</comment><comment><USER51> Get off my lawn!

</comment><comment><USER52> those are mostly minor annoyances that can be avoided by switching to a different product... What is wrong with tech is that proper regulations to protect customers have mainly not been written yet or are outdated (mainly for privacy issues).

Why don't you suggest alternate products that don't have these problems?

<comment><USER53> A drawer system is a victim of inadequate regulations?

<comment><USER52> no, we are victims of how Google and other companies are allowed to design their systems because the regulations are nonexistent or outdated.... forget about the drawer this is just stupid to think that it is a big problem.

</comment></comment></comment><comment><USER53> Claiming Reminders can be written in 600 lines of Ruby is pretty ignorant. That reminds me of those posts that claim they could have written <some famous app> in a single weekend. Reminders is far more powerful than a school Todo app programming assignment.</comment>


Do you want to develop digital services in Finland? - <AUTHOR> https://forgeservicelab.fi/en#block-views-what-why-blocks-block

<comment><AUTHOR> FORGE Service Lab is a community where your development project can get cloud computing resources, help and other things to speed up the development.<p>Or, if your company has something to offer, like testing services or something else. You can offer those to other FORGE users</comment>


Ask HN: I want to take a vacation and code; where should I go? - <AUTHOR>

I'm a hobbyist programmer who only gets to code mornings, evenings and weekends. I need a break from work - and the Pacific Northwest - so want to get out of town and spend it working on my current project. I'm targeting late Feb/March 2014. These are my requirements:
- Within the western hemisphere, between Hawaii and the Caribbean, inclusive. Probably no further south than Central America (I'll be traveling from the Pacific Northwest)
- Sun, temp in the range of 75-85 F (23-30 C) and near natural water (not just a pool)
- No family resorts
- Some low impact activities for when I want a break from coding
- Fairly low-key vibe; I don't want my attention competing w/ drunk college kids
- WiFi outdoors (I recognize this is a nice-to-have)
The few people people I've put this question to (non-programmers) have responded w/ raised eyebrows, wondering why I'd want to take a vacation to code. I can't be the first person who wants to do this.
Anyone have any recommendations?
Thanks in advance. <comment><USER2> I left Paris for the Caribbean almost a year ago. First I've been in Martinique, and now I'm in Guadeloupe. Martinique has been an ideal place to work on my side project.

If you don't speak french, I would recommend Saint-Lucia :

-the people are kind and welcoming

-life is cheap

-you can rent some remote house on a hill where no one will bother you.

-activities are within the usual island-vacation-pack : beach sports, volcano trek, museum, golf

<comment><USER3> How are the hurricanes?

<comment><USER4> Hurricane season in Florida/Caribbean is from June 1 to November 30.

</comment></comment></comment><comment><USER5> Not entirely what you're looking for, but I just finished hitchhiking and urban camping up the Arab peninsula, and Oman is pretty great for coding. Lots of cool and easy to access beaches, very friendly people, modern infrastructure, and I never encountered another Westerner, tourist or otherwise. There are resorts, but they're easy to avoid. Wifi is difficult to come by, but 4g hotspots are cheap and easy.

The UAE (Dubai in particular) is all kinds of awful, though.

<comment><USER6> I was born and brought up in Oman. I don't think its especially cheap to live there, and the internet connectivity isn't great. You do have 3G, but it is quite pricey.

<comment><USER5> I can't comment on total cost of living as I was camping and hitching rides for free, but (restaurant) food and random consumer items seemed noticeably cheaper than their bay area equivalents. 3g/4g data in particular was _way_ cheaper than the US - I walked up to a random telecom booth in the airport and prepaid for far more data than I needed with whatever I had in my pocket at the time...under $20? The equivalent service in the US costs around $80/month via ATT, I think.

<comment><USER6> I was comparing the 3G prices to India and other Asian countries. I guess US has some of the highest prices around.

</comment></comment></comment></comment><comment><USER7> A cruise might be your best bet! Read this: [http://tynan.com/cruisework](http://tynan.com/cruisework)

FYI, wifi is now a bit cheaper on some cruise lines. Last time I was on Royal Caribbeans they had an unlimited wifi package that wasn't so outrageous.

<comment><USER8> Agreed - the cruise idea is fantastic. I went on a cruise and only had programming as a secondary objective (secondary to hanging out with my girlfriend), and even when it wasn't top priority it was still much easier to get it done there.

Cruise ships have a lot of relatively quiet nooks where you can camp out somewhere with a beautiful view of the ocean and just code all day if you want. If you get a balcony room, you could just park yourself there the whole damned trip and take up their free room service for food (just remember to tip them very well at the end if you do this!).

</comment></comment><comment><USER9> Hawaii. Rent a condo/house on VRBO, that way you are not stuck at a resort with tourists. Typically can find these near resorts, so you are close enough to enjoy them when you want to close the laptop. I've had some good weeks of hacking between snorkeling and sunning on Maui, renting a condo on a golf course near the resorts north of Lahaina. Kauai and the Big Island have similarly nice places with reasonable housing and food, with the ocean nearby. Avoid the resorty areas of Oahu.

As for outdoor wifi, often just making sure I'm in a spot with LTE connectivity is sufficient to live with tethering to a phone or my ipad to do basic net things like access repos, web pages, and occasionally download packages.

</comment><comment><USER10> I've thought about a vacation to code, but the only place I want to go on vacation to is Las Vegas.

Aside from the natural water part (it is in the middle of a desert) i think sitting by a pool would make a great place to work. Most (all?) of the pools have outdoor music, wifi, people watching, etc... Mandalay Bay has sand but they've since started a Day club which I don't think would be very good for the life span of your laptop... If you got a cabanna, it wouldn't be so dangerous.

I have worked in the lobby of a vegas hotel in a pinch so it's not bad...

The only problem with my plan, its not exactly cheap to stay at a vegas hotel...

<comment><USER11> I imagine Vegas hotels are _much_ cheaper than any other sunny/beachy locale in the US because of the casino industry.

</comment></comment><comment><USER12> This is a tough question... so many possibilities.

First thing that comes to mind is the Cadillac Hotel in Venice Beach, CA. Fun, sunny, laid-back area, easy to get to, lots to do. Close to the Getty. Not too expensive. Biking/rollerblading at sunset is wonderful.

[http://www.thecadillachotel.com/](http://www.thecadillachotel.com/)

The Santa Monica library is also a great resource and alternate place to work if you are tired of coffee shops and hotel wifi.

<comment><USER13> Thanks for this recommendation. I was looking at Los Angeles to do something like this because I wanted to see some stand-up comics on the side at places like the Laugh Factory.

</comment></comment><comment><USER3> Pick up surfing in Malaysia or Philippines. Super cheap living.

</comment><comment><USER14> San Jose, PR. It's a common wealth, so travel there and back is easy. They use the US Dollar. I was there earlier this month, and the connectivity was really good compared to most places I visited in Costa Rica. Also, no roaming charges for most US cell carriers.

Weather was great, nice beaches, fairly nice people (Costa Ricans are nicer).

<comment><USER3> This huge Russian IT manager I know (6'3 / 250 lbs) was robbed at knife point in the doorway of his hotel in San Jose, CR in 2007. When he went to ask a taxi driver what to do, the driver handed him a knife and suggested he get his own wallet back. He did so and recovered his wallet, as the robbers failed to leave the scene. Street justice FTW.

</comment></comment><comment><USER15> Costa Rica might be the place. Find a reasonably well-developed town on the beach that isn't too touristy and I think that will fit most of those criteria.

</comment><comment><USER16> San Diego

</comment><comment><USER17> thailand</comment>


Show HN: Recipelab, a simple nutrition calculator / recipe analyzer - <AUTHOR> https://recipelab.org/ <comment><USER3> I like the concept and the execution is pretty solid - I particularly like the RDA % and bar charts for a meal. I have a few thoughts though:

1) Search found all the ingredients I needed or pretty close approximations which was cool. Multiple units of measurement was an excellent choice. Food data looks USA - it would be helpful to know that as a UK user.

2) I had to make a username and password which will put off some potential users. Why not make it log in with Facebook/Google as an option?

3) I entered a quantity of tortillas and couldn't amend it without deleting the item. Was I doing something wrong?

4) As an active 6'2" man I suspect I need more calories than average. How about activity level/height/weight options? Kudos for having 'pregnant' and 'Breastfeeding' as options. However they would be more powerful as flags - <USER2> wife is both pregnant and breastfeeding! Or even 'add person' so one can look at nutritional info for cooking for a family.

I'm interested to see how this tool evolves! Good work!

<comment><AUTHOR> Thanks!

1) Yes, it uses a USA gov' database. But I'm curious about why it looks American to you. Could you please elaborate?

2) To be frank, this is django's registration system, off-the-shelf (least effort). If I see enough interest in the site I will add simplified login. Thanks for highlighting it.

3) You did nothing wrong. I've stripped the interface down to the most basic functions.

4) You're totally correct. Adjusting the targets for body shape and activity is a priority item. As far as pregnant + breastfeeding, the source database has data for each but not both.

</comment></comment><comment><USER4> Is there a comparison to MyFitnessPal? I use that and it's able to import recipes from a link, or scan products using a barcode. How are you getting all of the data for different products? I tried looking up my two favorite brands of yogurt and they weren't there.

<comment><AUTHOR> Thanks for checking it out. In my understanding, MyFitnessPal is oriented toward the fitness crowd, with a focus on daily energy consumption.

After a lot of exploration I decided to limit the scope as much as possible and target home cooks and food bloggers, with a focus on recipes.

I spent a few weeks of my spare time trying to get the parsing of free-text recipes to work (essentially MFP's import from link). The results were decent, but I realized that if it's only 90% accurate I still need a lot of user interface functions (like MFP) to let users edit/fix the inaccurate bits. It started smelling bad so I decided to strip it down to its most basic function.

I have no plans for now to use databases of branded products - the ones I saw were quite expensive and would be more useful for sites like MyFitnessPal, perhaps less useful for recipes/cooking.

<comment><USER5> > I spent a few weeks of my spare time trying to get the parsing of free-text > recipes to work (essentially MFP's import from link). The results were > decent, but I realized that if it's only 90% accurate I still need a lot of > user interface functions (like MFP) to let users edit/fix the inaccurate > bits. It started smelling bad so I decided to strip it down to its most > basic function.

As a regular MyFitnessPal user I have to say, their recipe importer leaves a lot to be desired. Whenever I use it, I inevitably have to make corrections because it mapped the parsed ingredient to the wrong item. (And their UI for making corrections is not particularly great either.)

It's probably a good idea that you decided not to go down this road.

</comment></comment></comment><comment><USER6> If I have to sign up to try it, I will not try it. I'm sure I'm not the only one.

Yes I could sign up with an anonymous email address over a tor connection.

but I would resent that I was forced to do it

<comment><USER7> This got me as well. At least have a more detailed "how it works" type page. Maybe even a sample working page to try out the features.

Someone even commented above about using a google/twitter/facebook integration. I'd be more willing to do it from there.

<comment><AUTHOR> I've changed it so that you can try the calculator without creating an account. Login is only required for saving recipes or for changing the nutrition targets.

</comment></comment></comment><comment><USER8> I'm pretty familiar with the USDA SR dataset, and here are my thoughts (examining data with one of my experiments diarytail.com)

1) People in the enterprise space will be willing to deal with the weird nuances of the USDA-SR data. For example writing 'beef cooked' vs' 'beef raw'.

2) In the general consumer space, there is a huge problem in getting the general public to find ingredients they use. I found lots of people typing in brand name ingredients not matching what is in the USDA data set.

To me, the problem is partially solved with My Fitness Pal's huge dataset of user generated data. However the flip side is it is missing most of the micros as they are entered via a nutrition label.

Another solution might be to reference nutritionix.com 's dataset (I think the joyapp.com uses it also)

<comment><AUTHOR> Very nice site (diarytail.com), by the way.

Unfortunately nutritionix has only basic nutritional data as well, the sort of thing you find on a product label. Not bad but not great. Have a look at their sample data. Plus it's around 10k/year or 50k/license.

I'm thinking home cooks would get a lot of value from the tool, and reasonably good data even if they use raw instead of cooked (as an example).

</comment></comment><comment><USER9> Interesting, is there any plans on opening up your calculator/analyzer to an API. I just launched [http://imadefood.com](http://imadefood.com) a week ago, and there could be some mutual beneficial scenarios here. Let me know!

<comment><AUTHOR> Nice site!

I don't think I can offer much of an API. As I mentioned in another comment, I gave up on trying to parse a free-text recipe. I avoid a lot of headache by directing users to enter ingredients from the database list only.

That being said, I'm open to talk about it - see my HN profile for email.

<comment><USER10> I like your imadefood site. Reminds me of a kind of jsfiddle for recipies.

edit: @phugold - I like the idea of your site, but you might want to take note of the ability to immediately see recipies without having to register. It's great to be able to just dive right in rather than hitting a wall requiring login.

<comment><USER9> Thank you! Someone also mentioned "github for recipes" as well. A bit more simplistic than that, but I'll take it! :)

</comment></comment></comment></comment><comment><USER11> Very cool. I wonder does this take cooking into account? Meaning you slice up a potato and fry it you end up with different nutritional results than baking a whole potato for example.

<comment><AUTHOR> For many ingredients, you have the option of selecting raw or cooked. For popular items like potatoes there are dozens of options including fried, baked, salted, etc.

Try entering "potato fried" in the search box.

</comment></comment><comment><USER12> I would have signed up and tried the service, but I'm allergic to quotes like the following in a privacy policy, particularly when they are not disclosed on the sign up page.

 _If you are a registered user of recipelab.org and have supplied your email address, recipelab.org may occasionally send you an email to tell you about new features, solicit your feedback, or just keep you up to date with what's going on with recipelab.org and our products. If you send us a request (for example via a support email or via one of our feedback mechanisms), we reserve the right to publish it in order to help us clarify or respond to your request or to help us support other users._

Contrast that with:

 _When you register for recipelab.org, we collect certain personal information (ie - your email address and name). We will only use this email address to send you {monthly||weekly||daily} requests for feedback or updates on the product._

 _If you send us a support request, we may publish it to help us support other users. However, before we publish it, we will strip out any personally identifiable information._

<comment><USER13> > we reserve the right to publish it in order to help us clarify or respond to > your request or to help us support other users.

That rubs me a little the wrong way too.

</comment></comment><comment><USER14> Sounds similar to what Wolfram Alpha can do:

[http://www.wolframalpha.com/input/?i=2+slices+of+bread+%2B+1...](http://www.wolframalpha.com/input/?i=2+slices+of+bread+%2B+1+slice+swiss+cheese+%2B+1%2F2+avocado+%2B+1+slice+tomato&lk=3)

</comment><comment><USER15> "Substitutions" is misspelled on the front page.

<comment><AUTHOR> Thanks, fixed.

</comment></comment><comment><USER16> This exists already in a pretty intelligent way, check out [http://www.caloriecount.com/cc/recipe_analysis.php](http://www.caloriecount.com/cc/recipe_analysis.php) and paste in an ingredients list.

<comment><AUTHOR> Yes, that site does a pretty good job. At first glance the copy/paste approach is promising - in practice it's a bit more hassle and less precise than what recipelab does in my opinion, plus I'm starting from a clutter-free UI that works great on mobile devices.

</comment></comment><comment><USER17> I'd really prefer seeing a full example before signing up. You're not providing enough information about what I'll get out of this save a few tiny cropped screenshots.

<comment><AUTHOR> Do you think a full screenshot might be interesting?

But a simple screenshot image would not work for all cases, because the calculator page renders differently on mobile devices. On the other hand, if I offer a full working version without sign-up, there would be no reason to sign up. If I show a "frozen" version of the full calculator, normal users might get the wrong idea from that as well.

<comment><USER18> No, a screenshot would not be interesting enough IMHO.

It seems that you want to prevent people from experiencing your site without signing up first. Why is this? If you hide the value of your site behind the sign-up link, hardly anybody will bother signing up.

With a site like this, I would suggest enabling all features for anonymous users, except the ability to save recipes. Users can play around with the site and make their own recipe. The natural step after creating a test recipe would be to save it - this would be the best place to prompt for signup. I'd also recommend changing your on-boarding flow so that I do not need to confirm my email address immediately.

<comment><AUTHOR> I've changed it so that you can try the calculator without creating an account. Login is only required for saving recipes or for changing the nutrition targets. Thanks again for the advice.</comment></comment></comment></comment>


Malware/Rootkit Survives Disk Wipes and Hijacks Any New OS Installs - <AUTHOR> https://forums.malwarebytes.com/topic/213254-malwarerootkit-survives-disk-wipes-and-hijacks-any-new-os-installs/ <comment><USER2> after building a new pc (all new hardware) after i installed it and only updated windows, my boot menu got corrupted entries in there with chinese signs. flashing BIOS just made that broken entry a empty string, still couldn't trust anything in my computer. had to buy all new hardware because the vendor didn't acknowledge it was broken. still not sure if it was supply chain compromise or someone MITMed my windows updates (???)

there's some crazy low level malware out there just running free. there are litteraly 0 products who can protect you from that. even thngs like qubes os etc. don't offer any protections from such a compormise, thought they might make the initial vector a tiny bit more compicated.

<comment><AUTHOR> I've seen things like this. And I've tried all known virus scanners and protection tools. None work. You're not crazy. Perhaps the solution is older hardware (EBAY), Non x86 SBCs running Linux [https://en.wikipedia.org/wiki/Comparison_of_single- board_com...](https://en.wikipedia.org/wiki/Comparison_of_single- board_computers), a Purism Laptop: [https://puri.sm/](https://puri.sm/), and/or hardware which is so cheap such that you don't care if it gets bricked, i.e., spending $50 for a 10-pack of Raspberry Pi Zero's ($5 each) - [https://www.raspberrypi.org/blog/raspberry-pi- zero/](https://www.raspberrypi.org/blog/raspberry-pi-zero/) Welcome to 2018.</comment></comment>


Ask HN: CSS gradient hacks - are they worth it? - <AUTHOR>

I came across this site the other day, and really liked the results:
http://www.designdetector.com/demos/css-gradients-demo-3.php
I was wondering if any designers out there could shed some light on whether or not this has any advantages over images. <comment><USER2> No. This is a neat hack but is so narrowly applicable it's not worth bothering about for most folks.

You ask designers for input but I think it's an engineering choice to use simple and straightforward technologies you (and colleagues) fully understand rather than meddling with neat hacks, particularly if you're deploying out to the user browsers, which is a quagmire even if you're not getting too fancy.

Of course if you're in an environment where images are impossible/banned for whatever reason (and you can't use proper CSS gradients) then this technique could make you look like a magician, but I can't think of such a situation off-hand.

</comment><comment><USER3> This website: <http://dostuffright.com/> does something similar with the drop shadow. Kinda neat.

</comment><comment><USER4> The subtle gradients on mediatemple.com are good example of gradients done right.</comment>


Why you can't parse HTML with regex - <AUTHOR> http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags

<comment><USER2> > _HTML is not a regular language and hence cannot be parsed by regular > expressions._

But, but! In StackOverflow, "regex" doesn't mean "specification for a finite state automaton which recognizes whether strings belong to a given regular language", but rather "a character matching tool implemented in whatever language or its library that a question is about".

Some so-called regex engines will match nested tags.

The question has a problem because it makes no mention of what concrete tools OP is using to solve a concrete problem.

</comment><comment><USER3> The second answer is correct, not the top answer. If you have some knowledge about the form of your HTML, then regex might very well be the best tool for the job.

I've spend hours trying to get something to work in BeautifulSoup that could have easily been done in minutes with regex and a few lines of python.</comment>


Ask HN/PG: Leaderboard acting strange? - <AUTHOR>

This morning I was about 100 short of the leaderboard and made two submissions that got me about 60 points. A minute ago I was 95th and the 100th spot was at 3900 but now I'm 98th with the last spot at 4072. Any idea what's up with the fluctuating numbers? <comment><USER2> I just restarted the server. Users are lazily loaded, so the leaderboard will look odd for a brief period after a restart.

<comment><USER3> The 'submitted' link on the profile basically doesn't work <http://news.ycombinator.com/submitted?id=pg> <http://news.ycombinator.com/submitted?id=ivankirigin>

I suppose it is related to the lazy loading?

<comment><USER2> Yes, items are lazily loaded too. Responses to http requests get killed after 30 sec, so if the server has to load too much stuff to satisfy a request, it will die first.

It's always been this way. After the server has been running for a bit it stops being a problem.

<comment><USER3> Perhaps you should hide features that are likely going to be unavailable if the server recently restarted.

No need to send users down a rabbit hole, and certainly no need to waste the cycles on your machine.

<comment><USER4> Or limit it to far fewer results. I click the submissions link pretty frequently from my own profile as an easy way to check up on conversations on threads I've started. Now I feel kind of bad for doing it that way.

</comment></comment></comment></comment></comment><comment><USER5> I'd be interested to see what fraction of the leaderboard users' karma comes from comments vs. stories.

</comment><comment><USER6> And you care bacause?

<comment><AUTHOR> Pure curiosity of how it works. That's all. I don't really care either way where I rank.

<comment><USER6> I believe you...</comment></comment></comment>


How Lugg (YC S15) Got Its On‑Demand Moving App Off the Ground - <AUTHOR> http://themacro.com/articles/2015/12/interview-with-lugg-founders/ <comment><USER2> Nice work on getting the first couple of users, you did exactly what you were supposed to. Amazes me how many products try and launch without getting any feedback or users.

</comment><comment><USER3> So "How Lugg Got Its On‑Demand Moving App Off the Ground"?

> We told them, "Oh we'd like to do YC, but we got rejected." It turns out > that they were friends of friends with Sam Altman. They made an > introduction, and he encouraged us to apply again.

<comment><USER4> It's been said that luck is the meeting of preparation and opportunity. Sure, you can point at that piece of luck; or learn from the hustle and preparation that led to the opportunity being created.

<comment><USER5> "Opportunities multiply as they are seized." \- Sun Tzu</comment></comment></comment>


Designing Software for Ease of Extension and Contraction (1979) [pdf] - <AUTHOR> https://courses.cs.washington.edu/courses/cse503/08wi/parnas-1979.pdf <comment><AUTHOR> @kqr mentioned this paper several times in the discussion at [0]. It was not a paper I was familiar with so I sought it out and have shared the link here.

[0] [https://news.ycombinator.com/item?id=23914486](https://news.ycombinator.com/item?id=23914486)

</comment><comment><USER2> The contraction bit is sadly often missed when discussing this. But it's just as important!

<comment><AUTHOR> Interestingly, to me, good modular design creates both ease of extension _and_ of contraction. In some of my most successful programs (read: met user requirements over an extended period of time with relatively straightforward modifications) it was the modular design that permitted the extensions. But the same structure made it feasible to easily remove old features or migrate them to new programs based on user needs. Maybe the main program was a bit of a Swiss army knife, but we had a customer who only needed a few features. The design made it possible to extract a new program from the old one primarily through selecting which parts of the core (lower layers in this paper) were needed and wrapping them in a new (or simplified) GUI.

But maybe it's possible to do seemingly good modular design which favors extensibility, but fails at contraction/extraction. I had some experiences in my formative (college) years that lead me to want to write things the way I did for those successful programs. This is certainly something that requires exercising. If it weren't being exercised, maybe we wouldn't have been so successful. As in, if I tried to make a good modular design and only 5-10 years later we contracted the program or extracted new programs from it, maybe it would have grown enough cruft to make that a challenge, but extension would have still seemed easy.

As I think about it more, I'm leaning towards the issue being exercising the option of contracting the program. If you only ever extend, it becomes increasingly easy to blur the lines between the layers of the system.</comment></comment>