How does the Gmail unsubscribe button work? - <AUTHOR> https://blog.leavemealone.app/how-does-the-gmail-unsubscribe-button-work/ <comment><USER3> I want a button that says "ghost this mailing list" when I click it, their mail server gets a standard message like "user no longer exists at this address", or "user reached quota", whatever makes me get pruned from their database instead of having my email address validated and the "last_seen" column updated.

<comment><USER4> It unfortunately would probably only do the opposite as this validation occurs during the SMTP transaction when the message is delivered to the server.

Going back after and saying you don't exist is like answering the phone and going "nobody is home".

Edit: I suppose this ghost setting could be used for future delivery attempts though. Perhaps this is what you meant originally.

<comment><USER5> I work for an email service provider. While we usually get a response from the inbox provider that an inbox does not exist, we totally get async bounces all the time. Some providers accept the mail and realize later that they cannot deliver it.

<comment><USER6> Can confirm. Async bounces make up somewhere around a few percent of overall bounces for most senders.

</comment></comment></comment></comment><comment><USER7> I love the idea of the Gmail unsubscribe button, but unfortunately I can't bring myself to use it.

The issue is that there are good-faith and bad-faith unsubscribe links. Clicking the unsubscribe button can thus either have a good outcome (less junk mail) or a bad outcome I ardently want to avoid (letting a spammer know my address is active).

I'm sure Google knows this and does some verification and detection to try to prevent that bad outcome, but as an end user, I don't have much visibility into how well that works. It's a hard problem, but Google is smart, so it's _possible_ they've solved it, but I don't really know whether they _actually_ have.

So in practice, I always read over the email in question carefully to try to judge for myself whether it's safe to click the unsubscribe link at the bottom. It's annoying, but the effort seems worth it.

<comment><USER8> PSA that there was a sort of "email 2.0" spec called "Internet Mail 2000" (which gives you an idea of how long ago this was, heh) by djb, that would have partly eliminated all this crap. The idea is that you can pay the cost of read receipts (which are kind of a superset of what you are concerned about) to structurally disadvantage spamming so much (by forcing it to tether itself to DNS) that it ceases to be a viable marketing model; spam that is big enough to generate revenue is also either big enough to be caught or spread out enough among new domain registrations that the cost easily swallows the revenue. The struggle is that nobody likes read receipts, so one is stuck trying to define some sort of "halfway between" system to try and invalidate the read receipts, "sometimes you have to store the message until the person wants to read it, but sometimes Gmail will download it before the person reads it, so this signal is unreliable for whether it was actually read."

<comment><USER9> > there was a sort of "email 2.0" spec called "Internet Mail 2000" by djb, > that would have partly eliminated all this crap

Sounds like they were advocating a solution to spam [1]. I wonder why it didn't work...

[1] [https://craphound.com/spamsolutions.txt](https://craphound.com/spamsolutions.txt)

<comment><USER8> I mean in his defense, DJB has some serious chops that others have lacked, including writing what was at one point the most popular DNS server for anyone who cared about security as well as the first MTA which cared about security while transmitting email, and now two of the more popular stream ciphers, one of which underlies the current fastest secure hash function.

IM2000 probably would have succeeded if it had gotten the attention from him to go past a random idea into a well-specified protocol with a canonical implementation. Standards work is hard!

</comment></comment></comment></comment><comment><USER10> I follow these rules:

1) if I never signed up goes immediately to SPAM

2) if I did signed up I make the effort of going through their unsubscribe procedure

3) if I still get emails after (2) goes to SPAM

<comment><USER11> I follow a similar procedure, with one more step:

1) If I never signed up goes immediately to SPAM.

2) If I did signed up I make the effort of going through their unsubscribe procedure.

3.1) If I still get e-mails after (2), I file a request for my personal data under the GDPR (EU citizen here).

3.2) Once I got that, I use the GDPR to delete all of the data associated with my account / e-mail address.

4) If I still get e-mails after (3), it goes to SPAM.

With step 3, I hope that I can make them notice their bad behaviour. My goal is to drive up the costs of that behaviour (so they get incentivised to change it). Also, I'm generally interested in the personal data that a service has associated with me.

<comment><USER12> Nearly any business that gets more than a handful of GDPR requests has fully automated it.

It costs them nothing to process your request - you're wasting far more of your time crafting the request than of theirs.

<comment><USER13> You might be surprised. I worked for an organisation that got ~1000 requests a year up until recently, each request involved going into every system manually, taking screenshots, tagging files etc. Quite often a good few hours per request and on a few memorable occasions, several days work for a single request. It definitely does cost many larger companies, but to varying degrees.

</comment></comment></comment></comment><comment><USER14> If I get spam mails on lists I never signed up for, I either hunt down the X-Abuse header and report there (if they use a reputable bulk mailing service), otherwise I just paste the entire email on members.spamcop.net

<comment><USER15> > _if they use a reputable bulk mailing service_

If they use a reputable bulk mailing service instead of using their first- party domain then they are indistinguishable from a phishing attack.

<comment><USER16> > If they use a reputable bulk mailing service instead of using their first- > party domain then they are indistinguishable from a phishing attack.

With most bulk mailing services, the message will come from the "first-party domain". They will have configured that service as a legitimate sender for the domain via SPF/DKIM DNS records.

<comment><USER15> > _With most bulk mailing services, the message will come from the "first- > party domain". They will have configured that service as a legitimate sender > for the domain via SPF/DKIM DNS records._

It's not just the from:firstname.lastname@example.com that I'm talking about. If the unsubscribe link does not _also_ go to firstparty.com, then it's still indistinguishable from phishing.

</comment></comment></comment></comment><comment><USER17> It's really amusing to me that the top comments on this posts seem to be HN users complaining about spam and if this button is useful or not.

I had to dive into this a bit for something and work and it's just fascinating how much effort has been spent in trying to combat spam, build a reputation based system for emails etc. And this article does an amazing job of explaining list-unsubscribe...although the RFC is pretty easy to read too!

</comment><comment><USER18> My naive suggestion. "Hard" unsubscribe button that tosses all future emails from this list in the trash and mails the sender a note that the address has unsubscribed from this list and all future emails will be ignored.

Gmail has over a billion active users. Mailing lists will probably adapt to whatever crumbs Google leaves on the doorstep.

<comment><USER19> Isn't that exactly the "mark as spam" button?

<comment><USER18> Mark as spam doesn't notify the sender

</comment></comment></comment><comment><USER20> Pet peeve: unsubscribe links that take me to a page asking for my personal information (usually my email address). I'll usually close those pages and report the email as spam.

If you know my email address, then put a token in the unsubscribe link so you can retrieve my address on your end, rather than making me retype it. If you don't know my email address -- maybe you are sending to a list, not to me -- then I consider you spam because you don't actually have the direct ability to remove me.

<comment><USER21> I'll add another pet peeve to the list: unsubscribe links that remove you from only 1 of N number of lists they have. You click unsubscribe but a few days later you're still getting mail because you got removed from "news" emails but this one is a "promotional" email.

<comment><USER22> I recently unsubscribed from all but one (the account tips) of the Firefox newsletters, because they were sending too much stuff I wasn't really interested in. The next day I get a new Email from them.

Subject: "Was it something we said? _crying emoji_ ". Body: "If you want to go... we won't stop you. [...]" Footer: "You're receiving this email because [...] subscribed to Firefox Account Tips.

Yeah, thanks for the ~~tip~~ spam.

Felt like going through one of those dark pattern flows that Spotify or Amazon have when you try to unsubscribe from their paid plans, trying to guilt you into reconsidering.

<comment><USER23> That's pretty ironic coming from a company whose main value proposition is "we aren't assholes like other tech companies are." That and the annoying crap you have to disable when you install Firefox to get a blank new tab page.

(Still better than Google, though.)

<comment><USER24> How can you get a blank page? What do I need to delete?

<comment><USER25> The simplest way is to click the gear on the new tab page and uncheck items you don't want. If you uncheck everything you will still have the gear icon present. If you want to hide that then your are in userChrome.css territory.

<comment><USER26> I just have Options -> Home -> New tabs set to "Blank Page" and that works just fine without any icons.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> Multiple times the gmail unsubscribe button has removed me as moderator from mailing lists I moderate!

I tend to use the checkboxes to mark groups of emails as spam, then also chosen "unsubscribe me" without checking where they came from (since I don't want to open them).

When it happens to be spam sent to a mailing list, this feature unsubscribes you from the mailing list. When it's a Google group you moderate, good bye moderator status! Oops! (Filed a bug internally about this, no status updates so far.)

<comment><USER28> I don't think that's a bug. It sounds like you are asking to be unsubscribed from a mailing list, and they're honoring that request. It's on you to make sure you're doing that for the proper emails.

> without checking where they came from

</comment></comment><comment><USER29> I'm always fearful of clicking unsubscribe. It's a way of telling the spammer "hey, this email is really used by a real human, spam the hell out of me"

<comment><USER30> Any remotely legitimate mailing list will respect unsubscribe requests, lest they run afoul of the CAN-SPAM act and/or start getting blacklisted.

Years ago I worked at a large email service provider for bulk mailings on behalf of large customers and we took unsubscribes very seriously.

And for the really truly spam/scam emails, the unsub link is the least of your concerns since delivery and tracking pixels confirm the address is real and being used. The true spam usually doesn't even have an unsub link. In those cases mark as spam and hope that your email provider starts flagging them as spam before it ever makes it to your inbox in the future.

I'm an aggressive unsubscriber and 99% of the time it works. Very little junk flows into my inbox these days.

<comment><USER31> > since delivery and tracking pixels confirm the address is real and being > used

Does it work in Gmail? Since it doesn't load images until I athorize: [https://i.imgur.com/RZ93VIU.png](https://i.imgur.com/RZ93VIU.png)

<comment><USER32> Unless you use an iPhone, then there is no way to shut off images in gmail. Seems a bit strange that they'd not have that feature, when gmail was the first big provider to disable images by default.

<comment><USER33> That is inaccurate. I have disabled image loading in Gmail and this is reflected in the web interface, the Gmail Android app, the Apple Mail app, and very likely in all other mail clients. Just to be clear, you _do not_ need an iPhone to shut off images in gmail.

<comment><USER32> I mean in the gmail app for iphone. Gmail images are off when I use a desktop, they're off when I use the andriod app, but I was surprised to see images in email when using the gmail iphone app.

Last time I went searching, I found google documentation that said there are no image options for the iphone app.

<comment><USER34> This used to be the case but was fixed earlier this year. You can now disable images in the Gmail iOS app: [https://support.google.com/mail/answer/145919?co=GENIE.Platf...](https://support.google.com/mail/answer/145919?co=GENIE.Platform%3DiOS&hl=en)

</comment></comment></comment></comment></comment></comment></comment><comment> t-writescode Friendly reminder that the CAN-SPAM Act is a thing in the United States.

[https://www.ftc.gov/tips-advice/business- center/guidance/can...](https://www.ftc.gov/tips-advice/business- center/guidance/can-spam-act-compliance-guide-business)

</comment><comment><USER35> The bit at the bottom about the unsubscribe button appearing or not might be based on sender reputation?

<comment><AUTHOR> Nice! Do you have a source for that?

<comment><USER36> Source: [https://gmail.googleblog.com/2009/07/unsubscribing-made- easy...](https://gmail.googleblog.com/2009/07/unsubscribing-made-easy.html)

</comment></comment></comment><comment><USER37> I don't know about Gmail, but I once accidentally hit unsubscribe on a Yandex mail account I use solely for getting mails from mailing lists and it unsubscribed me immediately, no questions asked, and without any way to undo. I tried removing the sender from the unsubscribed mailing lists, removing and adding the subscription on the mailing list side half a dozen times, but nothing worked.

</comment><comment><USER38> Unsubscribe is a confirmation to spammers that your email is indeed real.

<comment><USER39> Doesnt a lack of a bounce already confirm it's real?

<comment><USER38> Maybe, but you actively unsubscribing confirms it's an active email account.

</comment></comment></comment><comment><USER40> [http://help.mail-list.com/m/59114/l/558254-rfc-2369-list- uns...](http://help.mail-list.com/m/59114/l/558254-rfc-2369-list-unsubscribe- headers)

There's and RFC for List-Unsubscribe headers.

</comment><comment><USER41> Anyone know how the macOS / iOS unsubscribe button works?

<comment><AUTHOR> It's probably very much the same!

</comment></comment><comment><USER42> > If you use Gmail or frequently send mailing list emails

Now that's your problem right there. _Don't_ use Gmail. You're not just giving up your own privacy, you're hurting the privacy of everyone who corresponds with you. There are plenty of non-US free email providers, and many/most of them are at the very least much better than Google in this respect.

Also _use a mail client_, not your browser. Thunderbird, KMail, evolution - even (ugh) Outlook.

<comment><USER43> This comment isn't constructive or substantively related to the article it's on. But on the subject of privacy:

I'd rather have my data in the hands of Google -- a company with strong compliance and the world's best non-government infosec outfit -- than in the hands of any of the other companies listed.

<comment><USER42> "Strong compliance" \- to what? It strongly complies with the government's expectation of access to all of your data.

"best non-government infosec outfit" \- but that outfit is not securing your information _from_ Google, the US government or Google's business parties; it secures it from other individuals and unaffiliated organizations.

<comment><USER43> I'd rather have my data in Google's hands, where I basically know who can access it and when (US government with a warrant or NSL, Google employee with a specific business need, no foreign governments, no hackers) than in the hands of some company that for all I know exposes their production DB to the public internet with default username and password enabled.

I guess there's always E2E encryption, but I'd like to be able to recover my emails if I lose my password.

</comment></comment></comment></comment><comment><USER44> We've had "Add List-Unsubscribe Header" on Trello for a long time, but for some reason I had it in my mind that there was a ~60 character limit.

I appreciate the post because after revisiting it, I think that info was gathered from a few-years-old blog discussing a specific limit in (maybe?) Gmail, but it sounds like it can be broken down into multiple lines.

<comment><USER8> I don't think there are "multiple lines", just that you can have quite large HTTP headers and this one happens to contain a comma followed by a space, which the text view is using to word-wrap.

See e.g. [https://web.archive.org/web/20180605011201/https://www.list-...](https://web.archive.org/web/20180605011201/https://www.list- unsubscribe.com/technical/) for a better example.

</comment></comment><comment><USER45> I just mark as spam. Too many unsubscribe links want me to type in my email address, ain't nobody got time for that.

If it's a newsletter I actually signed up for, I respect that and will unsubscribe, but the majority is unsolicited spam where a company feels is OK because I happened to have bought a product they can now email me 8 times a day.

</comment><comment><USER46> Most of the time where this shows up I never subscribed in the first place.

Gmail needs to handle that in a much better way to ensure they end up in spam for everyone else.

I'm not hitting unsubscribe if I didn't subscribe in the first place because it sends a signal to the sender that I don't want them to get.

<comment><USER47> I thought gmail "mark as spam" could have a global effect if enough people used it for the same address.

</comment></comment><comment><USER48> The worst are emails that require a login or other input information on their unsubscribe page, or ones where the mobile unsubscribe page does not work. I report these as spam just so they are punished for the dark pattern, but I think these might also be violations of the CAN SPAM act.

</comment><comment> seph-reed So what's wrong with hitting spam?

<comment><USER49> I try to be reasonable here. If it's something from a business I transacted with in the past or recognize the name, I will unsubscribe. I don't consider it spam. And I don't want to hurt their reputation.

I do keep track of if I already unsubscribed from a related list. Sometimes "unsubscribe from all" is completely ignored. Which really angers me.

If it's a random, clearly bought newsletter list from a related list, it depends on my mood. Likely spam.

Other notorious example: business A founder also founds (unrelated) business B. They just email their entire A client base with zero association to A. Big peeve of mine.

\-- Edits (some more ramblings) --

My personal favorite: the "I want to receive marketing email" checkbox that rechecks if you have an unrelated issue with your transaction. Say, invalid CC details.

Still, even with these boxes, I think my standard is just: "I did business with them, I will get at least 1 marketing email. I'm ok with that. I will unsubscribe and not hear from them again." Anything past that is unacceptable.

To be clear: that's not how I think it should be. It's just how businesses, even small, genuine mom and pop shops, have been taught to operate. It's cultural. It reminds me a lot of tipping in the US. I'm vehemently anti- tipping "culture" because a standard 20% is the opposite of rewarding for performance. But I still tip at a baseline of 18%+.

It's too ingrained. And I'm not going to protest by not tipping and try to change it.

I think we've come too far unless changed by law or restaurant management. Same goes for marketing emails.

<comment><USER15> > _I try to be reasonable here._

I try to be a little more reasonable here. If it's a business that required me to sign up to do business with them and didn't allow me to opt out of their marketing emails then I have no problem whatsoever clicking the Spam button. And, if their marketing emails go to a third party domain -- such as a bulk emailer -- then it goes into the Phishing bucket _regardless_ of whether or not I opted out of their marketing emails.

<comment><USER50> So much of my "spam" is from services I definitely signed up for because I have a legit use for, Or product I'm glad to pay a fair price for, but they never even asked if I wanted to get emails from them during signup/checkout- the emails just start coming in.

What's the deal with this?

<comment><USER51> I suspect it's because it works for enough people that it pays off.

Every now and then I forget how annoying it was last time, and I think it would be nice to donate money to some sort of charity, and then they proceed to spam me for the following year. A couple years later I forget about he experience, and the cycle begins again.

<comment><USER52> I donated $20 to doctors without borders four years ago (a friend wanted that in lieu of bday presents). I've since gotten close to 50 letters from them and other charities. That cost far outweighs the $20 I gave them.

<comment><USER50> Was this part of that feature happening on Facebook where people can create "campaigns" or just an ad hoc request to donate?

<comment><USER52> She requested it on facebook via one of those things, but I donated directly on their website since..facebook.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER53> Outlook has this functionality. I don't know if it's using the unsubscribe link scrape method, or the List-Unsubscribe header method.

What it does is show an Unsubscribe button above the email, which loads the unsubscribe URL into a small web panel below it.

</comment><comment><USER54> Whenever I click "it's spam & unsubscribe" i often keep getting more emails.

Then I started clicking "block this address" from another drop-down - <USER2> got any new mail. Much recommended approach.

</comment><comment><USER55> is there a way to unsubscribe from junk mail in my physical mailbox owned by USPS ?

<comment><USER56> [https://dmachoice.thedma.org/](https://dmachoice.thedma.org/) but it costs a few dollars.

You can also contact the company directly and ask to be removed from their list.

</comment></comment><comment><USER57> Small editing issue:

I think the author missed adding a hyperlink in the summary at "You can check the source of an email to do this - here's a guide on how to do that. "

<comment><AUTHOR> Fixed. Thanks!

</comment></comment><comment><USER58> Am I the only person on the internet who doesn't get spam emails? Or just the loneliest?

<comment> t-writescode I work very hard to unsubscribe to everything I know about. In general, I have very low spam volumes.

Occasionally, I'll sign up for something new, and get spam and then I have to do it all over again.

</comment></comment><comment><USER59> here's the deal. if I give you an email for a transaction, that doesn't mean I want your future marketing. That is spam and that is how I mark it. If you want to keep sending me marketing, then ask permission.

</comment><comment><USER60> this seems to be better designed on outlook, i would just create a rule to instant delete, filter, this mails

we need Rules on gmail... filters do the labelin work but they dont actually move the mails

<comment><USER61> gmail filters allow various actions, including delete, archive, mark as read, forward, etc...

is that not what you meant?

</comment></comment><comment><USER62> Quick tip: it does not.

</comment><comment><USER63> It doesn't.

</comment><comment><USER64> TL/DR: It doesn't.

</comment><comment><USER65> At work we have about 200K mailing list we target after some segmentation is done,so it's about a few thousand people for any give campaign. We had some settings wrong,which meant thst no reply wasn't enabled.People would rather respond to an email and ask to unsubscribe or rant about it but rarely click unsubscribe button.

<comment><USER66> > People would rather respond to an email and ask to unsubscribe or rant about > it but rarely click unsubscribe button

It's a bit ironic to complain about getting unwanted emails from a bunch of users when they're literally just replies to unwanted emails that you sent them

<comment><USER65> I'm not complaining- it was merely an observation.</comment></comment></comment>


Amazon Ramps Up $13.9 Billion Warehouse Building Spree - <AUTHOR> http://www.bloomberg.com/news/2013-08-20/amazon-ramps-up-13-9-billion-warehouse-building-spree.html <comment><USER2> I enjoyed this article and found it illuminating. I work with a number of manufacturers (in jewelry, apparel, furniture, watches, etc.) who sell using Amazon Warehouses (the FBA program). Already, before the new warehouses, it smokes competitor warehouses that I've tried in terms of time and reliability. 52% of Amazon non-media sales come from 3rd party sellers like the companies I work for, last I checked.

Some trends I think will come soon from Amazon FBA:

1) When Amazon perfects daily grocery delivery it will change the whole ballgame for UPS and USPS. Amazon will cut them out of the picture. I think the cost of shipping furniture especially will plummet.

2) Amazon launched FBA Export a while back to let merchants fulfilling through FBA U.S. ship anywhere in the world easily. Right now only a few SKUs are eligible, but they are working to expand this program ASAP. In a year or two, a ton of products on Amazon U.S. will be available anywhere.

3) A lot of mainstream people don't know but Amazon warehouses don't just ship to Amazon customers. Merchants can enable multichannel shipping for a price ($6 compared to $2.50 for Amazon orders), so a lot of eBay sellers, Rakuten sellers use FBA to fulfill.

<comment><USER3> As I followed Amazon's buildout over the years, I couldn't help but wonder why FedEx is either overlooked in (or overlooking) this market.

I worked in one of these facilites many years ago, asked, and got mostly shrugs.

Their 'Global Distribution Centers' offer 24/7, 365 fulfillment, and exclusive 'last minute' access to shipping hubs.

<comment><USER4> I was also curious about this, I assumed the competition between UPS and Fedex was fierce enough to keep them both fairly innovative, but it sounds like Amazon is finding room for improvement...

<comment><USER5> FedEx is about business documents mostly, as far as I know.

Not to mention, they're a middle man. Amazon gains a lot by cutting out the middle man.

</comment></comment></comment></comment><comment><USER6> > The warehouse strategy carries risk. Fulfillment has become Amazon's top > operating expense, squeezing profit margins and contributing to a $39 > million loss at the Seattle-based company last year.

Not building warehouses carries more risk.

<comment><USER7> 39M USD loss is a token loss, too. Cost of building, staffing and operating a warehouse is going to be far in excess of that.

</comment></comment><comment><USER8> From the Instacart CEO: "We don't have to build warehouses, lease a fleet of trucks or manage perishable inventory, which substantially reduces our operational cost."

If they guarantee delivery in 2 hours and they don't have these costs, they're paying someone else who has these costs, right? How can they really be a competitor to Amazon, then? I have to think that Amazon is operating on such razor-thin margins precisely because they think they can do a much better job than using someone else.

<comment><USER9> I imagine that they are really the ___Uber_ __in this area... They provide a good flexible service ontop of existing infrastructure at a premium cost to users.

While that's just fine - I would love to see a service that lowers the cost to me as a consumer for food.

Actually, let me rephrase that: I, personally, need a service that helps me reduce the amount of food I waste. With a family of (5) in my household, we waste a lot of food - and it drives me nuts.

I hope that something along the lines of Blue Apron could help me throw out less/buy less.

<comment><USER10> Not sure I understand. You want to eat less or have more food deliveries so it's always fresh ?

<comment><USER9> I want to more closely buy the amount of ingredients required in what I would use them for.

I would love to be able to shop based on meals - where I say "on Wednesday night we are going to serve turkey meatloaf with broccoli and salad for 4 people" and understand the ingredients and proportions that match that.

I wind up having stuff rot because I only use a portion of it for whatever I make, and I don't have a set menu so the food is random/too-often-the-same etc...

I want the entire experience to be optimized.

<comment><USER11> Soup and probably other meals are traditionally used to recycle unused ingredients and avoid that "always the same" feeling. It might be easier to do that.

<comment><USER9> Agreed. and I am actually quite good at slowcooking stuff in my CrockPot.

Maybe I have just been too lazy? However, I am not 100% convinced that food is not sold in the wrong proportions.

</comment></comment></comment></comment></comment></comment><comment><USER12> Interesting that they complain about Amazon's operating margins. Isn't the consensus that it is intentional?

<comment><USER13> That's just what financial reporters do. Whether the margins are intentional or not, they're still information that bloomberg readers want to know about.

<comment><USER12> Bloomberg readers _should_ want to know why, rather than assuming that Amazon is actually trying to increase it's profits, like most companies. It could have been discussed in this article.

</comment></comment></comment><comment><USER14> I do wonder what is going on with Amazon shipping. For my orders over the last year, by far the longest time is spent not shipping. The status will say "shipping now" for several days. Once actually in package service it is quick. I use the "free" shipping.

I'd guess that they process Prime members shipping before the free shipping. And I'd suspect some of the waiting is time to ship from the manufacturers to Amazon's warehouses. I also wonder if they transfer long tail items between warehouses to consolidate orders. There is speculation that Amazon intentionally delays free shipping orders to push people towards Prime. Maybe they just have capacity problems?

By comparison with other vendors like Walmart I usually see shipping in less than 24 hours.

Amazon forum complaining about slow shipping, people getting Prime stuff a lot faster, no response from Amazon people:

[http://www.amazon.com/Super-Slow- Shipping/forum/Fx4HQD057HT5...](http://www.amazon.com/Super-Slow- Shipping/forum/Fx4HQD057HT5OQ/TxRFEIDS67V903/1/ref=cm_cd_pg_oldest?_encoding=UTF8&asin=B000MG3LDA&authToken=&cdSort=newest)

Giving credit to Prime members to accept slower shipping:

[http://www.geek.com/news/amazon-prime-customers-receiving- cr...](http://www.geek.com/news/amazon-prime-customers-receiving-credit-for- accepting-slower-shipping-1529819/)

Another random example complaint:

[http://www.pissedconsumer.com/reviews-by- company/amazon/very...](http://www.pissedconsumer.com/reviews-by- company/amazon/very-slow-free-shipping-and-delivery-20120718332793.html)

<comment><USER15> Agreed. Amazon's shipping service is very poor unless you use Prime.

I've bought from Amazon when I had a Prime subscription and when I didn't. With Prime, the goods were always shipped within a day. Without Prime, they let it sit on the shelf for 5 or 6 days before even shipping it.

Compare that to other companies like Best Buy, newegg, walmart, or even no name ebay sellers where products are always promptly shipped without need for any "subscription" nonsense.

<comment><USER6> Would you rather it ship today and arrive a week from now? Or ship in 3 days and arrive in 4? When you make your purchase, you are given a promised arrival date...the ship date should be irrelevant.

<comment><USER15> False choice.

With Amazon, it sits on the shelf for 5 days before it ships and then takes another 5 days to arrive.

With something like walmart, it ships immediately and then takes another 5 days to arrive.

Amazon's in-transit time isn't any faster than other merchants. They all use UPS or Fedex, so no difference there. The only difference is that Amazon takes 5 days to ship while other websites take 1.

<comment><USER6> Except that it is not a false choice at all. Amazon delivers by their promised _delivery_ date. I have ordered items with my prime membership that took _two days_ for it to ship, and yet it still arrived on time (arrived the same day it shipped, via a same day carrier). A super saver shipping promise is 5-7 business days. If it takes 5 business days to ship, then your package is getting a second-day ship method. From the several hundreds of packages I have received from Amazon, both before and after my prime membership, only 1 has ever arrived after it was promised to arrive...and it was the carrier's fault.

<comment><USER15> > Amazon delivers by their promised delivery date.

Except that there are alternatives to Amazon, hence my use of the word "choice". I'm not saying Amazon doesn't keep its promise. I'm saying when Amazon is compared to its competitors it falls short.

> A super saver shipping promise is 5-7 business days

Actually it's 5-8 business days ([http://www.amazon.com/gp/help/customer/display.html?ie=UTF8&...](http://www.amazon.com/gp/help/customer/display.html?ie=UTF8&nodeId=527692)) and usually Amazon milks it to its limit. 8 business days equal 10 to 12 real days.

Compare that to merchants that ship immediately. It usually takes 3-5 days to arrive at my home. That's a lot faster than Amazon's 10-12 days.

<comment><USER16> Do competitors usually give that shipping for free though? I thought the point of super saver shipping was you don't have to pay for the shipping but the tradeoff is you have to wait however long it takes to get your package. If the competitors are somewhat faster but you still have to pay then the scenarios can't really be compared (I am a prime member so I have paid less than $1 per delivery for guaranteed two day shipping so I have never even considered alternatives).

<comment><USER15> Pretty much everyone offers some "free" shipping these days, which is just a marketing scheme anyway. Free shippoing or not, the only number that matters is the total, all-inclusive price. It's not uncommon for me to find a lower total price at a seller that charges for shipping than the Amazon's "free" shipping price.

WSJ actually just published an article on how Amazon is frequently underpriced by other sellers: [http://online.wsj.com/article/SB2000142412788732342380457902...](http://online.wsj.com/article/SB20001424127887323423804579023280029907354.html)

There's no one seller that is consistently the lowest price. But Amazon is consistently the slowest in shipping.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> It is an interesting vote against local manufacturing. There is a small number of people who are betting that the 'future' won't be warehouses so much as they will be small 'build on demand' factories which create the item you ordered, when you order it. When you consider the 27.5yr depreciation life of this sort of physical plant you might guess that Amazon is betting against some sort of 'kindle' equivalent for goods (i.e. Shipping the information rather than the good itself)

<comment><USER3> While I'm in favor of CNC'ing things on demand, it's doesn't scale very well. Machine time adds up - multiple tooling, batch processes like cleaning/finishing/painting,etc.

'The Market' seems to agree, as warehouse buildout (and purchasing) is booming right now - as it seems more economical to produce things in large runs and then hoard them in port cities like LA,New Orleans, Memphis, etc...

<comment><USER18> Nobody's attempted to build a fully automated, general purpose(ish) factory yet. I'm sure there'll be plenty of complexities and limitations, but there are limits and complexities with existing systems that we've just accepted as "the way things are".

I'm fairly confident that general purpose automated manufacturing will become a reality and will be transformative, but I doubt it will completely take over the market. I'm pretty sure there'll be room for both.

</comment></comment></comment><comment><USER19> When you really need something, there is a premium for shipping within hours. Whether it's diapers or a six pack of beer, speed matters. This is a huge pivot from Amazon, which was previously won on price and size of catalogue.

Note - If Amazon gets licensed for beer and can deliver it in hours, game over for a lot of players!

<comment><USER20> Also for diapers... I know you were being facetious, but speaking as a parent, a company that can deliver diapers and formula within a few hours is going to get a lot of immediate attention and devotion!

<comment><USER21> Amazon provides the subscribe and save service to fill this need. You can set up the frequency of delivery and the quantity of units and never have to worry. They give you a nice discount on the order if you do it this way as well

<comment><USER20> Clearly you are only passingly familiar with young children. ;) As @mathattack implies, one cannot predict an infant's near-future need for diapers with the same confidence with which one can predict e.g. when a furnace filter needs changing.

</comment></comment></comment></comment><comment><USER22> If they didn't spend the money on growth it would count as profit and they'd have to pay taxes on it.

</comment><comment><USER23> One more proof that Amazon is a retail company and not a technology company (only technology is allowed is for the retail sake).

</comment><comment><AUTHOR> Yeah, slow and steady growth...

<comment><USER24> Amazon's never been about "slow and steady" growth. They've been about long- term growth at the expense of short-term profit.

<comment><USER25> Or, more to the point, using their dominance and abundance against their competitors. They may have razor thin margins, but that's only because they're leveraging their pricing to cut into the margins of their competitors.

Lower prices here and there, raise them elsewhere to offset the cost. Their accounting department definitely has to be kept on their toes, especially to make sure things don't fly off the handle _too_ much.

<comment><USER26> Amazon: Fast, Cheap, and Good: Pick all three.

They've really delivered. I've tried to not like Amazon, but they keep impressing with their fast service, availability, low price.

<comment><USER27> Exactly. When Rise of the Warrior Cop came out, I wanted to pick it up on release day. None of the local bookstores had it in stock, but all were happy to order it for me. It would have taken them all 4-5 business days.

I went to Amazon that night, and with Prime, I got it in 2 days, for about $10 less than the local brick and mortar stores.

How can anyone compete with that? Especially if they build out their same-day service?

<comment><USER28> Yeah, I think same day will strike a complete death blow for retail if they can pull it off well.</comment></comment></comment></comment></comment></comment>


Trojan T-shirt targets German right-wing rock fans - <AUTHOR> http://www.bbc.co.uk/news/world-europe-14465150 <comment><USER2> I'm a German and I wonder if the word "right-wing" is understood in the same way by US Americans as it is understood by us. Because of our past, we treat everything coming from the "right" as being totally wrong and stupid (which might be good or not). But I think in other countries, with another historical background, right-wing might mean just conservative, not skinhead racist. As far as I know, right-wing in China is even the cool new thing, that young people follow (because it is against the depressing government, which many young Chinese don't like at all). Tell me about how you understand "right- wing" in non-German countries, please!

<comment><USER3> The part I didn't understand was that the slogan for the concert was "Never again communism - Freedom for Germany". Do I take this to mean that anyone opposing communism is considered a right-winger in eastern Germany?

<comment><USER2> Nope. Just if you say something like "Against Foreigners" you just go to jail. So the new skinheads (neo nazis, as we call them, don't know if that is an English word, too) try to act more political correct, while being in the open.

<comment><USER3> They're called neo nazis here in the US too. They're not mainstream right- wingers, but an extreme element on the fringes. I'm still missing the link between not wanting communism to return to eastern Germany and neo-nazism, though. I assume you mean that the slogan is a political misdirection.

<comment><USER2> Yes, it is meant as being misleading. There is no direct link between anti comunism and neo nazism. I just think neo nazis are against pretty much every thing, so I am also not confused to see them using anti comunism slogans. Their power and attention comes from being against something. What that something is, that is probably not so important.

</comment></comment></comment></comment></comment><comment><AUTHOR> First the t-shirt says: "hardcore rebels national and free" after washed the original writing disappears. Instead it says: "What your t-shirt can do you can do too." Images: [http://www.businessinsider.com/trojan-t-shirts-neo- nazis-201...](http://www.businessinsider.com/trojan-t-shirts-neo-nazis-2011-8)

</comment><comment><USER4> On the technical side, how would I make something like this? Can you order these kinds of wash-once-to-change-message shirts? If not, what kind of ink would you use to DIY it?

<comment><USER5> I guess combining one paint which isn't water-soluble with another that is. Acrylic paint would probably do the trick?

<comment><USER6> I'd like to know if the paint that obviously was removed from the shirt caused any other issues, tainting other clothes for example?

I mean - if you dress in black completely it might be a non-issue of course.

</comment></comment></comment><comment><USER7> Conning people doesn't seem like a good way to get your message through to them.

<comment><USER8> Actually, it does work. I was working in a similar group (locally and much smaller) for some time. You can reach everyone with clever and impressive actions for a simple reason: they are impressive and resonate so much that even the "enemy" has to admit that they were impressed. You won't turn anyone, but you make sure that they know someone to turn to once they do. Its an offer to talk.

Considering that exiting the right-wing circle often involves ditching most of your friends, someone to talk is the most important asset "on the other side".

</comment></comment><comment><USER9> I think this would be the most bad ass guerilla marketing technique.

Say you are WePay - and you go to a Paypal event and hand out hundreds of PayPal shirts, which become WePay shirts after the users take them home and wash them for the first time.

Or, we can simply go to TeaParty campaigns and troll unsuspecting users there :)

</comment><comment><USER10> here's an article in the english section of the german news magazine SPIEGEL, explaining the intentions of the organizers in detail:

[http://www.spiegel.de/international/germany/0,1518,779446,00...](http://www.spiegel.de/international/germany/0,1518,779446,00.html)

</comment><comment><USER11> on a sidenote, Trojan Skinheads are not political in any way. They are chaotic neutral ..

<comment><USER11> people who voted this down probably have no idea about the spirit of '69.</comment></comment>


Youtube.com is down? - <AUTHOR> http://youtube.com/

<comment><USER2> It was down for me, then briefly up, but is once again reporting 502.

Edit: The main page reports 502, but I had a video open in another tab that, once refreshed, played ok.

</comment><comment><USER3> It is up now.

<comment><AUTHOR> It was down, for about 5 mins. Maybe only for certain users.

</comment></comment><comment><USER4> And were back...

EDIT 12:23 - Spoke too soon. 502 again.

</comment><comment><USER4> down for me, intermittent 502 errors.</comment>


Post-Mortem and Security Advisory: Data Exposure After travis-ci.com Outage - <AUTHOR> https://blog.travis-ci.com/2018-04-03-incident-post-mortem <comment><USER3> Kudos for a thorough and transparent writeup, and (by the looks of things) understanding that processes fail rather than individuals.

That said, I have to admit to having at least three eye-bulge WTF moments while reading this.

I'm also surprised that there isn't a Remediation step of "firewall the development machines away from the production database".

(And isn't the change to database_cleaner to make it throw when run against remote databases by default a serious break of API compatibility? What if someone's depending on that behavior?)

<comment><USER4> Indeed!

What a great way to see failure: what went wrong, how to improve without blaming it on the poor guy. I wish all companies were like this - very forward thinking!

Funny thing is... this morning on my commute I was watching [https://www.oreilly.com/ideas/developing-a-chaos- architectur...](https://www.oreilly.com/ideas/developing-a-chaos-architecture- mindset) \- which I recommend to anyone!

<comment><USER5> I think it is now widely understood that best practice is to blame processes, not people.

However, i do have a niggling worry that letting people off the hook risks stunting their personal growth. If you can make mistakes and not bear any responsibility for it, what is driving you to get better at doing your job?

<comment><USER6> I think when the circumstances actually show someone was grossly negligent, it's very hard to convincingly pretend that only processes are to blame. Remember that the current thinking is working against thousands of years of a cultural pattern assigning responsibility.

It's also interesting how many seem to be using this idea meant to be applied to _internal_ processes and individuals and applying it to the companies themselves.

The most significant example was Gitlab: they had something like four backup mechanisms for their database, yet three(!) of them had not been working for months, with nobody noticing. Then, someone actually dropped the production database. The last working backup system also had some problems I can't remember right now.

Yet in the comments here, there was barely a hint of criticism of Gitlab. Instead, they were lauded for not lynching the poor guy who made the last of what must have been dozens of terrible decisions leading to the incident.

Travis' incident here does actually seem like the sort of freak accident that one can't rule out completely, at least not at a company of their size. But forgiveness might not always be warranted.

<comment><USER7> > I think when the circumstances actually show someone was grossly negligent, > it's very hard to convincingly pretend that only processes are to blame.

On the contrary - the entire reason why you transition from culture to process as you grow is because when you measure by results, there's no difference between unintentional negligence (leaving a production terminal open à la Travis), intentional negligence (let me bystep this annoying check, let me procrastinate on fixing the backups a la GitLab), and malice.

If you get large enough, you will have grossly negligent people, by statistical inevitability. You can either accept this statistical inevitability and design your process for it or you can continue to believe that you (and everybody else) really actually do only ever hire the very best.

<comment><USER8> GP is pretty clearly talking about blaming the company (processes) and not individuals.

The entire post laments the fact that entities external to the company will repeatedly forgive the company if the company goes through the motion of publishing a blameless post-mortem. Even if those post-mortems repeatedly indicate the company really didn't learn anything, or change the processes that led up to the incident.

But by golly, they're forward thinking company, and therefore blameless.

</comment></comment></comment></comment></comment></comment><comment><USER9> So one of the ways of analyzing the root cause here is that the autoincrement index of the user table in their database is security-sensitive, and relatively normal DB operations like "Let's roll back the DB" have serious security implications involving ID reuse. What are some ways to make this less dangerous? (The rest of it was an operational failure, but it would have been less trouble if it weren't a security failure.)

I can think of the following:

\- Don't use auth cookies that are signed messages consisting of a UID + expiration date and other data, use auth cookies that are opaque keys into some valid-auth database. This is significantly less efficient (every operation needs a lookup into the DB before you can do anything; if you move it into a cache you now risk the cache being out-of-date with your DB). AFAIK using signed UIDs has no security downside other than this, right?

\- Identify users by usernames, not by UIDs. This makes renaming users (which GitHub allows, so Travis is forced to allow) difficult and security-risky.

\- Use UIDs that are selected from a large random space so make collisions unlikely, e.g., UUIDs or preferably 256-bit random strings. This seems fine and probably preferable from a security point of view. Is this fine from the DB point of view?

Anything else? Maybe a DB restore-from-backup option that preserves autoincrement counters and nothing else - is that a standard tool?

<comment><USER10> I use a mix: UIDs (auto-incrementing primary keys) for internal app use (e.g. joins), but use UUIDs for referencing records outside the app (anything sent out over the API).

The UIDs just make for easier to read logs, and easier to inspect and hand- write db queries. The UUIDs just seem much more secure when communicating with client applications.

I also agree that authing with a DB is preferable, if you can afford to do that at the scale your app needs (which is most apps out there).

<comment><USER11> Why not just UUID for the primary key also and only have one column? Having 2 columns seems like extra complexity for not much gain. Modern databases usually have a native UUID type column which stores and compares better then using the string/char type.

One more benefit of using UUIDs for primary keys is that clients can generate models along with the primary key and know what the key will be BEFORE they submit it to the database. That is, it works really well with distributed systems. Auto incrementing primary key is really state stored at the database level and a source of contention

I would even argue that UUIDs are far better to read logs. You do a grep for that UUID in the log and it's easier to find then to grep for integer primary keys. If your contains disparate models then UUIDs will find your entries easier then integers.

If you ever have to rollback your database but keep some of the new data, it's easier also. You just export the data you want to keep, do your rollback, and them import that data knowing that auto incrementing primary keys will never be an issue.

You can copy objects easily from one database to another and know the primary key will be the same, and not clash.

For hand writing db queries, it's also better to use a UUID since then I can use that same query cut and pasted into other databases with the same data and know that I don't have to check primary keys. The UUID is just a copy/paste.

<comment><USER12> A UUID primary key has to live in every index in the table and every foreign key that references it.

We have a table with a UUID as its primary and it alone consumes 22GB, then a index reference it, so now another 22GB... That one primary key uses over 100GB of storage, a developer recently went to add another table that reference it and we had to decide whether we were okay taking another hit. If we used your example of using a char type (we use blob) it would be double the size...

It doesn't sound like much, but in a database with all primary keys being UUIDs you are going to inflate the size of your DB quickly, or have to forego using foreign keys. I imagine if we used only UUIDs we would have to double or triple our database infrastructure. Additionally, we would be forced to introduce partitioning sooner.

Now we have a tech debt ticket to add an auto increment to that table so we can reclaim disk / memory.

<comment><USER11> I would argue that's a reasonable hit for the gains. I would never add a tech debit ticket to change just to reclaim disk / memory, that's an anti pattern.

Memory and disk space is cheap.

Also UUIDs are just 128bit integers internally. My example clearly points out that most modern databases have a native uuid type (and if not use use a 128 bit integer).

Using a char type to store UUIDs is very wasteful and no wonder your indexes are so huge.

The tech debt ticket you really have is to convert type of uuid column from char/blob to native.

Note also that if you need to expose your data, you can either expose the auto incrementing primary key (which leaks data, in that other people can work out growth rates of various models, like how England worked out how many German tanks were being built based on a auto incrementing serial number), or you create a another column (as someone else suggested) which is a uuid. Which you would need to index anyway.

If index size is an issue then you can just integers which are randomly generated. You can calculate your collision chance based on how large your integer is and how often you create. If you happen to pick 128bit integers, congratulations, you're using UUIDs.

<comment><USER12> >Using a char type to store UUIDs is very wasteful and no wonder your indexes are so huge.

That was a hypothetical, its 22GiB using BLOB storage.

>Note also that if you need to expose your data, you can either expose the auto incrementing primary key (which leaks data, in that other people can work out growth rates of various models, like how England worked out how many German tanks were being built based on a auto incrementing serial number), or you create a another column (as someone else suggested) which is a uuid. Which you would need to index anyway.

I state that we use the auto-increment inside of the application, everything exposed by the API is a UUID. While we lose 32 bits of space per row, just one foreign key or index saves enough space to justify it.

Best of both worlds.

>Memory and disk space is cheap.

Not when you need to move from multiple r4.8XL to r4.16XLs (think multi-az, multi region). It would work out to around double the price, we can hire a developer for the cost of hosting that infrastructure. Even when your business has the money, you don't want to be the guy telling them to drop another $24k a month on hosting costs because we didn't think one year ahead.

This of course ignores all the issues that come with replication and partitioning, multi-master etc.

</comment></comment></comment></comment></comment></comment><comment><USER13> > The shell the tests ran in unknowingly had a DATABASE_URL environment > variable set as our production database. It was an old terminal window in a > tmux session that had been used for inspecting production data many days > before. The developer returned to this window and executed the test suite > with the DATABASE_URL still set.

I was expecting something like this. I remember, I configured my terminal windows to change their background when I'm on production systems [2], after around I read about gitlab database incident [1].

1 : [https://about.gitlab.com/2017/02/01/gitlab-dot-com- database-...](https://about.gitlab.com/2017/02/01/gitlab-dot-com-database- incident/)

2 : [http://www.drinchev.com/blog/ssh-and-terminal- background/](http://www.drinchev.com/blog/ssh-and-terminal-background/)

<comment><USER3> Though note that wouldn't help here, at least according to a common sense reading of this part of the post:

> we connected our development environment to a production database with write > access

i.e. they weren't logged in to a production machine, they were logged in to a development machine that was permitted to connect directly with write access to the production database.

<comment><USER9> Since it's a local shell doing something like this is a lot easier - <USER2> change $PS1 to add some colors when certain variables are set.

I use something like this today on my personal machine to distinguish between my personal and work email addresses in $EMAIL (for silly firewall reasons it's easier to originate work-owned OSS on my personal machine), and on my work machine to tell me _which_ production zone I'm talking to - but I don't have it color when I have any variable set at all, I probably should do that.

</comment></comment></comment><comment><AUTHOR> I was surprised about a tmux session connected to production DB for days. Though it was idle there are a lot of things that can go wrong during window switching. My colleague also pointed out the subtle error of assuming the value of DATABASE_URL in the system instead of being set explicitly by the test script that could have avoided this.

That being said I am amazed at their transparency over the whole issue and a thorough write up of the whole incident. It's something we can all learn from.

<comment><USER14> I am not a tmux user so maybe this is obvious, but how are the environmental variables transferring from the production machines to where the tests are being run?

Or, is this stating that the developer started running tests from the production environment?

<comment><USER15> > but how are the environmental variables transferring from the production > machines to where the tests are being run?

tmux[1] is a terminal multiplexer, a screen alternative.

Some speculation, they might have stared the session on a remote machine; set `DATABASE_URL`; use it for something; detach from the session; to later reattached with the bash env still in tack; without the developer being aware he executed the tests commands.

Another tmux "problem" is developers using it to tail logs with a infinite back-scroll to only fill up server memory. One solution was to kill all tmux session nightly via cron. Might have helped here.

Finally as tui lover, try tmux! Really keeps the frustration at bay if you ssh to a box and your session drop to know your terminal workspace is still in tack.

\- [https://github.com/tmux/tmux/wiki](https://github.com/tmux/tmux/wiki)

</comment></comment></comment><comment><USER16> If you are interested in a list of steps you can take to avoid this happening to your data, here are some suggestions. I don't believe that any single measure is sufficient. And I also believe that it's valid to balance the strictness of your controls against the the amount of protection you really need.

1\. Vault the passwords. People and machines should fetch passwords on-demand using identity credentials.

2\. Create a read-only database account. In all cases, use the account that matches the need. Running reports? Use the read-only account.

3\. Restrict access to read-only and read-write database accounts. Provide this account information to a limited set of people and tools.

4\. Provide a fairly straightforward way for people to get temporary elevated access. If it's easy to get elevated access, then users will not be tempted to "hold on" to elevated access longer than they should (e.g. by leaving a terminal open for a very long time).

5\. Rotate the credentials of all the accounts regularly. This ensures that temporary elevated access will become long-term access. It also greatly reduces the harm created when credentials are leaked, exposed, or forgotten about (e.g. in an environment variable in an old window).

Note that none of the steps above require a heavy investment in automation. You can start with basic (even fully manual) processes for key management and access management, and evolve to automation as you grow.

Finally, keep in mind that this type of accident is not just a "small company" problem. Recall this AWS ELB outage on Christmas Eve of 2012 - [https://aws.amazon.com/message/680587/](https://aws.amazon.com/message/680587/)

        "The data was deleted by a maintenance process that was inadvertently run against the production ELB state data."

<comment><USER5> > 4\. Provide a fairly straightforward way for people to get temporary > elevated access. If it's easy to get elevated access, then users will not be > tempted to "hold on" to elevated access longer than they should (e.g. by > leaving a terminal open for a very long time).

Or to hack around it in some other way, eg by adding some sort of backdoor to a production application.

This is such an important lesson, and one that unfortunately is lost on many command-and-control type IT departments.

</comment></comment><comment><USER17> They should really consider using a CI system to run their tests.

<comment><USER18> haha that made me chuckle.

</comment></comment><comment><USER19> I think the root issue here is that the production database "user" has too many privileges, and the reason for that is migrations. This is compounded by the test user essentially needing to be a db superuser to create and destroy test databases, as well as run the migrations for them. I've noticed this lately with Django, but I'm guessing that it's a general problem.

When I design a DB system, Ideally the production 'user' can only do those things that we reasonably expect them to be able to do, and truncate isn't one of them. Also drop tables, potentially delete entries, and any maintenance tasks. DDL modifications are right out.

Those tasks can be run from a specific user, and locked down to a certain types of connections that aren't allowed from production.

<comment><USER5> When i first worked with migrations, they were run manually as part of the release process. Over time, we automated them - by making them part of the release tool. It never occurred to us to make them part of the application itself. For us, it was therefore completely natural that the app's DB credentials did not have DDL permissions.

The whole idea of migrations being run by the app still seems really silly to me. I suppose this is the obvious move for developers who (very sensibly!) build applications, but don't build their own release tools, to whom it would never occur to put migrations anywhere except the application.

<comment><USER20> > The whole idea of migrations being run by the app still seems really silly > to me.

I have one app that kinda runs migrations - there's a helper app which is built at the same time and (normally) runs before the main app. But this is based on a per-user database which means there's no "release" process that could run the migrations.

(Similar to iOS apps, really - when you get a new version of Teleappchatbook, it'll often have an "upgrading..." step which I assume is migrations / index updates, etc.)

</comment></comment></comment><comment><USER21> Wow, the issue with the signed token is very interesting. Found it surprising the authentication method specifically wasn't mentioned in the remediation.

Food for thought: the security issue wouldn't have happened if (1) travis used UUIDs instead of sequential IDs as a pkey, or (2) used a secret token for auth instead of a signed (presumably) JWT.

<comment><USER10> 1) They don't need UUIDs as a pkey, but just as another field. I like using auto-incrementing IDs as the primary key, and UUIDs for referencing records over internet-facing APIs. Best of both worlds!

2) Totally, JWTs are evil.

<comment><USER22> I don't know much about JWTs. Why are they bad?

<comment><USER23> JWT itself is a nice container for signing a small amount of JSON and being able to easily pass that around. I use it a lot for situations where I want to ensure someone hasn't futzed with the data, and/or I want an auto-expiring token of some sort.

JWT, by itself, is not an authentication and authorization system, but people often use it as such.

</comment></comment></comment></comment><comment><USER24> Wondering why does any developer need update/delete/drop access to a prod database? Or why would ad hoc scripts have this ability?

<comment><USER18> only devops should have this info and they should be guarding it fiercly and only use when they know why/when... it would be easy enough to mirror all production data to a 2nd db w/ full read/write that's updated daily or weekly from source. That should give plenty of data for devs to work with.

<comment><USER25> isnt the idea of guarding access fiercely from developers the antithesis of devops? That just makes 'devops' people 'ops'

<comment><USER26> Least privileged access isn't necessarily the antithesis of devops.

You could argue that dev-ops model include automation for operations on production systems, with direct access to production systems limited to a reduced set of staff. Developers can create the change-sets to modify infrastructure, but those change sets are reviewed, validated, tested, and then executed on production via CI/CD Automation.

Infrastructure as Code, and Immutable Infrastructure lends itself well to that approach.

</comment></comment></comment></comment><comment><USER27> A suggestion for production database access, create a readonly login (in addition to a write one).

Login using the readonly login the majority of the time, and only switch to the write login when required.

</comment><comment><USER28> Finally I can show a solid example to my team mates who ridiculed me when I said we needed restrictions on access to prod servers. This is a great write- up!

<comment><USER29> Um, you shouldn't need a solid example for such a crystal clear best practice. I mean, everyone has access restrictions to PROD, it's simply to valuable and too costly to have to bring back up after a fuckup.

</comment></comment><comment><USER30> >" Using our API logs, and with information from our upstream provider about the IP address the query originated from, we were able to identify a truncate query run during tests using the Database Cleaner gem."

I'm assuming by "upstream provider" here they mean ISP/IaaS provider. Either way they didn't have enough information under their control to identify the source of the query. The reliance on a third party for accurate logging information seems like a big blind spot.

What if the upstream provider didn't have the logs? Or the request for access to those took an excessive amount of time? I didn't see anything in the remediation steps to address this.

</comment><comment><USER31> Interesting writeup. I loath setting environment variables on long running terminal sessions exactly because it's not obvious once they're set.

I prefer to use a subshell for the command and set the environment variable each time:

$ ( export FOO=bar; my_cmd )

<comment><USER32> Or just:

env FOO=bar my_cmd

<comment><USER3> Or just:

FOO=bar my_cmd

<comment><USER33> Or just:

         FOO=bar my_cmd

That's the same but with a leading space, to prevent 'bar' from showing up in your histfile.

<comment><USER34> On bash, this assumes that $HISTCONTROL is set to ignoreboth [0]. This is in the default ~/.bashrc on Ubuntu, but I can't speak about other distros.

I don't use zsh, but I believe the equivalent is $HIST_IGNORE_SPACE [1].

[0] [https://linux.die.net/man/1/bash](https://linux.die.net/man/1/bash)

[1] [https://unix.stackexchange.com/questions/6094/is-there- any-w...](https://unix.stackexchange.com/questions/6094/is-there-any-way-to- keep-a-command-from-being-added-to-your-history)

</comment></comment></comment></comment></comment><comment><USER35> One thing that immediately caught my attention: the fact that it is possible for a single query/command/request to wipe everything.

To be frank at a place I worked there had always been something like this too: if you were logged in as super admin, wiping all data is just one POST request away. That was super convenient when testing things, but having the same in production made me uneasy. Fortunately before any incident happened I added additional checks that required special command line flags to enable this API. Perhaps still not super foolproof but I felt much better.

<comment><USER36> I wish people would talk about this more. I am of the belief that if a single command can lead to a failure like this, you can't simply plan on that accident not reoccurring. You should basically assume that it will reoccur.

Ideally, I think databases should integrate checks such as these. For example, how often does a production users table need to be truncated intentionally? Even by superusers? Usually very, very, rarely. So imagine if the database made you jump hoops before you could do that. People rely on permissions for this sort of thing but given how complicated permissions can become as the team and the database grows, it's not hard to screw up the permissions and access control. I believe catching this kind of doomsday scenario is best when built in deeply at a very low level.

<comment><USER37> What about a database that would require separate DDL from data commands? Instead of having a single super-user that can do both DDL and INSERT/UPDATE/DELETE, you would instead have a DDL user, and a data user. That would probably prompt people to only use the data user in their application?

</comment></comment></comment><comment><USER38> Shouldn't there be a remediation step of making it impossible to login into another users' session? E.g. generate a random number for every provisioned user and add it to the token.

</comment><comment><USER39> All of my production terminals have dark-red background and my screen hardstatus also red. This is my default in rc files, and I have to explicitly link rc files to get my dev-only black background with lime hardstatus.

<comment><USER5> How do you set the background colour?

</comment></comment><comment><USER40> Lots of focus in the comments on the database access issue, but trusting the user specified (signed) token doesn't seem like a great idea. Not validating the token against database seems like a painful shortcut

</comment><comment><USER41> This makes me think of the Google SRE book. They advise that, if there is a problem this big, any SRE should have the power to turn off the production load balancers until the problem is fixed.

I don't think that TravisCI did anything wrong. However, if they had turned off the load balancers as soon as they realized that there was a huge issue, it might have protected customer data more. They optimized uptime over completely fixing the issue. Also, perhaps nobody felt that they had the authority to turn off the production service.

</comment><comment><USER42> So the session keys mapped to usernames, rather than IDs in the database? Otherwise, when the database is restored with the old user IDs, the session would become invalid instead of continuing to work. This is what I'm seeing:

1\. Tables truncated. 2\. In this window, someone creates an account with a username that existed in the dropped database. 3\. They see a blank user page because a new user record was created. 4\. Database restored. 5\. It's as if you're logged into the original user's account.

<comment><USER43> This is not what happened. The tokens were mapped to user IDs and when people signed in, the db created new users which may have had the same IDs as old deleted accounts. When they restored the DB, these tokens pointed to other users and granted access to these other users' accounts. Quite an unfortunate situation. May have been mostly avoidable if UUIDs were used instead of incrementing IDs, but hindsight is 20/20.

<comment><USER42> The part of that that I don't get is how a new user could have the same ID as an old (truncated) user since "our system created new records for them, with primary keys generated from the existing sequence (PostgreSQL does not reset id sequences on truncate)."

Do they mean that the only potentially exposed accounts are those that signed up after the database was restored?

<comment><USER43> It's possible their truncate also restarted the sequence: "TRUNCATE TABLE users RESTART IDENTITY;"

[https://stackoverflow.com/questions/13989243/sequence- does-n...](https://stackoverflow.com/questions/13989243/sequence-does-not- reset-after-truncating-the- table?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa)

Although, I am confused about how sign ins created new users. When they say sign ins, do they mean new accounts?

<comment><USER42> Yeah they must mean new accounts, if not then I'm lost. I guess it could have reset autoincrement but they said it didn't. The only other thing I can think of is that the signed token that's put in localStorage is sent to the server like "someuser|sometoken", the server inspects sometoken, says it checks out, then takes the client at its word that it's someuser.

</comment></comment></comment></comment></comment><comment><USER44> Reminds me of this:

"Why Auto Increment Is A Terrible Idea" (2015) [https://www.clever- cloud.com/blog/engineering/2015/05/20/why...](https://www.clever- cloud.com/blog/engineering/2015/05/20/why-auto-increment-is-a-terrible-idea/)

(update: link fixed).

<comment><USER45> Working link : [https://www.clever- cloud.com/blog/engineering/2015/05/20/why...](https://www.clever- cloud.com/blog/engineering/2015/05/20/why-auto-increment-is-a-terrible-idea/)

</comment></comment><comment><USER46> Amazed to see how transparently they have written the post. I think we all can learn from such outages[0]

[0]: [https://about.gitlab.com/2017/02/10/postmortem-of- database-o...](https://about.gitlab.com/2017/02/10/postmortem-of-database- outage-of-january-31/)

</comment><comment><USER47> Classic case of Developer returning to window with prod env setup. I am sure it was a "blameless post-mortem" i.e action item contains change in tooling and processes rather than trying to change human behaviour.

<comment><USER48> > _... i.e action item contains change in tooling and processes rather than > trying to change human behaviour._

Contrary to what we might hope, in my experience the former is often much easier to ensure and enforce than the latter.

</comment></comment><comment><USER49> Aren't these the folks that spammed every github repo with a spam pull request to integrate their system into your code? I kinda lost all respect for this project and their developers after that incident.

<comment><USER50> Apologies that you were affected by this. The script creating these pull requests was created and run by a third party not affiliated with the company. We were similarly upset by this.

<comment><USER51> > created and run by a third party not affiliated with the company.

Are you saying you never hired the third party, they went about and did this of their own accord without talking to you first?

<comment><USER50> Correct, this was from an overenthusiastic user, someone we did not know and had no direct contact with.

At the time we weren't actually making money yet, and most of the contributions to Travis CI came from outside collaborators.

To add to the confusion, we did indeed have a bot in the early days that would comment on pull requests, but only if the repository was using Travis CI already (this has now been replaced by GitHub's status API). However, this was not the same bot account that kept opening unsolicited pull requests on random projects.

<comment><USER51> Thanks for the clarification -- is there an official writeup about the incident somewhere? (should have asked that initially, sorry)

<comment><USER50> I was looking for one as well, but it seems we did not write a blog post. I will do some digging when I find the time, as I know we at least messaged some people that voiced their frustration directly.

</comment></comment></comment></comment></comment></comment><comment><USER52> Has anyone estimated whether SaaS in general reduce or improve the uptime in aggregate over all users ?!

The obvious arguments is that a specialized SaaS is more reliable, but the rare outages are horrific...

</comment><comment><USER53> Why does an extremely dangerous tool, such as a database cleaning tool/library, rely on an environment variable to define the target?

<comment><USER54> What's the ideal way to do so? Most of the production systems I've seen distinguish between prod and staging with only this.

</comment></comment><comment><USER55> Site is down for me, but this mirror isn't:

[http://archive.is/klfF5](http://archive.is/klfF5)

</comment><comment><USER56> What is a "read-only follower"? Is this a common term when handling databases? Is it different than a slave?

<comment><USER57> Many people are using leader/follower instead of master/slave.

<comment><USER56> First time I hear of this, who uses this and why?

<comment><USER58> "master/slave" evokes the unhappy history of slavery, which people care about to widely varying degrees.

"leader/follower" or "primary/replica" are much more neutral terms that won't prompt negative emotions in many people.

People who choose one of the latter two options do so either because they feel it is more accurate, or because they wish to avoid the negative connotations of "master/slave", or a mix of both.

<comment><USER56> It's better to stick to tried and true terminology that everybody understands than to needlessly introduce new and redundant designations for concepts that have been in use for decades, just to avoid upsetting rather irrational American sensibilities.

<comment><USER59> Leader/follower and primary/replica are common terminology now. It's unfair to paint the sensibilities as irrational - this is exactly an example of the kind of nonchalance that helps lead to underrepresentation of minorities in tech.

You might not care, but there are a lot of people that do. Taking steps like this improves the comfort level others while affecting yours none. Why is that not worth it?

<comment><USER60> Primary/replica is also _much clearer about what's actually going on_ than master/slave.

</comment></comment></comment></comment></comment></comment></comment><comment><USER61> I'm currently job-seeking and I've seen many jobs ads asking for CI experience. I'm not fond of using SaaS solutions and would like to fiddle with CI in private (e.g. using a private gitlab repo.

What would be the steps to setup an own, private and open source CI solution for, say, a Go, PHP, or JavaScript project?

<comment><USER62> Jenkins has existed for a long time.

It has plugins for many things, and you can just run shell scripts when those don't work for you.

The Job DSL plugin is good for putting everything in code and generating jobs programatically.

There are a few other CI tools, GitLab itself you mention has been a full featured CI solution for quite a while and you can host it yourself.

<comment><USER61> Great, it also has a Docker image apparently.

</comment></comment></comment><comment><USER63> Amazed at their transparency!

</comment><comment><USER64> Attack of Little Bobby Tables

</comment><comment><USER65> Not being snarky. How hard is it to setup DB replication and do testing/QA on that DB? Isn't it the SOP?

Why the remediation list doesn't include it?

<comment><USER66> Presumably because that's already SOP. It sounds like the query was supposed to run against a development or staging DB, but an environment variable that the dev wasn't aware of caused it to run against prod instead.

</comment></comment><comment><USER67> Great writeup.

This is the third case I'm aware of where CI deleted the production database. Others are GitHub (back in 2010: [https://blog.github.com/2010-11-15-today-s- outage/](https://blog.github.com/2010-11-15-today-s-outage/)) and LivingSocial.

<comment><AUTHOR> A video of Zach Holman talking about dropping DB in production twice where they ran CI over production credentials : [https://www.youtube.com/watch?v=AwXhckRN6Mc#t=5m50s](https://www.youtube.com/watch?v=AwXhckRN6Mc#t=5m50s)

Great video overall about how to handle these situations</comment></comment>


Ask HN: Who are the top UI designers on the web today? - <AUTHOR>

It would be interesting and useful to know who the top 5, 10 or even 20 designers are and what they've done that's so impressive, innovative, useful or loved. <comment><USER3> With all due respect to the aforementioned people, but personall I think the real rockstars are the leading designers behind sites like Yahoo, Amazon, CNN, BBC and digg's Daniel Burka. Designing for such a large audience while still keeping an eye on a company's goals and vision is what really is impressive. Making something look good is the easy part -- making it work flawlessly in an pre-existing environment with an incrediblyt large audience is the challenge.

<comment><USER4> A lot of my UI pals disagree, but I think the guys at Amazon and CNN are killing it, certainly operating at the highest level.

<comment><USER5> CNN are great at cross-linking and surfacing content, and the latest update is furthering that goal. Amazon...? Not so sure. They succeed despite their fugly, confusing UI rather than because of it.

<comment><USER4> Fugly is up for debate (one probably not worth having).

Confusing I'd like to hear your defense for. I don't _personally_ find it confusing.

Regardless, what is a better measure for the success of a product's interface than its ability to support a company's objectives? Amazon's site is doing just that.

<comment><USER6> I personally think Amazon's pretty fugly

I also find myself scrolling around and looking for stuff alot on amazon - <USER2> much _recommendations_ and stuff...

<comment><USER7> Which is exactly what they want you to do :)

<comment><USER8> Just because they want you to do it, doesnt mean its very good.

<comment><USER9> it's good for business.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> The work they do over at <http://metalab.com/> is some of the best I've seen.

Edit: Oops! He's right, <http://metalabdesign.com/>

<comment><USER11> We have been incredibly impressed with the work Metalab has done for us - <http://urbanairship.com/> and our admin interface at <https://go.urbanairship.com/> . Couldn't recommend them highly enough.

Also, totally love Refresh Media - <http://refreshmedia.com/> \- they did <http://bacn.com/> for us.

<comment><USER12> I visit Urban Airship and my first instinct is to hate how little padding there is between your iPhone image and the image right below it, which strikes me as overlarge.

<comment><USER11> Thanks for the feedback! that's actually something we did after our contract with them, so it's no surprise that we didn't do that well :)

</comment></comment></comment></comment><comment><USER13> You're asking about UI designers and most people have provided links to Designers - two different beasts in my view.

I think stackoverflow if one of the best examples of brilliant UI I've seen recently - certainly one of the best I've seen on the web - but it probably wouldn't make my top list for beautiful design.

<comment><USER14> _You're asking about UI designers and most people have provided links to Designers_

This is the crucial point in this whole discussion. If the user has a task to accomplish, the user may indeed want the task to be accomplished in a beautiful environment, but most of all the user wants the task to get done. User interface design is first of all tested by seeing whether or not any typical user anticipated for the site or for the application can accomplish the task successfully.

</comment></comment><comment><USER15> I've always been a fan of MetaLab's work

I also really like what Sofa does: <http://www.madebysofa.com/> (checkout the enstore demo, epic: <http://www.enstore.com>)

Ryan Singer of 37signals was very impressive when I saw him speak at RailsConf this year. I definitely think he's up there. <http://twitter.com/rjs>

</comment><comment><USER16> A couple that come to mind:

<http://subtraction.com> <http://cubancouncil.com> <http://madebyelephant.com> <http://jasonsantamaria.com> <http://markboulton.co.uk> <http://hicksdesign.co.uk>

</comment><comment><USER17> I think our designer (<http://dekorte.com/>) did an awesome job:

<http://shop.stylous.com/>

</comment><comment><USER18> Is there a way to measure it, other than subjective tastes?

</comment><comment><USER19> I'm a big fan of Nathan Borror's stuff: <http://nathanborror.com/>

Also Wilson Miner: <http://www.wilsonminer.com/work/>

And Greg Newman: <http://carbon8.us/>

Keep in mind that I don't know many designers, so I've pretty much just listed all the ones I know.

</comment><comment><USER20> I don't think it comes down solely to the designer, instead think of the company and team of professionals who make a great UI possible. A great UI design trapped inside of a photoshop file is useless. It takes the effort of multiple people to make it happen - I think a better question to ask is what companies are creating the best UI's that are actually being used today.

<comment><USER21> agreed completely.

</comment></comment><comment><USER22> Whoever designed the heroku site.

<comment><USER23> I would have to disagree, I think there a quite a few users like myself that find it difficult to look at light font on dark backgrounds for too long.

</comment></comment><comment><USER24> I don't know if we're talking "top 10 in the world" but I want to recommend Amy Hoy - our own user _ahoyhere_ \- <http://news.ycombinator.com/user?id=ahoyhere> \- <http://slash7.com/about/>

</comment><comment><USER25> probably Jason Putorti from Mint is on the list, so impressive, he got a designer in residence gig at Bessemer VC

<comment><USER26> Yeah, he's the one I'd mention. I like how Mint has their own widget in place of the <select> tag.

</comment></comment><comment><USER27> Top designers, or most famous designers? It's not the same.

</comment><comment><USER28> I have found the vast majority of my favourites through twitter. And they're amazing for showing off WIPs and sharing freebies with you. One of the below (I can't remember who) disliked the Wave interface so much they re-mocked it themselves just for fun.

Jonas Rask - <http://twitter.com/JonasRask>

Jonathan Castro - <http://twitter.com/DDrDark>

MetaLab Design - <http://twitter.com/metalab>

Philipp Antoni - <http://twitter.com/PhilippAntoni>

</comment><comment><USER29> Clearleft <http://clearleft.com>

Stuff they've done:

<http://silverbackapp.com/>

<https://addons.mozilla.org/en-US/firefox/>

<http://www.tourdust.com/>

<http://www.scifi.co.uk/>

<http://clearleft.com/made/wwf/>

<http://clearleft.com/made/ratemyarea/>

</comment><comment><USER30> What do you think of Fantasy Interactive (<http://www.f-i.com>)? Not trying to self-promote, just want to see what people think. We've done some big portals like <http://ea.com>, <http://events.nationalgeographic.com/events>, and <http://atari.com>, but also more experience-focused sites like <http://panamera.com>

I'm a big fan of the HappyCog crew, Information Architects, and Huge, to limit it to three.

<comment><USER31> Big fan of F-I work... I like <http://www.kontain.com/> especially

</comment></comment><comment><USER32> Not sure about top 10 but I really like Kyle Neath who now works at Github

</comment><comment><USER33> Douglas Bowman(stopdesign.com) and Dan Cederholm(simplebits.com)--IMHO.

</comment><comment><USER34> <http://www.thenobleunion.com> \- we work with them often. Excellent iPhone and web design. Think they are reworking their site.

</comment><comment><USER35> Naz Hamid has done some wonderful work. I particularly like the work he did here on a CMS project: <http://weightshift.com/design/aiga-cms-design>

I also think the work skinnyCorp does (<http://www.skinnycorp.com/>) on all their sites is fantastic.

</comment><comment><USER36> How has no one mentioned Nick La?

<http://www.ndesign-studio.com/>

Also, Dragon Interactive is pretty cool, although they can be mega douches at times.

<http://dragoninteractive.com/>

</comment><comment><USER37> Haven't seen the Sofa guys on here yet. Minimalist, Apple-esque app design: <http://www.madebysofa.com/>

My favorite interactive design agency: <http://www.2advanced.com/>

<comment><USER6> I wouldn't consider 2advanced to be a company that's good at UI... They're good at flash, I guess

</comment></comment><comment><USER38> One designer not mentioned on this list is Jason Fried of 37 Signals. Basecamp is very useful. Web App designers don't get the attention of flashy visual designers but after repeated usage of an application, you really get a sense of how well thought out it is.

</comment><comment><USER39> How about Ryan Singer at 37signals? Not only is he a great, functional designer but I think he's an awesome teacher, too.

<http://37signals.com/papers/introtopatterns//index>

<comment><USER12> My personal opinion is that he's very good, but not great. His designs work very well, but he never strikes me as ambitious or risk-taking in his work, which I feel is a must for great.

</comment></comment><comment><USER40> Might not be exactly what you were asking, but I like Envato.

<http://envato.com/>

Particularly, I like their themes site:

<http://themeforest.net/>

</comment><comment><USER41> My pick: \- Ryan Singer of 37signals \- HappyCog \- ClearLeft \- Unit Interactive \- Made by Sofa

Also, I think UI design is different from Visual design. I found that some of the posts below are referring to visual specialists only.

</comment><comment><USER12> Let me cast my vote for Shaun Inman, whose two web apps are brilliantly designed and brilliantly functional. His homepage is pretty dandy, too: <http://shauninman.com>

</comment><comment><USER42> As much as I really don't like the guy, I think Mark Boulton would be up there. His UX work for Drupal (some of which is at <http://d7ux.org>) is really top knotch.

</comment><comment><USER21> i think the names of the top 5, 10, or 20 designers aren't public because they are busy working on uis instead of promoting themselves on the web. it's all a matter of opinion. a lot of the people mentioned below have 1-2 things under their belt, excepting some of the more seasoned folks like bowman, vinh, and miner. the reality is no one person is responsible for a great ui. it's a confluence of talent. the only reason you know about any of these products is because the talent of multiple individuals produced it.

</comment><comment><USER43> I can't believe no one has mentioned: Jon Tan His taste is exquisite his websites are beautiful and functional.

<http://jontangerine.com/>

</comment><comment><USER44> Many designs on <http://csszengarden.com/> is very nice, and it has designer contacts there as well.

</comment><comment><USER8> Its impossible to accurately classify something subjective into a top X list. The lists will always be skewed into the tastes of the judges or list compiler.

Take music charts for example. The top 20 could be renamed, the '20 records that teenagers like at the moment'.

That said I would like to say that UI designers that work for one company who have one product, do not deserve the same amount of kudos as really skilled agency UI designers who work on upwards of 10 UI's a year. They solve 10 times more problems than those who work on one product.

<comment><USER7> Disagreed. I find that the agencies don't tend to "solve" problems, because they're not there when their solutions are measured/exposed to the real world. Agencies tend to take a stab at solutions. In house designers need to solve things.

<comment><USER8> Perhaps that is the case when the relationship is not right. In a lot of cases we run the sites for our clients and dont have to ask them to make UI changes. We spend a lot of time fine tuning and getting this part right. We do this on a permanent basis.

But I accept your point if this kind of relationship doesnt exist and the site is 'handed over'

</comment></comment></comment><comment><USER45> 30 top web designers on Twitter: <http://twitter.com/tor/status/6525201103>

</comment><comment><USER46> IMHO Top UI design <http://thesixtyone.com>, anyone knows the designer?

</comment><comment><USER31> Timothy Gray of <http://www.shelfari.com/> Great UI and UX design.

<comment><USER31> Few more apps that I consider great UI/UX design implementation... <http://www.foodspotting.com/>, <http://hunch.com/>

</comment></comment><comment><USER47> Aaron Sittig - <http://www.facebook.com>

<comment><USER7> Facebook is awesome user experience. The visual design is pretty decent, but it's the UX that's awesome.

</comment></comment><comment><USER48> surely off-topic and probably silly, but I was wondering where do all the designers get the pictures they use? is there a market going on for such thing? (I assume they won't shoot the photos themselves -- division of labor and all :D)

<comment><USER8> One such site is <http://www.istockphoto.com/index.php>

<comment><USER48> thank you very much!

could this stand as a sub-topic for top photo shops, branching the 'top UI designers' main theme? (yes, I'm trying realy hard to subvert this topic :p)

</comment></comment></comment><comment><USER49> Nobody has mentioned Khoi Vinh? Creative Director from the New York Times?

<comment><USER16> I mentioned him :) His personal site in subtraction.com</comment></comment>