Amazon is stuffing its search results pages with ads - <AUTHOR> https://www.recode.net/2018/9/10/17797720/amazon-is-stuffing-its-search-results-pages-with-ads <comment><USER3> Their basic underlying assumption appears to be that they're no longer in danger of losing customers and can now act accordingly.

Now that they've captured 50%+ of US ecommerce, they've also raised many prices, to the point where they're far from the cheapest price in many cases. A random example... the cat litter I just purchased is $17.89 at Amazon, $13.49 at Walmart. Last week I bought a chair mat for $21.95 on Ebay - identical item on Amazon was $33.95 (both including free shipping but Amazon came with an additional 8.25% sales tax).

They can do that because so many people don't price compare anymore before going to Amazon.

Not to mention the issues of buying an expensive brand item and getting a fake instead, as well as the large numbers of fake reviews. Indeed, it is pretty hard to get started in Amazon FBA these days when your competitors all have hundreds of glowing reviews for similar products and you have none.

They've also moved many items to the add-on category (no more free shipping unless you have $25+ in the cart, even with Prime).

My Prime membership expires in a couple months time, and this time I'm not renewing it, after being a member for several years.

<comment><USER4> This is why I've been unhappy to see Bezos gaining steam. As his other products become more successful, you can expect a lot more of this.

Alexa is not your friend, Alexa is a tool to make Amazon money. Amazon's move into media is not an effort to improve the media system, it's a recognition that there's a lot of money to be made by controlling the media that people consume. Same thing with the fire phone, it wasn't an attempt to make a better phone, it's an attempt to gain access to all of that juicy information that Google gets from Android.

Bezos is out to make an empire, everything else comes second.

<comment><USER5> Xyz company is not your friend. Ok, like we didn't know they are trying to make money off the other side? When is the last time a company trying to make a profit is our friend?

<comment><USER6> This viewpoint fails to consider the nontrivial competitive advantage gained by building customer loyalty. There are plenty of brands that do well despite higher prices, longer waits, etc due to their consistent positive interactions with customers; A culture of trust between customer and company is frequently worth the cost.

Amazon seems to have decided that the higher margins of counterfeits and prioritizing ads over search quality are worth the loss of user trust. I wholeheartedly hope this will be a costly miscalculation on their part, as I would prefer a marketplace with fewer ads and reliably genuine items.

<comment><USER7> I agree with you. Amazon's questionable inventory tactics have been a factor for me in deciding to sometimes shop elsewhere. Although, I don't mind the ads, they're selling screen space, I can happily scroll past them, the adds are often very relevant, and they are clearly marked sponsored.

</comment></comment></comment></comment></comment><comment><USER8> In some sense it feels like Amazon is progressing along a similar trajectory as Facebook:

1\. Start company with a minimally viable product

2\. Hire smart people to iteratively optimize the usefulness of the product to attract as many users as possible. Incentive structure rewards maximizing new users

3\. Product reaches "peak usefulness"; seemingly everyone uses it / loves it

4\. Re-focus business efforts to maximize revenue (e.g. stuff ads everywhere). Incentive structure now puts greater emphasis on rewarding short-term revenue increases

5\. Revenue climbs as usefulness declines; users begin to abandon the product

6\. Company continues revenue-maximization strategy because data on user dissatisfaction / abandonment significantly lags behind data showing correlation of "ad-stuffing" with more revenue

7\. User abandonment is finally apparent enough that management realizes it should do something; commence back-pedaling efforts to try to "win users back"

Seems like Amazon is approaching step #5 right now. Will be interesting to see how this plays out over another year or so.

<comment><USER9> The thing is that in 3) the company still doesn't make money.

<comment><USER10> Is that a hard rule though? Is it possible for a company to stay at 3 and make a profit? Is it pressure from investors that pushes them on to step 4, or the forces of competition?

<comment><USER8> I don't think it's a hard rule; it seems to apply to certain companies who pursue a strategy of deliberately not making any money at first and just acquiring as many users as possible to reach critical mass and ward off competition, and then after-the-fact slap on monetization practices (specifically Ads) that degrade the product and piss off users.

People don't like it when you mess with their products, especially once they already have something they really like. It's all about expectations.

</comment></comment></comment></comment><comment><USER11> Unless you know exctly brand/make/model you wan't to buy on Amazon, the search is useless. Hundreds of similar results from dozens of vendors of varying quality.

I liked it better when they sold things themselves, now they're just an eBay clone selling cheap knockoffs trying to hide the fact.

<comment><USER12> Even then it's borderline useless as when you search for the exact pair of trousers you want you'll now get other ranges, and dozens upon dozens of fake- brand cheap knockoffs.

When you do find the item, you'll find in amongst the "customers also looked at/bought" 4 or 5 other pages containing the exact same item. At different prices and stock levels. None of which showed up in the search you just did for that item.

It's completely broken and absurd.

<comment><USER13> You simply cannot buy clothes online. It's far too unreliable in every way. Even if they deliver what the advertise it's hard enough determining what size to buy when in person, the amount of work it takes to buy something int he right "size" online is astronomical. Easiest examples are shoes and pants. I can wear size 8.5 to size 11 shoes or 29 to 33 pants. These all fit the same.

<comment><USER14> I've found the exception to this is first-party webshops. For example, buying Patagonia gear from Patagonia.com is by far the easiest way. Their sizing is consistent across their line, and the website has a much better selection of colors and stock than any reseller does.

<comment><USER13> Yeah, that seems reasonable if you're unable to walk into a store. Even if the sizing is internally consistent it still seems difficult for a first time purchase though.

</comment></comment></comment></comment></comment><comment><USER15> I might actually be done with Amazon.

Every few months, I spend some time searching for the perfect LED light bulbs for my apartment; the built-in fixture requires 6 candelabra-base bulbs. These are hard to find, but there are a few products catering to this market, so I randomly buy stuff to see if I ever get anything good.

I found some LEDs that claimed to have a high CRI and were brighter than normal, so I took a chance on them. There were a couple legitimate reviews and 5 or more obviously-fake 5 star reviews. The LEDs turned out to be not that good. They were bright, the dimmability was good, and the layout of the internal LEDs was quite unique (corncob style but with some on the top, which is great). But they emitted a very greenish light, which makes them unusable. I have a spectrophotometer so I took a reading of the spectrum and, as you'd expect from the visual appearance, they emit very little red light (and no cyan light, as is the typical emission pattern of low-CRI LEDs).

I wrote a review covering these points, the positives and the negatives, and included the plot of the emission profile. This review was immediately taken down because:

1) Your review should focus on specific features of the product and your experience with it.

2) We do not allow profane or obscene content.

3) Advertisements, promotional material or repeated posts that make the same point excessively are considered spam.

4) Please do not include URLs external to Amazon or personally identifiable content in your review.

In other words: please do not include actual data in your reviews. What Amazon prefers is that the seller buys their own product under a fake name, and then says "5 stars: Great!"

This is probably my last Amazon purchase. I might do a proper write-up and spam it around the Internets and see if I can attract the attention of some higher-ups... but probably not. Instead, I will just shop elsewhere. B&H is way better for electronics anyway, and if you just want shitty LED lights eBay beats Amazon there.

Maybe I will also start buying every LED light I can find and doing proper reviews... but there is no business model there to make my time worthwhile. Reviews only work if you can actually find a product that's good and get people to buy it through your referral link. If every review says "you will hate this product, save your money", there is no way to get funding. So the fake 5 star reviews will continue, and everyone will continue to hate LEDs.

<comment><USER16> Downvoting this because it's irrelevant to the topic at hand (Amazon is stuffing search results pages with ads).

It's your right to free-associate all amazon gripes onto this thread, but I'll just use the system to indicate that they are noise to me.

<comment><USER17> The downvote is fine, but the comment explaining it is excessive. Different people will have different senses of what's relevant vs irrelevant. Although this specific article is about ads on Amazon, I'm personally more interested in the larger question of why/whether Amazon is shooting themselves in the foot with bad policy choices. As such, 'jrockway' was on topic for me.

<comment><USER16> Meh, when I'm downvoted I prefer to receive a justification. So I return the civility.

</comment></comment></comment></comment><comment> j-c-hewitt The ads have been there for a long time (I know because I've either spent or managed spending on millions of dollars in clicks). It's not even the first time that it's been written about in the tech press. The share of voice provided to ads on Amazon's search results is not particularly greater than what you see on Google or FB.

The headline ads are a little newer, but they're still more than a couple years old. On many Google searches you have to scroll past the fold to get past the sponsored results.

With search, unless you provide some avenue for sponsored results, you will get a winners-win-more effect in which the top 3 results for any given search will capture the entire market. You will get people complaining about 'discoverability' when they are not complaining about advertising. You can't stop people complaining about stuff: you can only trade around what people are complaining about today.

I will say that as Google has generally subtracted or crippled features that are useful to advertisers over the last several years, Amazon has generally kept things simple and effective. Also, virtually no customer information is exposed to advertisers through use of the platform. Everything about a customer is essentially exposed if you are the seller, however, but that isn't different from running a print catalog.

<comment><USER18> > With search, unless you provide some avenue for sponsored results, you will > get a winners-win-more effect in which the top 3 results for any given > search will capture the entire market.

If a seller really cared about this, they could implement something like a Robson Rotation for search results of relatively equivalent quality for the search criteria.

<comment> j-c-hewitt There are millions of SKUs. On any popular search result there are hundreds if not thousands of SKUs, not all of which are of equal quality. The organic search results (and partially the paid ones) already rank things and will often swap spots frequently during any 24 hour period if they're otherwise close to even in relevance.

</comment></comment></comment><comment><USER19> It's not hard to imagine that Amazon has created a situation where groups within the company are incented to drive up this high-margin ad income (no annoying logistics! no inventory!), divorced from the "external" fallout of ever-so-slowly turning the site into garbage.

eBay, have you anything to say about this? Or how about you, Hewlett Packard?

<comment><USER20> Ad revenue is easy to measure. Long term erosion of the trust of your users isn't.

Metric-driven decision making is not a silver bullet.

<comment><USER21> I really doubt that it's difficult to measure proxies for user trust. Return rates, conversion rates, search funnels, customer feedback ratings, basket size, prime churn rate, etc.

</comment></comment></comment><comment><USER22> Monopolizing one part of the value chain, enables you to leverage that monopoly into controlling the next.

Owning the railroads enabled standard oil to price it's competitors more, put them straits, and buy them up cheaply.

Owning the OS, Microsoft can make sure they own the Office Product productivity.

It's why the electrical grid needs to be separate from the energy makers (and regulated).

Amazon and Google as the 'front of the internet' can just suck the surpluses out of every other business; sure, you can still sell 'cereal and gadgets' \- but all of your surplus / profit margin will be sucked up by them.

... but I have a sneaking suspicion that something disruptive will come along before they entirely take over.

<comment><USER23> When did standard oil own railroads? I thought Rockefeller had large disagreements with the railroads and they had to build oil pipelines, and that in sense contributed to downfall of certain railroads too reliant on the oil money.

<comment><USER22> Owned or controlled and colluded with them in illegal/shady ways. It was a major source of competitive advantage.

</comment></comment></comment><comment><USER24> I think amazon is winning solely based on their physical infrastructure which they keep investing in, and hence the share growth. It has nothing to do with how good their website or customer support is, or how many ads they show or not show. All are this ideas they just trying without anything to loose.

<comment> village-idiot Their website is garbage for how popular it is. It's really hard to navigate the sections for controlling your kindle, digital purchases, or settings.

<comment><USER25> Whilst their website is terrible, I actually think the specifics of what Amazon does makes it very difficult for the website to be good. They sell a million versions of a million products in a million sectors. What is useful for me to know when buying a hard disk is not the same thing that is useful for me to know when buying a bottle of wine. So they either have 1 size fits all, or a huge cost in building almost custom websites for every different sector.

I'm not sure it's actually possible to do that well.

<comment> village-idiot If it's possible, Amazon has the resources to do it, they just clearly have decided not to spend them.

Fwiw I knew a guy who claimed that the templating language for the main website was written in C headers. I can't verify that, but if true it'd go a long way to explain why it's so bad.

</comment></comment></comment></comment><comment><USER26> Maybe also due to Amazon's tightlipped PR policy, it is not uncommon to read in comments about some article related to amazon such as this one accusations about systematically increasing and even user-specific "dynamically adjusting" prices supposedly to maximize profits.

I don't work there anymore since a bit, but I am somewhat annoyed by the unsubstantiated confidence with which these claims are made so I'd like to add my anec-data.

First of all, when I left the pricing system had in fact strong incentives in place to _lower_ prices as much as possible (for Amazon sold items. Third party sellers set their own prices.) Amazon would often make (small) losses on many items, especially the ones shown on the home page.

Moreover, there is definitely only a single price for each offer listing (which equals more or less an ASIN and merchant combination). This is evidently the case for 3P sold items, since as a seller you definitely do not have that infamous "auto-gauging" tick box the last time I checked. (And let's not forget, most of the growth of Amazon comes through third party sellers). As for Amazon sold items, I invite anyone to actually show evidence that the same offer listing has at the same time a different price for different users/devices, otherwise it is just empty talk.

</comment><comment><USER27> In the details page of the product, they used to show people brought this product also brought this list. Now that is replaced with sponsored products in most of the pages. They moved that list to bottom of page. IHMO this is deceiving users. There used to be lot of product recommendations and I find it useful. Now most of these recommendations are gone and they are replaced with sponsored ads. Not good for users.

<comment><USER28> The worst part is that they are visually similar. There's text at the top of the section that tells you whether it's sponsored or organic, but the layout and format is otherwise identical. If you recognize things more by shape and color and such as you're scanning the page (as I do), it's easy to not notice sometimes.

While nobody loves ads, I'm OK with them if it's always very obvious which you're looking at. This skirts that line.

Which is an unfortunate trend. Even Google's web search results page is making ads less visually distinctive. If I remember right, they used to have a different background color, and now they just have a small box that says "[Ad]".

</comment></comment><comment><USER29> Has anyone compiled a list of alternatives to Amazon? I'm not expecting a one- stop-shop where I can buy everything like at Amazon. But a list of online retailers who are better than Amazon at specific categories of products. Because I'm at the point where I would rather pay a little extra and get quality products from a not evil company than continue shopping with Amazon.

<comment><USER30> Just looking at my recent purchases:

Electronics - Newegg and Monoprice

Board & Table Top Games - Miniature Market

Pet Food - Chewy (owned by PetSmart now)

Clothing - Depends on your style, but I tend to stick to Eddie Bauer and Duluth Trading Company for good quality, affordable clothes

Comic trade paperbacks - <USER2>

Books - Barnes & Noble (new) and eBay (used)

The big one I've still struggling to find a replacement for is shoes. Zappos seems to dominate that market and other online stores have been hit and miss. DSW has probably been the best, but I'd love to find something better.

<comment><USER31> Can't you go direct to a brand's site for shoes, if you already know what you want?

For marketplaces/browsing, there are sites like ASOS, Gilt, Yahoo! Japan, eBay, etc. You can also browse any department store's online section to find something you like, and then order straight from the brand.

<comment><USER32> I actually had no idea I could by from the brands directly. Nike's store seems fairly decent.

</comment></comment></comment></comment><comment><USER33> This is one of the reasons I've let my Prime subscription lapse and started doing more shopping at Target. In addition to fewer (or no) ads, I can often pick an item up in an hour instead of waiting two days, and the prices are better.

<comment><USER34> In the UK, getting to a town center to do some shopping is so horrific that Amazon is the only remaining option.

<comment><USER35> Costs more to take the bus in to town than it does for most (slow) deliveries.

For some things there are no bricks and mortar stores left. I wanted test cables for a multimeter. 20p from China! (takes weeks mind you); I spent about £3 to get some delivered from a company in Europe within 2 days (probably drop-shipped from China) ... I can't think of anywhere in my UK city to buy them.

Bus for me and my toddler in to town and back is £2.80+£1.40 (it's about 3 miles).

<comment><USER34> For me it's a 20 minute car journey which actually takes an hour each way, and about $10 to park the car for a couple of hours.

</comment></comment></comment></comment><comment><USER36> They should get rid of ads for Prime members.

OR else, create Prime Plus for $0.5/month that shows no ads

Or else, raise prime by $6 and remove ads for prime members.

I know some of you are mad that I am suggesting they raise the cost of prime. But if you're willing to cancel prime over this, then you should be okay with that.

</comment><comment><USER37> Amazon's original strengths to me were fast shipping, low prices, and trustworthy reviews to sort items. I'm not looking forward to what they're turning into as they continue to get away from that old strategy.

</comment><comment><USER38> So do physical stores. Is it really so different from Heineken paying for prominent shelf space in the supermarket?

<comment><USER39> In Amazon's case, it's more like you walk in for Heineken and see three advertisements:

* Best Heineken (a Chinese no-name brand with 1,551 fake reviews)

* Top Rated Heinekan (a Chinese no-name brand with 6,731 fake reviews)

* Genuine Official Heienekan 16oz Can Beer Great Tasting Best Loved By Users Drink Up (a Chinese no-name brand with 5,532 fake reviews)

<comment><USER40> And then at the end of the shelf there's the actual Heineken, with the caveat that the grocery store doesn't buy from just one authorized distributor, but rather any distributor can walk in the door and drop off a case of what they claim to be Heineken to sell on consignment.

</comment></comment></comment><comment><USER41> > "At Amazon we work hard to continually invent new ways for customers to find > the right products to meet their needs. We take the same approach with > sponsored products and sponsored brands. We are focused on creating value > for customers by helping them discover new brands and products."

I hope everyone else see's whats happening with this statement - that famous "customer obsession" of Amazon's is now turned towards advertising buyers, who are now more the customer than you, the person purchasing the product. Now you are just a data point.

</comment><comment> m-p-3 Is there an addon of some kind that exists to make those ads more obvious (some kind of highlights) without outright removing them?

<comment><USER42> I am not a developer, but something like AdBlock or uBlock Origin should be able to at least highlight the "Sponsored" text to make it more prominent?

<comment><USER43> uBlock Origin appears to remove a lot of the sponsored results. Some sections like "our brands" or "expertly reviewed" may still appear above organic results.

<comment><USER44> I used the element picker, "##.acs-private-brands-container-background" seems to block the "Our brands" element.

</comment></comment></comment></comment><comment><USER45> I don't understand the "Amazon is eating into Facebook's and Google's business" argument, and I've seen it now over and over. Is there really a lot of overlap there?

It seems to me that the brands that advertise most heavily on Facebook and Google are the brands that don't sell much, if at all on Amazon. It seems that the overlap would be those products that are sold on Amazon and advertised via Google or Facebook, which I don't think I've seen much of. To me this seems like it would only make selling on Amazon less of a meritocracy and more of a competition for the deepest pockets.

But even if it's true that there is overlap, then suppose a company cuts its Google ad budget to focus on Amazon. Then all searchers who still use Google will now go to their competitors instead. So why would they do that? If Amazon's ad business is growing it seems likely that it's coming at the expense of the sellers more than the other ad platforms.

</comment><comment><USER46> Google trained me long ago that anything sponsored at the top is a scam. I now skip below the ads without a second thought.

</comment><comment><USER47> I chuckled a bit when I realized the only thing I buy from Amazon anymore these days is... books!

Everything else I no longer trust Amazon for. Hell, even the books might be counterfeit: [https://news.ycombinator.com/item?id=13924546](https://news.ycombinator.com/item?id=13924546)

<comment><USER48> Same here, which is why I cancelled my prime membership. You can get almost any book you want slightly cheaper on abebooks.com (also owned by amazon).

</comment></comment><comment><USER49> This is of course not the least bit surprising. The search box on Amazon is a pretty high value place, when someone types something into it they have _already_ indicated they are looking to buy something so commercial intent is well established. Amazon has more third party sellers on their site now than Amazon, so there is a viper pit of competitive selling seething down there. The next step? Let those vendors give up margin over to Amazon by bidding to be in search results.

More money for Amazon both in ad revenue and additional sales (for all its evilness advertising exists because it works).

Interesting question to see how Walmart responds.

</comment><comment><USER50> I went on the other day to buy a specific brand of pants. I was in the app. I searched, clicked the first link, seemed like it was coming from a third party seller. Selected the size I wanted and ordered. It arrived and was a completely different brand. Went back and looked at the search and it was an ad. What I thought was a third party seller's store name was the brand name. From this experience I learned that I can't trust their search results. The mechanism for finding what I want will also now give me things I don't want.

</comment><comment><USER51> As Amazon's ad revenues increase, I expect them to gradually drop their prices, making it EVEN more impossible for other online retailers to compete.

Amazon retail's profit margin is 3.8 percent, on more than $51 billion in sales in 2016. They're probably thinking they can shrink this all the way down to 0, and make up the difference in ads.

The big question for me is when they will get into the search space. The growth of their ads business and Alexa's success is a massive threat to Google long term.

</comment><comment><USER52> The HN crowd are probably 5x more savvy to this sort of thing than the average user. Amazon is the Wal-Mart of e-commerce, it now has generic brand association with a large majority of consumers. So it's working for them, and it's going to keep working for them even if every HN reader shops elsewhere. It will take some serious competition to get them to change their behavior, and I don't think Jet is going to get there, unfortunately.

</comment><comment><USER53> I live in Europe, in a country without Amazon, however I often travel to the UK so I've had a few opportunities to use their services.

My experiences have been very mixed. The website, in my opinion, is a mess. It's extremely hard to find things unless you know specifically what brand and model you're looking for. Even then you might find that someone decided to sell that item for an inflated price.

Generally when you're browsing for something you want but haven't decided on an exact brand and model yet then discoverability is terrible and listings become irrelevant pretty soon after scrolling down. Even trying to find a gaming laptop for a friend to buy is a useless endeavour.

Shipping within the UK also seems to be terribly implemented: more often than not your packages are going to get lost, misdelivered, forcing you to fight for the goods you've already paid for and potentially get them way too late if you want them by a specific date. In August, when I was visiting, they managed to misdeliver 4 items I ordered and it's always Amazon's delivery team that messes up.

The only reason I really like them is the book market and second-hand book market. It's great to be able to buy so many books at such great prices.

In my own country, however, e-retail seems way more sophisticated: discovery, delivery and reliability are king. They optimise the listings for you to be able to browse and find things efficiently, to discover stuff you want quickly. And it's very rare for the shipping to take more than 1 day, unless there's a seasonal event like Christmas/New Year's. So I buy with confidence and I browse knowing that I'll usually find what I'm looking for. When it comes to buying from Amazon while I'm visiting the UK - I always expect issues. And they always happen.

</comment><comment><USER54> That's very interesting, it looks like there is a natural tendency in private market to go to corruption. We see it where retailers ask money from the factories to put the product in some height on the shelf. Online we see it with "search result" placement. We see it in facebook where if you don't pay, they reduce the number of people who will see your post.

</comment><comment><USER55> I consult for some of Amazon's biggest sellers. They absolutely have to pay for these ads to compete. Amazon fees all in can easily reach 25% of the retail price, and that's assuming you're paying very close attention. They skyrocket if you decide to take a pee and leave your analytics dashboard closed for a few minutes!

</comment><comment><USER56> I don't think it's simply the lack of consumer price comparison that is making it easy for Amazon to front ads without consequence. I think it's also that for many different consumer products, consumers don't know precisely what they want, or more particularly, how to model their requirements. If they knew their precise requirements, they would easily be able to judge whether or not a given ad was fronting a product actually is the right one for the job (that sometimes happens), as well as whether or not the products were a good deal.

It would be interesting if there was a repository of the performance specs for different common household products. Right now it's the job for critics and reviewers to be the experts here, but how difficult would it be to simply read a five minute primer on the theory of detergents or the theory of headphones and then go out shopping with impunity?

</comment><comment><USER57> Regular supermarkets have been doing this forever, they just aren't upfront about it.

[https://www.vox.com/2016/11/22/13707022/grocery-store- slotti...](https://www.vox.com/2016/11/22/13707022/grocery-store-slotting- fees-slotting-allowances)

</comment><comment><USER58> Do you guys realise that Amazon just acts as a marketplace now, and the products being sold are from third parties?

Now that they've captured 50%+ of US ecommerce, they've also raised many prices, to the point where they're far from the cheapest price in many cases. A random example... the cat litter I just purchased is $17.89 at Amazon, $13.49 at Walmart. Last week I bought a chair mat for $21.95 on Ebay - identical item on Amazon was $33.95 (both including free shipping but Amazon came with an additional 8.25% sales tax).

Amazon is NOT raising prices, its the merchants who sell on Amazon who are raising prices. The main value that Amazon provides is fulfillment, and driving users to these product pages from third parties.

</comment><comment><USER59> Amazon is the first trillion dollar company at the same time I went out of my way to order shirts and guac mix from walmart.com because I have way more faith in products I find in the page or two of results I get there than from the thousands on Amazon, 99% of which are buried in fake reviews for fake products.

And I do it substantially cheaper. I even checked this week when I was in a physical store to compare prices and I was paying the same for the freely shipped stuff on the website as I would have going into a location. Competitive Amazon products had at least a 30-40% markup, even for the same brand.

And thats all on top of knowing how reprehensible a company Walmart is, but its not like Amazon is any better nowadays. At all.

</comment><comment><USER60> Another factor I've seen helping amazon is the fact that their amazon gift cards are basically being used as a form of currency. I often trade stuff online and everyone often uses amazon gift cards. The reason is because you can redeem an amazon gift card immediately to your account to make sure you don't get scammed. With most other gift cards, even after the trade the other person can still use the gift card.

I'm surprised other companies haven't realized this. If people are able to use your gift cards as a form of currency then it really helps the overall brand because eventually people are going to buy stuff with the gift cards.

</comment><comment><USER61> Can someone explain why Amazon works so hard to direct searches into searches within its product categories? An example would be a search for 'Baking Soda' leading you to the 'Grocery & Gourmet Food' or 'Industrial & Scientific' categories. I'd much rather search for 'Baking Soda' and have it return every kind of 'Baking Soda' Amazon carries, as I want the best price and don't care how it's been categorized by the retailer. It's almost always faster to search for anything sold Amazon via a Google search outside of Amazon's own search engine.

</comment><comment><USER62> This is THE example of why any website that has "relevant" or "best" as a filter type is absolutely shit...

I have really loved Amazon but their practices lately to drive growth are driving me away. My Prime membership is coming up soon and this may be the last year I renew if I do at all.

Ads are half the problem, counterfeit goods are the other... I actually could careless about things being counterfeit but it's a risky proposition with electronics and often risky...

As long as they got Gran Tour I'll probably subscribe... because Clakrson, Hammond, and May are worth the $99 a year alone. Everything else... not so much.

</comment><comment><USER63> I came to the point that I am actively avoiding Amazon for any purchase.

It happened multiple times that they give better price to someone coming to Amazon from the search engine in incognito window than to their logged in Prime member searching for a product directly on Amazon.

Now I always use günstiger.de or idealo.de or some other meta search engine to find what I need, as well as many other people. The same way internet search engines killed big portals back in the time, Amazon is going to be killed by specialized search engines, though it may take a while.

</comment><comment><USER64> At what point does this become semantics. Isn't a sponsored ad with strong context (including the context of the search) simply a paid search result? The issue is that there are competing masters, one that the ad wants to get served and two that the person wants to find the search results most relevant to their context, but amazon is attempting to provide a paid option that meets that context first, and if it meets that context slightly less and is paid, then thats a sacrifice they are willing to make.

</comment><comment><USER65> This seems like a huge chance for some company more thoroughly obsessed with item quality to wade through the muck and offer some value in terms of product reviews, since reviews are being gamed.

Seems like lots of free time to buy and try products in various niches, an easy to use browser extension and an extensive set of affiliate links, and a coupled website (probably not youtube since negatively reviewed companies might just copyright-strike) are all it would take...

<comment><USER66> you're describing Wirecutter and its many clones

<comment><USER65> A little late, but this just means that it's a good idea and people are doing it and making a living off of it...

But I don't frequent sites like Wirecutter because there's too much editorializing for my taste. I feel like the model could be tightened up so much more.

</comment></comment></comment><comment> trevor-e I was amazed the other day at how difficult it is to find products on Amazon now. I knew exactly which product I wanted to buy beforehand too.

I was trying to buy "BSN Syntha-6 Edge Protein Powder". I dare anyone reading this comment to find a 2-pound jug of cookies and cream flavored whey protein (not Micellar Casein or Hydrolyzed). I found it on GNC's website very quickly and bought it there.

</comment><comment><USER67> _> Nearly 8 percent of views on Amazon product pages came from sponsored links in May, more than double what it was a year earlier, according to data from analytics firm Jumpshot, which collects URL data from a panel of 100 million people._

It is fitting that an article talking about increased number of ads is using statistics from a company that collects data for ads targeting.

Guess why the ads are working so well?

</comment><comment><USER68> Sooo, just like Google? If these ads become commonplace, savvy users will start to ignore the "first x results" and Amazon will make click/ad money on the the same people that click Google Ads.

While we all wish a competitor (just a click away! /s) would arise to compete, we all know from our experience with Google that lock-in is real.

<comment><USER69> > While we all wish a competitor (just a click away! /s) would arise to > compete, we all know from our experience with Google that lock-in is real.

In the case of Google search, it is. Duck duck go is good enough to be my daily driver.

</comment></comment><comment><USER70> What really kills me with Amazon is that, leaving outside of the US, I always tick the box for "Deliver in $COUNTRY", and regularly (95% of the time) find out at checkout time that no, in fact, this seller does not deliver in $COUNTRY.

To be fair, I gave up on Amazon after that, except for Kindle's books.

</comment><comment><USER71> If someone needs a project, please develop an "Amazon Advertising AdBlocker".

Their class names are fairly consistent.

<comment><USER72> If they are consistent you can probably use your ad blocker's element picker to selectively remove them.

</comment></comment><comment><USER73> Gone are the days when you go to Amazon.com for bargain shopping or in search for better deals. I look at Amazon's Prime service as a shipping company. I like 1-2 days shipping service and sometimes I would pay a buck or two more for a product of Amazon Prime can ship it.

</comment><comment><USER74> Honestly, they are screwed once people start using ali express instead. I know shipping times is a bit long, but I'm sure there will be some solution to this in the future. I wouldn't be surprised they built their own fulfillment centers on the continent.

</comment><comment><USER75> The article made it seem as though sponsored product search results unique to just amazon and the other big players. Walmart does it - although not as well. Criteo has a product thats used by a lot of large online retailers that offers this as a service.

</comment><comment><USER76> I wonder if there is a way we can block these. This is getting really irritating especially in the electronics category where there is a ton of Chinese off market brands which take up a lot of space and then you get to the ones you need

</comment><comment><USER77> It's so weird, supermarket always sold better placement to some product and it was never an issue. Amazon is clear in their doing too, is that the issue? You don't notice it thus it's alright?

</comment><comment><USER5> The uptick could be temporary because it is catching people off guard with rhemused to seeing most relevant result in top. But the allure of the exposure will always be there even after people smartens up

</comment><comment><USER78> I wonder if it would be cheaper for small-time sellers to cozy up to brick and mortar store representatives than it would be to fight for space on Amazon.

Does Amazon lose market share... back to brick and mortar?

</comment><comment><USER79> For me Amazon became the cloud & entertainment provider, I have stopped using the store in the US after a while. Said to see this degradation of service that Jeff Bezos was so proud of once.

</comment><comment><USER80> What tools (if any) do people here use for price comparison across many sources when looking for the cheapest option for a given item/SKU? What about when you are looking for quality?

</comment><comment><USER81> >"Nobody is scrolling beyond the first page when they do a search," Jason Goldberg, SVP of commerce at SapientRazorfish, a digital marketing agency, told Recode. "If you want to be discoverable, you have to find a way to show up in search results."

Bullshit. Amazon search is sketchy and reviews (both number and quality) are often unreliable, so I spend far longer than I want to looking through reviews to spot the fakes and try to find products that are actually of good quality.

I've actually started buying important things from elsewhere again. Many companies on Amazon are in a race to the bottom in both price and quality. That's fine for some products, but not for anything that's going to get regular use.

</comment><comment><USER82> The early adopters (e.g. us techies) are leaving. Do companies have metrics for who are these people are, do they care that we form such very small numbers?

</comment><comment><USER83> Google has been doing this for years. I installed an ad blocker specifically to clean up Google search results, they look much better now.

</comment><comment><USER84> Sorry, but all this hate, how is this any different than your local grocery store charging brands for higher placement on the shelves?

</comment><comment><USER85> Anyone else having the experience on the iOS app when trying to tap on a search results keeps taking you to the Echo product page?

</comment><comment><USER86> Physical stores have done this for years with shelf space and displays. Can someone explain why this is a big deal?

</comment><comment><USER87> I can't find anything anymore. I was looking for Fitbit straps and the genuine ones were many pages in.

</comment><comment><USER88> And here I was wondering why Google or another search engine was required to find what I want on Amazon.

</comment><comment><USER89> Arrogance and stagnation make for a bad combination. I hope Amazon can shape up soon.

</comment><comment><USER90> "Amazon is stuffing its search results pages with ads "

Like Google, you mean?

</comment><comment><USER91> Damn it people, stop messing with organic search...

</comment><comment><USER92> I remember when they used to be a great bookstore.

</comment><comment><USER93> is it better than the search without ads?

</comment><comment><USER94> First comment, but god damn do I live this site.

Learning new shit everyday.

First I end up switching back to firefox, now I get to hear stuff like this! :D

</comment><comment><USER95> wow, revealing how the internet works

</comment><comment><USER96> Amazon doing some questionable things off late like

1\. Removing prime subscription renewal notifications. 2\. Injecting ads into search and not differentiating them much with results. 3\. Replacing different recommendations in the details page with ads.

Unfortunately you won't notice them if you are not keen. I recently contacted their CS about a fake product. The reviews clearly says it is some cosmetic product the product itself is a dashcam. They did not do any thing about it. They said they get back to me about it and never did. Not sure why they are turning their back on these things. Its definitely not customer obsession !!</comment>


Restaurant website traffic now 40% mobile - <AUTHOR> http://www.happytables.com/news/restaurant-website-traffic-now-40-percent-mobile/

<comment><USER2> I don't get why many restaurant owners don't bother putting a decent website online. If they actually do have a website, most of the time it's outdated, ugly, doesn't offer any useful information (such as opening hours, specials or a menu) or is just completely unusable. I'm not even getting started on any 'fancy' stuff such as usability on mobile devices or online reservation.

I know a few restaurateurs and they simply don't care, even more so than other types of small brick-and-mortar shops. First, they operate on very thin margins and most of the time simply can't or don't want to afford extra services such as a website. Moreover, most of their first-time customers still get there by word-of-mouth. It's as if most of the sector marketing-wise is still pretty much stuck in the early nineties.

<comment><USER3> As much as getting a website up and running for a small business seems _so easy_ for the median reader of this site, the reality is different. Restaurant owners don't know how to do it. Regardless of the software you put in front of them, they probably won't come up with a good result and chances are that if they select the software themselves it will be a poor choice.

The internet is full of bad advice and the phone book is full of bad consultants. It's actually quite hard to be a good consultant to this kind of business. They're hard to work with and aren't willing to spend very much.

</comment></comment><comment><USER4> It's so sad/frustrating when you need to look up a restaurant's page on-the-go and then you find out it's flash-only.

<comment><USER5> Or the PDF menu is embedded in an iframe, making scrolling a pain.

</comment></comment><comment><USER6> This is an interesting market that you are targeting, but I'm curious what your revenue looks like. It looks like you are banking on people upgrading to a premium account to make money, but to me it seems like you are giving away fully functional websites for free. You are giving so much away that there really isn't an incentive to upgrade.

> There is no setup fee or minimum term (even for our free plan).

>Can I stay on the free plan forever? Definitely. If you operate a small business or are simply growing, we want to support you. Our premium features are aligned to those of a larger restaurant, so once you're more established you'll have the ability to upgrade, but it's all up to you!

and you don't put ads on the restaurant's site either, so there is no revenue coming in to you on that end. Maybe I missed something when I was digging around?

<comment><USER7> Hi Ryan, so there are number of restrictions on the free website, but the main one they end up "converting" on is the "powered by happytables" logo at the bottom of each page (in other words, removing the branding).

</comment></comment><comment><USER8> 46% of apartment traffic is now mobile as well. I work for a company that hosts 10,000+ apartment websites and pulled the google Analytics that we aggregate across all of them.

</comment><comment><USER9> i am frustrated by the number of restaurant websites that simply _don't work_ on iOS at all. Like you can't even look at a menu or figure out the hours.

Of all the websites I look at, restaurant ones are the most likely to fail horribly on mobile.

I guess it's because restaurant owners/managers 'buy' a website from a small contractor, and what they look for and are willing to pay for is something that looks 'pretty' \-- lots of flash, lots of big graphics, etc.

<comment><USER10> You mean they don't work because they use Flash? As much as I hate saying it but if you did not choose a mobile OS that limits your choice of software, you could simply install flash. You cannot really blame the restaurants for using technology that was hailed for its interactivity by some people and webdesigners for so long.

<comment><USER11> >As much as I hate saying it

It sounds as if you love saying it.

Anyway, I agree with gp: so many restaurant websites seem to focus on being cool more than on providing the information I need: address and menu. And yes, Flash is a big part of this but even if that worked on iOS, these websites would still be annoying.

<comment><USER7> That's exactly our goal with Happytables, focusing on: accessibility, information and engagement. All the additional fluff can really bog down those sites.

</comment></comment></comment></comment><comment><USER12> Doesn't everyone just use Yelp? Gives me review highlights, hours, and often the menu too not to mention undoctored photos from other customers.

</comment><comment><USER13> I presume that to use "connecting to domain" feature a restaurant owner would need to alter DNS records, which does represent an extra hurdle.

So my questions are:

1\. In reality is utilizing this feature a real hurdle?

2\. Any other way to do this w/o mangling DNS records?

<comment><USER7> Founder here. Indeed an additional hurdle! It's probably the hardest part of setting up the website. Handled via CNAME (similar to Shopify), this way they can still control any other services (i.e. e-mail).

</comment></comment><comment><USER14> Man... my stupid childish head... I can't unsee the "butt cheeks" in their favicon now</comment>


Why is infrastructure as code such a big deal? - <AUTHOR> Honest throughs and opinions  on the subject <comment><USER2> Reproducibility and peace of mind! There's nothing worse that getting woken up at 3:30 AM and realizing that it'll probably take a few hours to rebuild a server because it's dead and not coming back.

When infrastructure is code, you just spin up a new machine...no thinking involved. Also, since you're used to deploying this way, the process is smooth and as quick possible.

When you build the server by hand...you're often searching around for old ssl certificates, installing packages you forgot were dependencies, etc. Takes a long time to get it right.

If you're skeptical of the cost/benefit of the approach, you realize really quick that it's worthwhile when stuff starts to fail. Especially if it happens more than once.

</comment><comment><USER3> Another benefit that hasn't been mentioned in the comments is that it serves as documentation. As new developers come into your team, you need to describe the infrastructure setup. Having your infrastructure defined in a clean repository is one of the best ways to do that.

<comment><USER4> Well, IaC is way better than nothing but it isn't a substitude for documentation. I had to learn an AWS CloudFormation codebase and although it's pretty clean it's still loaded with quirks (for how stuff worked years ago and to workaround shortcomings of our application).

Most IaC describes how something is configured, you should also describe why something is configured. Most IaO platforms allow you to write comments, make use of it!

</comment></comment><comment><USER5> The alternative is setting it up by hand which is error prone. When you're starting out, that might be OK but it'll grow and the difference between your staging and production accounts will drift.

Also, then let's say you get a mega-enterprise customer that wants to host it in their own cloud account, you simply apply the same configuration to that new account. It's quite something to see thousands of cloud resources created over the space of an hour, thinking how long it'd take to do it by hand.

<comment><AUTHOR> great thank you

</comment></comment><comment><USER6> I think consistency is a big benefit. Having the infrastructure described as code means that configurations are recorded in a uniform way (as opposed to just clicking around in a web interface to spin up resources and _maybe_ documenting it, for example). You can also check this code into source control and have a handle on how changes were made over time, which can be useful if a problem occurs and you want to isolate the change that broke your setup.

<comment><AUTHOR> thank you!

</comment></comment><comment><USER7> Short:

The old way straight up setting up a new server could eat hours of time, let alone the network, etc ... now I can copy and paste.... and I know there are no variations, someone didn't miss a setting, and so forth.

That's a crazy big cost saving.

Longer:

Physical hardware is hard.

I worked with data center networking for a long time and the efforts to generic-ify everything enough to be predictable was endless.

Eventually you have so much identical redundancy and when VMs came around you just wondered ... "wait this has to be able to go farther" and here we are.

</comment><comment><USER8> Infrastructure-as-code is just automation. Automation eliminates the capacity for human error which can occur wherever a person manually follows some steps that might be read wrong or recalled incorrectly, and it eliminates human bottlenecks like waiting for a person to set up a server, waiting for a person to update documentation, waiting for a person to publish new packages etc. I think human error and human bottlenecks are the root cause of a lot of stress in our industry and with automation we can minimize this burden on ourselves.

</comment><comment><USER9> One cool thing about it is that, if the infra is just a bunch of k8s pods, you can efortlessly create any number of per-developer environments using the same deployment scripts as are used for production deployments (ideally, you just change address of the k8s cluster and some environment-specific stuff like secrets), and these environment are guaranteed to be extremely production- like.

</comment><comment><USER10> It's kind of like the saying "Teach a man to fish...". You can build a server once, but let's say one region in your hosting service goes down, can you really build a new server or cluster for that matter from scratch in time before your users notice?

</comment><comment><USER11> We delegate to developer to make modification on the infrastructure even they don't have the access right to the servers. The change can also be reviewed by all the parties.

</comment><comment><USER12> Because it sounds like something new and shiny and something you can boast about. When really the basic ideas behind it are as old as the hills.

<comment><AUTHOR> haha nice</comment></comment>


Democratizing Innovation (free book from MIT Sloan School of Management) - <AUTHOR> http://mit.edu/evhippel/www/democ1.htm <comment><AUTHOR> good feedback... I'm reading thru it right now and have found there is great stuff there... I'm half way thru already!

specially for entrepreneurs I found that knowing the way people go around your products and providing the tools for people to take them to the next level is key nowadays.

there is so much a startup can make, but by providing the right features for the majority and a platform for your lead users and crafters to build upon, makes you set your self as the founding platform while focusing on the long run and keeping the boat going.

somehow I feel this is a shot to the close environments that the Apple's of world are putting around products like the iDevices... build great products but also give the chance for others to innovate freely on top of that!

</comment><comment><USER2> This is a great book.

Personally I'd be put off by the mention of a School of Management in the headline, it's not really about management. It's related to product design and how to learn from your users.

Or rather, how to learn from people who are nearly using your product, or using your product in extreme ways. These "lead users" are the (potential) customers who are going to tell you something more interesting than "faster horses" when you ask them what they want.</comment>


Wall Street's Repeat Violations, Despite Repeated Promises - <AUTHOR> http://www.nytimes.com/interactive/2011/11/08/business/Wall-Streets-Repeat-Violations-Despite-PromisesStsssss.html

<comment><USER2> The background here is that the SEC is currently seeking the power[0] to impose fines without judicial review:

      Even as Judge Jed S. Rakoff of Federal District Court in
      Manhattan rejected the Securities and Exchange
      Commission's proposed settlement with Citigroup because he
      did not find it in the public interest, the agency's
      chairwoman, Mary S. Schapiro, sought to enhance the
      S.E.C.'s authority to impose higher financial penalties to
      deter securities violations. An important part of her
      proposal would allow the S.E.C. to seek the increased
      penalties in administrative proceedings, a move that may
      allow the commission to avoid having to pursue its cases
      in federal courts in the future, skirting the kind of
      scrutiny Judge Rakoff applied.

This article is a press hit[1] by the SEC's PR folk[2] which encourages people to call for stiffer penalties against those evil firms. Bingo, SEC gets a powerup. The "grassroots" outcry means they can now just ticket & fine companies without the meddling checks and balances of the federal courts system.

The SEC's fines have about as much to do with preventing a financial meltdown as local police traffic fines have to do with preventing a murder. It's just safety theater.

That is, everyone is familiar with the local/state government phenomenon of cops who ticket to make quota at the end of the month. Whether you are "breaking the law" today depends in large part on whether state/local government needs to make money off you. Tickets for expired registration are profit centers, robbery investigations are cost centers.

The federal government is subject to many of the same dynamics, except that it isn't called on the carpet in the same way because federal agencies can intimidate those with knowledge of the situation into silence. Once in a while data comes out on the corruption and malfeasance of (e.g.) FDA[3] or EPA[4] or SEC[5] bureaucrats, but for the most part they have the guns so they are dishing out the punishment.

The NYT's role in all this is to act as stenographer. There is no summary of how much the SEC sought to earn in fines from pursuing these prosecutions, nor how many of them were overturned or thrown out of court.

[0] [http://dealbook.nytimes.com/2011/12/05/s-e-c-seeks-more- powe...](http://dealbook.nytimes.com/2011/12/05/s-e-c-seeks-more-power-but- does-it-need-it/)

[1] <http://www.paulgraham.com/submarine.html>

[2] <http://www.sec.gov/news/press.shtml>

[3] [http://www.washingtonpost.com/business/economy/fda- chemist-p...](http://www.washingtonpost.com/business/economy/fda-chemist- pleads-guilty-to-insider-trading/2011/10/18/gIQA6wwyuL_story.html)

[4] [http://epw.senate.gov/public/index.cfm?FuseAction=Minority.B...](http://epw.senate.gov/public/index.cfm?FuseAction=Minority.Blogs&ContentRecord_id=2197064b-802a-23ad-49a0-d86c2080b4f8)

[5] [http://abcnews.go.com/GMA/sec-pornography-employees-spent- ho...](http://abcnews.go.com/GMA/sec-pornography-employees-spent-hours- surfing-porn-sites/story?id=10452544)

<comment><USER3> I would add the the SEC is a notoriously corrupt organization in that they need the banks help in order to understand what is going on AND people who work for the SEC tend to find their way into cushie bank jobs. The idea of giving them more power doesn't seem to me like any kind of solution at all.

The real problem is that people just don't care enough about white collar crime. When someone mugs you for $5 you want vigilante justice, but when billions (trillions?) of dollars are on the line, we are happy to just shrug our shoulders.

<comment><USER4> Who said people were happy about it? I think there's a lot of anger at banks at the moment for perceived corruption. But there's a huge gap between voters caring and measures that are actually effective being put in place with lots of lobbying in between (and I have no idea what measures would be effective; and many people that do probably are also in a position to get a cushie job helping to avoid or lobby against those measures at a bank).

<comment><USER5> Punitive damages on a geometric scale for repeat offences in a 10 year period? 3 times profit made plus any damages for first re-offence, 9 times for second re-offence. Sounds similar to training a child to go to sleep on their own...

<comment><USER4> I'm guessing the devil is in the detail (and I don't know much so I'll shut up shortly) - could companies set up shell legal entities to offend for them / shop around for a better jurisdiction to do dodgy deals in / is current law clear enough that serious penalties wouldn't end up having a chilling effect on all business (OP was implying that the current status is that if the SEC wants to, they can probably find some law you're breaking). Serious penalties would probably demand a lot of legal boilerplate to make absolutely sure 'good business' wasn't effected and the government doesn't have power to destroy politically inconvenient companies without due process; the more boilerplate, the more likely the more expensive lawyers at the banks will outwit the lawyers writing the laws. Add some (possibly sponsored) political rhetoric about regulations killing jobs in a recession and I think it gets very hard to actually nail it down.

</comment></comment></comment></comment></comment><comment><USER6> Mitt Romney says corporations are people. Ok, time for a "three strikes" law for Wall St.

<comment><USER7> Perhaps if more of them received the death penalty, it would make the ones obeying laws look better???

<comment><USER8> As long as you also include labor unions. Many people want laws that only effect corporations, when labor unions are pretty much the same thing, have just as much power, and can be just as corrupt.

<comment><USER9> I'll grant unions are very corrupt, but there's no way I believe they have just as much power. I'll believe that when top execs at labor unions pull in as much money as public company CEOs.

<comment><USER10> > there's no way I believe they have just as much power. I'll believe that > when top execs at labor unions pull in as much money as public company CEOs.

It seems silly to equate organization "power" with "how much does the top guy make".

For example, Obama makes less than the president of GM. (Heck - Obama makes less than the president of the UAW.)

</comment></comment></comment></comment></comment><comment><USER11> I'm not sure what this is supposed to prove. Say you're the CEO of one of these companies and you are absolutely convinced that you don't want any of your employees doing any of this stuff. How do you actually accomplish that?

<comment><USER12> You can never do this 100% reliably, but depending on how important it is to you, there are measures that can be taken.

To go to the complete extreme, you could employ someone to shadow every one of your current employees, and report to you if they break any of these rules.

At the other extreme, you could issue a memo saying "Don't let me catch you doing any of this stuff, which will make you more profitable and earn you a bigger bonus, hint hint."

In between turning a blind eye and adding ridiculous amounts of oversight, the CEO can make very clear what is and what isn't acceptable, what the penalties are for employees who break these rules. S/he can also instigate appropriate oversight to make it more likely to catch offenders.

Like I said at the start, this can never be 100% reliable, but it can cut down the rate of offence a lot.

[edit] tl;dr : If it is important enough to you, you'll find a way to ensure it doesn't happen.

</comment></comment><comment><USER13> The political class bought votes with so much borrowed money that they finally could not repay, now they want us to blame the people who give them the money. Like the alcoholic who blames the bar tender for his car crashes.

<comment><USER9> Huh?

</comment></comment><comment><USER14> This article is nothing more than yet another example of shameful class warfare perpetrated against America's Job Creators.

<comment><USER15> And who would these "America's Job Creators" be? The banks? I don't suppose you can provide any kind of non-Fox news citation for this nonsense?

<comment><USER14> not without the same sarcasm I think you missed !

</comment></comment></comment><comment><USER16> Funny that SEC asking for a law to avoid having to follow the US Constitution</comment>


"I've isolated the bug to a database query" - <AUTHOR> http://thedailywtf.com/Articles/The-Query-of-Despair.aspx <comment><USER2> One company I was at did a merger with another startup, and most of the other company's engineers quit. Amongst the piles of Visual Basic we found a stored procedure that was about 5 pages long. It took about 18 hours to run; its job was to do a daily report.

I hate being afraid of code. I spent a day with it, got to understand it, then rewrote it as a couple of queries and some Java code, whereupon it took about five minutes to run.

[... then there was the guy who implemented bitwise AND and OR by precomputing some 65536 entry tables. Wow. Why do I find all the really howling bad stuff so close to databases?]

<comment><USER3> "Why do I find all the really howling bad stuff so close to the databases?"

Because the database and queries against were were coded as an afterthought.

Because the programmers ran all the db access tests against tables with 4 rows of data when the customers would windup with 100 million rows in production.

Because most undergrad education around databases is poor and antiquated. Plus, it is not SQL focused - much time is spent on talking about data modeling. Not wrong, but not often helpful in practice.

Because DBA's are (rightly) focused on making sure your db is properly backed up, replicated, and fault-tolerant. They may not even understand SQL queries beyond the basic one table select.

Because the users will force data into the systems that the developers never anticipated.

Because db jobs are segregated by product for _development_. If you've done SQL coding on Oracle for the last 10 years, good luck applying for a job doing SQL coding on MS SQL Server.

Because SQL hacking doesn't get much respect. I've been in job interviews where deep into the process they ask me a question with data in two tables that requires an outer join. And answering correctly differentiates you _significantly_ from other applicants. I guess it is the "fizz buzz" of SQL these days . . .

<comment><USER4> I work with large databases every day. I cannot tell you the amount of people I've interviewed that weren't able to solve a left join question. Let's leave right and full outer joins out of the discussion--a left join is as advanced as I dare get off the bat in an interview.

I've once had a guy I worked with ask for help when he was with a new company. I told him to look at the query plan. He replied, "Yeah, but that doesn't tell you much."

Relational databases are oft-misunderstood and underestimated time and time again. Where are the Codds of yesteryear?

<comment><USER5> I've never understood the typical programmers aversion to query writing. Writing SQL is in general pretty easy (basic set theory).

I'm guessing it's all about what people have experienced. I have been coding against RDBMSes since my first job while still in college. I have even read Celko for fun at one point :)

IMHO, learning the _hard_ parts of SQL only come from experience with particular RDMSes and dealing with large datasets. For example, knowing that the only way to get a query to run against a large Teradata table without running out of spool space is to create and use temp tables only comes as a result from trying to run the query.

<comment><USER6> One of the biggest hurdles that typical programmers must overcome is that SQL is a declarative language, and therefore quite different to the imperative languages they are used to.

<comment><USER7> Imperative programmers all secretly believe in a declarative style, that's why they compile with -O and hope the compiler's "query optimizer" will make it fast for them.

</comment></comment></comment></comment></comment></comment><comment><USER8> Just so folks know, there are tools that will decompose queries and make nice little pictures out of them. With something like this, you'd have to use it just to get started.

Once you've visually decomposed it, you'd physically decompose it by splitting it inside-out. Then proceed to understand and debug inside-outwards.

Not fun, but not impossible. Just a huge pain in the ass. Making it more fun would be a database with bad RI, nulls, and duplicate data all over the place. Don't get me wrong -- from looking at the image it definitely looks like aspirin will be required. :)

<comment><USER9> I've always believed those tools must exist, but sorting out the SEO crap and advertising copy that gloms up search results for them is painful. Can you name any Linux/Mac tools like that?

<comment><USER8> Not linux, but I have a windows example. Being an El Cheapo, what I do from a windows box is fire up MS Access, link into the database (and this works even if the database is on a *nix box somewhere), then drag the text of the query into the graphical query builder tool. It's taken some pretty complex stuff apart for me in the past.

It's been a while since I last deep-dove in a complex database, so I don't have any other examples handy. Sorry. Maybe somebody else can pick up the thread. I remember reviewing a bunch of them several years ago -- these tools have been around for a long time.

</comment></comment></comment><comment><USER10> I once worked on a site where the original developer clearly didn't know joins existed, so if he wanted data from two related tables, he'd get all the required results from table one, then loop through them, one by one, querying table two for the corresponding record. Sometimes this went 3 or 4 tables deep, the site would take nearly a minute to load a table of products.

<comment><USER11> I see this pretty often with people who haven't spent much time with relational databases. I suppose it's understandable since thinking in sets with SQL is a different paradigm than they are used to, but I even see tons of PHP/MySQL tutorials online that use this method when they should be using a join.

<comment><USER12> In the old days, MySQL didn't support foreign key indices, so the LAMP developer community had this mentality that "Joins are slow" or even "Joins are evil" and actively encouraged people not to use them. So its not really surprising this kind of thing is still out there - PHP tutorials are somewhat infamous for promoting 'worst practices'.

<comment><USER13> Yeah, I know a Rails developer who thinks four-way joins are a design smell.

</comment></comment></comment></comment><comment><USER14> I'm currently reading "Mastering Relational Database Querying and Analysis" by John Carlis, which posits that SQL is inherently flawed for several reasons. To paraphrase from the text:

First, both the syntax and the way querying is generally presented in textbooks, lead you to think that your task when querying is to _display one unnamed table_. The author objects to each of those four words.

Second, many people have found querying with SQL terribly difficult. Even experts find SQL hard to create and read. Do not be surprised if an analyst struggles to understand his/her own SQL. It is impossible for users to understand any but the simplest SQL.

Third, SQL practice suffers from the notion of a "correlated query" -- which has a monolithic subquery that is executed repeatedly via looping, once for each value of a candidate row picked by an outside SELECT.

The book has much more to say on the topic of SQL before going on offer relational algebra (built on top of SQL) as a n alternative.

<comment><USER15> I disagree with the readability part. I do heavy use of "WITH" to name my intermediate steps and comment the tricky parts (as I would do with any other programming language) and my colleagues find them pretty readable(or that's what I'm told). That's how I "reverse engineer" such monster queries, refactoring them in intermediate relations with names. Pretty often the same subselect is used more than once.

In fact, due the lack of side effects it's much easier to do than with procedural code.

<comment><USER16> Doh! I must be one of the SQL bunnies everyone else is superior to...

I didn't know about the WITH statement. Thanks for enlightening me.

</comment></comment></comment><comment><USER17> A lot of the problems I have seen with queries (other than DBA issues) is the conflict between application developers and report writers. A lot of databases are designed for transactions and resources are not often available to do a proper reporting database or at least summary data.

I have a very simple rule for myself - "if a user of the application is concerned about a certain attribute or state an element (e.g. person, truck, plane) is in, then a report will be required showing all elements with that attribute or state."

If your database design cannot support that rule, then trouble will happen and you will have serious performance problems.

To give a simple example, suppose you are running a group of storage garages. You have a table with all your customers, a table with all your storage units, an assoc table joining customer and units with active flag + date of start, and a table with all your payments. Good enough to do transactions and figure out for a unit if they are payed up.

On the other hand, writing the report to tell who hasn't paid is going to be kind of a pain. It is a simple example, but not much different from what you find in large systems.

<comment><USER18> Is your point here that the data being stored is insufficient (e.g. you'd want an end date, not just an active flag; this doesn't cope at all with prices changing over time, bulk discounts, or different customers paying different rates; there's no concept of invoices, or whether payment is due based on calendar months or based on opening date; etc) or that you're ignoring all that sort of stuff just to keep the example simple (so assume everyone pays a fixed rate per unit, due weekly; someone can't close their account until they're paid up; etc) but that you'd still want a more complex schema so as to be able to more easily generate a "Who owes us money?" report?

If it's the former, then sure: you need to be able to model all these things properly. If it's the latter, then I'm not so sure. The SQL to create that sort of report is going to be non-trivial, but it shouldn't be overly complex for someone who knows what they're doing, and if you have the correct indexes it shouldn't take very long to run either.

If you want to start doing all sorts of fancy data warehouse slicing and dicing, you're usually better extracting daily (or more/less frequent depending on needs) dumps of your transactional database into a different structure more suitable for reporting, than in restructuring your 'live' database and having to deal with all the resulting denormalisation issues, etc.

<comment><USER17> As stated, I believe all the information needed to generate the report is there. My point was that what is sufficient for an application is generally insufficient for basic reporting. So, when budgets get short, the database work to make queries by reporting easier is often ignored and complicated queries and processes become the norm. Those laying the problem solely at the feet of the DBA's are missing the other group that tends to make these queries necessary: App Developers. I once had an app developer tell me one type of invoice was impossible to make because some of the relationships and data were intrinsic to the application and would need to be modeled the same as the application. He wasn't far off.

There is a point between transactional and data warehousing that needs to be hit. Simple summaries or considering the question of "how do I retrieve everything in this state" will sometimes suffice. Building a system to get single transactions in an out tends to make routine report take all night or be impossible to get in a reasonable amount of time.

If your schema and app requires non-trivial SQL, then expect maintenance nightmares and lack of ability to scale.

<comment><USER19> Not always. An app requiring non-trivial SQL can be an indication of a problem requiring non-trivial workflows.

If the problem is complex, you want to model relations, not ignore them.

(But clearly not in this hairy query.)

</comment></comment></comment></comment><comment><USER20> When I've seen this article back in my RSS reader, it reminded me of one particular query that was generated in the application I'm maintaining. My irrational fear of sending too many queries to the database (I've outgrown that in the last 6 years) caused a single query to be generated which was 4KB in size.

Which of course is much less than the one in the picture, but still very, very bad.

Some so we've refactored the beast. Now it's 2-3 smaller queries (which are much easier to optimize for PostgreSQL and, above all, individually cacheable) which lead to a nearly 100% speedup for common cases. Also, the code is infinitely more readable which means that it's much easier to extend it.

I'm incredibly happy that we've seen the light and fixed it before it grew to proportions like the ones on the original article _shudder_

<comment><USER21> 4KB? That's nothing. Search for 'media.sql' in any recent Adobe installation media. You'll find 3 MB+ SQL files, containing:

      -(BASE64?) encoded InstallerIcon,
      -(BASE64?) encoded EULAs in various languages
      -GUIDs like {01C3BD72-7371-4472-B179-B4DFE6DDD251}
      -and my personal favorite: a 25 KB XML fragment

<comment><USER22> There was a source control system whose code I had the pleasure of reading several months ago whose favorite way to store data was as a SQLite3 database, with a single table, with a single column, with a single row, containing JSON. Words failed me.

Based on what you're describing, I now believe those developers were poached from Adobe.

</comment></comment></comment><comment><USER23> People in that thread are bragging about their 10-page queries with 20 joins or 8 unions.

I'm looking at a query here that is 37 printed pages, with 92 joins over 25 unions.

<comment><USER24> I think at this point I am bragging that I have never written one. Ok, I have used views of views, but.......

I have, however, had the misfortune of troubleshooting those 10 page queries. Finding a stupid typo in one of those is like looking for a needle in a haystack.....

</comment></comment><comment><USER25> My first assignment in my first ever job working for a 'proper consultancy' was to babysit an overnight process which was a SQL Server stored procedure. Back in those days they had a 64k limit, so it was split into 3 or 4 sections. It took around 12 hours to run.

I'd like to say I rewrote it, but I didn't. I just left.

</comment><comment><USER26> Reading the comments below, I get the impression that all the "good" DB people hang out on HN, not like those "other" incompetent nits out there who don't know what a join is. Hubris, people.

<comment><USER27> Maybe HNers are just better at keeping their mouths shut when they don't know anything about a topic.

<comment><USER7> I guess you're new around here ;-)

</comment></comment></comment><comment><USER28> Really nasty piece of SQL code, definitely not for human-based processing. What do you think about tools that may decipher and visualize such complex queries in a more structured way, like DBClarity (<http://www.microgen.com/dbclarity/>)? Have you been using something similar recently?

(disclaimer: I work for mcgn)

</comment><comment><USER29> The original version of foursquare.com contained a lot of stuff like this (though not as epicly bad). It was a very small amount of poorly written PHP code surrounding a bunch of unreadable SQL statements. It's amazing that it worked at all.

Dens is a great guy, but I hope I never have to rewrite his code again.

</comment><comment><USER30> I was just trying to figure out a stored procedure that queries one table, loops over the rows, and within the loop queries another table using the values from the first query. Now, looping over these rows, it has a third query and a corresponding loop over those rows.

And all that for inserting the values taken from the three tables into a 4th table. This could have been done with a simple 3-table join query. Hell, it could even have been done with a single insert statement! I wonder how people fail to recognize an N+1 selects problem when it's staring them in the face.

Well, to be fair, this problem I described isn't exactly an N+1 problem is it? More like an N(M(L+1)+1)+1 selects problem. ;-) (Unless I've got my math all wrong there?)

How I hate working with PL/SQL stored procedures! :(

</comment><comment><USER31> Reminds me of the stock update system for one of our major clients at my first job. The predecessor of myself and my colleague had thought it a brilliant idea to build a clothing ecommerce site, with a complete list of all stock going back to the year dot with ASP and Access (that's Classic ASP, not .NET). Towards the end the stock update would take pretty much an entire afternoon to run.

Eventually we got the approval to change to MySQL for the database. When they ran the first stock update with the new version they rang us up to check it had worked because it was near instantaneous.

The moral of the story: Access is BAD! VERY BAD!

</comment><comment><USER32> I've seen and had to debug longer stored procedures for sure but never a single query. I can't see the last page so maybe this is a stored procedure. It's hard to tell.

</comment><comment><USER33> I worked for a company where there were queries somewhat like this, however they were obscured because they would create views on the fly. So a query would look deceptively simple only to realize (not exaggerating here) there were four levels of views underneath it. Bugs were a pain, but the worst was trying to optimize those queries. Just untangling what the actual query was made life really difficult.

</comment><comment><USER34> I've seen SQL that looked like this but didn't wind up being very complicated. I've also seen seemingly simple queries that were actually very tricky!

I can't read much of the query, but at least a few lines are checking for null values. I wouldn't be surprised if 80-90% of the query is simply output formatting. Depending on the DB platform, some formatting and null-check statements are fairly verbose.

</comment><comment><USER35> People can write queries that large without formatting? or was that the result of some query generation application?

<comment><USER36> It's got to be from a point-and-click Query-O-Matic tool.

<comment><USER37> Or an ORM.

</comment></comment></comment><comment><USER38> Please forgive me, this is OT: can anyone here recommend a good online resource for learning SQL "the hard way"?

<comment><USER39> <http://www.db-class.org/course/video/preview_list>

</comment></comment><comment><USER17> I will say, Ingres was not my favorite database, but its query plan display should be used to explain how a database query works. It showed a tree of operations for each query. If you saw FSM (full sort merge) or Cartesian Product you better mean them or re-write the query.

</comment><comment><USER40> I'm pleased that most of the commenters recognize that SQL has a need and is it's own language, for good and bad. I really expected to find a troll popping out "NoSQL rulez" type comments, and the level of understanding of how and where SQL can help is very encouraging.

</comment><comment><USER41> Soooo... what is it supposed to do? Looks like a million sub-selects and joins! Already have a headache just with looking at it. I'd probably right it all over again from scratch.

</comment><comment><USER42> A place I used to work at used an ORM which gradually constructed SQL throughout the flow of a request. One of the calls we generated was probably a couple of pages long.

</comment><comment><USER43> I'd love to see an EXPLAIN on that. ;)

<comment><USER44> __________________________* 1. row __ __ __ __ __ __ __ __ __ __ __ __ __* id: 1 select_type: SIMPLE table: lots_of_em type: not_good possible_keys: none key: none key_len: n/a ref: NULL rows: googol filtered: 0 Extra: You're screwed, Do not pass go.

<comment><USER45> This is an old link, but you just reminded me of: <http://howfuckedismydatabase.com>

<comment><USER10> That's awesome, especially this - <http://browsertoolkit.com/fault- tolerance.png>

</comment></comment></comment></comment><comment><USER46> htsql (www.htsql.org) is a business reporting language -- one line in htsql can generate 5 or 6 lines of SQL.

This query could be condensed considerably if rewritten in htsql.

(htsql automatically generates SQL code that covers all corner cases and executes faster than hand-crafted SQL.)

</comment><comment><USER47> While we're sharing horror stories...

I once encountered a stored procedure that returned HTML in a result set. It literally created the UI of a webpage. It returned several columns of HTML that the app would place in strategic parts of the page. Well, as years went by, the app required a more innovative and web 2.0 UI. Rather than remove the HTML from the sproc, more columns were returned with more HTML, Javascript and the like. One time I had to fix some Javascript that rendered on the page. When I finally found where the errant JS was coming from, I realized I had to file a database change ticket to fix the UI :)

<comment><USER48> Sounds like Oracle's Portal product?

<comment><USER49> I would guess Oracles mod_plsql

</comment></comment></comment><comment><USER50> See, this kind of crap is what happens when you have the programmers sit on the bench and let the "engineers" take charge. Why do you hate society, you Luddites?!

</comment><comment><USER51> This is kinda impressive.

</comment><comment><USER49> Im sure 12 pages of Java code doing this procedurally instead is much more maintainable.

</comment><comment><USER52> Sorry, I call BS.

I might have just been lucky enough to always work at professional companies and startups where this kind of stuff can never happen.

But something tells me there is no reasonable way an SQL query can grow to these proportions.

<comment><USER53> All the good techniques and technologies you use now? Yeah, those were invented because the way a lot of people used to do things was terrible.

I'm not old enough to have built systems like that, but I have inherited some for maintenance and debugging. That's not likely to be BS. Back in the day "do it in a stored procedure" was common advice. All the logic for untold applications and millions of lines of code lived at the database level.

Count yourself lucky perhaps to have entered the market when you did, but don't discount that you're standing on tall, sometimes ugly, shoulders.

<comment><USER12> There's still plenty of DBAs out there preaching "do it in a stored procedure", because in theory it lets them do things like changing normalization without affecting the application interface. (In practice, I've never seen happen that way.)

But in general, people did thing that way because there wasn't any common middleware or services interfaces. The RDBMS was the "app server".</comment></comment></comment>


Replication of Marshmallow Test: Delayed Gratification May Be Less Predictive - <AUTHOR> http://www.mdmag.com/medical-news/replication-of-marshmallow-test-reveals-delayed-gratification-may-be-less-predictive-of-later-outcomes <comment><USER2> FYI this test is also "diffused" in the book The Influential Mind.</comment>


DLang's betterC mode improves on C - <AUTHOR> https://dlang.org/blog/2018/02/07/vanquish-forever-these-bugs-that-blasted-your-kingdom/ <comment><AUTHOR> The -betterC option in the D language makes think that switching to D (instead of C where it would be used) would be a painless way to use a better language with an easy transition path for programmers used to C.

The D language folks should do a better job evangelizing the use of D for specific purposes rather than try to be everything for everyone. I see D as a suitable replacement for everything that Go is currently used for and with the betterC option it is also, I think, a good replacement for C in embedded development.

<comment><USER2> I agree that D has a messaging problem. The folks doing Rust have been very effective at promoting the rust language as a very good C/C++ successor for systems programming based on a few key features, but I think D is actually a better overall next step for people from the C/C++ world.</comment></comment>


Show HN: ReactNativeJobs.io - <AUTHOR> http://www.reactnativejobs.io/ <comment><AUTHOR> I have updated the navigation for mobile. It should be a lot easier to navigate now on a small screen. I'd like to make the site look better overall, but all in due time.</comment>


Show HN: Timing attack to check presence of devtools - <AUTHOR> https://github.com/awalGarg/devtools-timing-attack <comment><USER2> From timing attack point of view its nice . A more practical solution to check if devtools is open might be checking

window.outerHeight - window.innerHeight

<comment><AUTHOR> Thanks! Your trick won't work though when devtools are opened in docked mode or during remote debugging :)

</comment></comment><comment><USER3> Very cool. I was able to generate a false positive by highlighting text and moving my mouse erratically.</comment>