The Decline of 'Big Soda' - <AUTHOR> http://www.nytimes.com/2015/10/04/upshot/soda-industry-struggles-as-consumer-tastes-change.html?_r=0 <comment><USER2> _When Mr. Nutter was fighting the soda tax battle, he kept a bottle of Mountain Dew and a container with 17 teaspoons of sugar — the amount in the bottle — on the table in the center of his office. It was a good "conversation piece," he said, about the surprising number of calories in a typical soda._

I think this is key. Better information and transparency is having an effect. I'm glad to hear we're approaching a point where the RDA for sugar will be listed on products just like fat, sodium, and potassium. I expect this will have a major impact on consumer behavior and industry behavior, much like calorie listings on fast food menus. It still requires government regulations, but instead of challenging First Amendment principles, it embraces them.

Most consumers aren't idiots. But they're aren't scientists or academic researchers for the most part either. Bring them relevant, accurate information and they are capable of making rational choices. This article is encouraging evidence.

<comment><USER3> That's why we haven't had anyone start smoking in decades..

Soda sales are down but obesity is still up, so the calories are coming from somewhere. If you believe the other common stories, we're drinking the calories.

I'd wager they come from the rise of coffee-based beverages - which would not be considered soda - but I don't have data to back that up.

<comment><USER4> For now I think Starbucks et al are benefiting from being culturally identified as more upper/middle-class. Obesity stereotypes seem to have fixated mostly on a "people of Walmart" type image of the fat, slobbish lower classes. So even though a 12-oz Starbucks caramel latte is far worse than a 12-oz Coke, it gets a pass.

<comment><USER5> Sugar-wise, the can of coke is about 70% worse.

12-oz caramel macchiato = 23 grams of sugar [http://www.starbucks.com/menu/drinks/espresso/caramel- macchi...](http://www.starbucks.com/menu/drinks/espresso/caramel- macchiato#size=126197&milk=63)

12-oz can of coke = 39 grams of sugar [http://www.coca- colaproductfacts.com/en/coca-cola-products/c...](http://www.coca- colaproductfacts.com/en/coca-cola-products/coca-cola/)

<comment><USER4> Good point, the source of calories isn't the same mix, even though the latte has more total calories (200 calories/12oz, vs. 140). Though I was thinking of the caramel latte rather than macchiato, which is a bit closer, at 27g of sugar: [http://www.starbucks.com/menu/drinks/espresso/flavored- latte...](http://www.starbucks.com/menu/drinks/espresso/flavored- latte?foodZone=9999#size=177970&milk=63).

And since it's autumn, it's now time for their signature seasonal drink, the pumpkin spice latte, 300 calories and 39g of sugar per 12oz: [http://www.starbucks.com/menu/drinks/espresso/pumpkin- spice-...](http://www.starbucks.com/menu/drinks/espresso/pumpkin-spice- latte#size=183886&milk=63&whip=125). You can bring that down to 240 calories and a mere 37g of sugar if you ask for no whipped cream.

<comment><USER6> Another issue is size. In Europe or at least at many of the places I've been lately the sizes they have at Starbucks USA do not exist.

A latte for example in Paris or Barcelona is no more than ~6oz or so, maybe less whereas Starbucks USA they start at 8oz and go up to 24oz. I have not visited a Starbucks here to see the sizes. Every coke I've ordered has come in a 8oz bottle as well unlike the unlimited refills and or big-gulp sizes seen in the USA.

<comment><USER7> Ugh, as an americano drinker I'd be annoyed as hell by this. I have a 52oz water cup at my desk, for me much of things is about the quantity of liquid so I dilute everything. I get my alcohol drinks heavily cut with mixers so they last longer. If drinks are small I finish them way too fast. So a tumbler of a mixed drink lasts me 8 minutes but the same drink in a pint with little ice and more mixer is more like 40 minutes. One leads to shitfaced quick and is a nice buzz. Same for coffee. Sitting at my desk with a small drink I'll finish it without noticing I'm drinking it. 20 oz coffee with the same # of shots and It'll take 30 minutes to finish and I'll actually notice and enjoy it.

I'm an adult, I can pay attention to what I order, and I'm not embarrassed to order things that are not directly on the menu. I'm tipping the bartender and the barrista so they make it the way I want it anyway.

<comment><USER8> Coffee in Italy, France and other places in Europe is meant to be drink in small cups. Like a shot. I've asked for Americano in France and people are always making fun of me. But I will never understand why you would drink a shot for pleasure when a drink can last way longer. So yeah totally agree with you.

<comment><USER4> This varies a lot within Europe. In the Nordic countries, the norm is to drink larger cups of coffee, traditionally brewed filter coffee, but lately also various latte/etc. type drinks (overall, similar to what Americans drink), slowly over a longish period of time. That's embedded in cultural expectations as well: going out "for a coffee" is a popular type of casual socializing, and means sitting at a table and chatting for maybe an hour or so while you sip a coffee. Not the Italian-style 30-second outing, where you drink a shot of espresso at a bar standing up.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Just take some time next time you're in a supermarket and walk down the cereal aisle, it's just incredible. It's all sugar and carbohydrates in colorful cartoony boxes meant to appeal to children. Just a cup of cereal has many calories and people and children eat huge bowls of this stuff.

So I think an anti-cereal campaign might also help. I often wonder what would happen to national obesity rates if cereal aisles in supermarkets just disappeared overnight.

<comment><USER10> There some much healthier options in there if you know where to look, but unfortunately its not easy to identify them, and even things like granola which are colloquially thought to be healthy are often jam packed with sugar.

<comment><USER11> Exactly. They put extra sugar/sweeteners in absolutely everything these days. You have to look at the ingredients and calories on anything you buy.

Even super healthy-eeming things like granola or dried fruit usually have sugar added. Yogurt is still promoted as a healthy snack, but most of them have as much added sugar as a candy bar.

<comment><USER12> > Yogurt is still promoted as a healthy snack, but most of them have as much > added sugar as a candy bar.

At least plain yogurt is still made only from milk, milk solids and yogurt cultures.

...unless that "nonfat milk solids" means sugar?

</comment></comment></comment></comment><comment><USER13> It's interesting to me that diet soda consumption has dropped from its peak. I would have guessed that sugary soda would have been replaced by diet soda, but it appears that water has replaced sugary soda more frequently. Mind mildly blown.

<comment><USER14> A lot of people just can't get over the terrible taste of artificial sugar. If someone ever creates an artificial sugar indistinguishable from sugar or corn syrup, it'll be a multibillion dollar invention.

I'd give my right arm for zero calories coke, I'd rather have a perrier than diet coke.

<comment><USER15> I agree. Though Coke Zero is significantly better than Diet Coke.

</comment></comment></comment><comment><USER16> Like many of the posters here, I realize that much of the food and drink sold is unhealthy. On the other hand, I don't think that nanny state regulations are the answer. Plenty of unhealthy things abound (when overused): chips, candy, pizza, fast food, cookies, ice cream, video games, television, etc. Maybe some of the same marketing tactics used to sell the unhealthy things can also be used to counter them with healthier alternatives.

<comment><USER17> Under a socialized healthcare regime, every sick person is a problem for the whole society in that we have to pay for it. So, being careless with your health is a crime against the people. If you are in the USA, consider that you have been funding healthcare for some of the worst offenders for decades by paying taxes which fund hospitals. Now with the healthcare reforms in mind, which have itemized your contribution to social healthcare separately from your taxes, take a walk around Walmart and consider all the preventable health issues you see in the crowd. Do you really want to pay for that? I feel like you and I should have the freedom not to, and our collective freedom not to is more important than any individual's freedom to eat and drink his way to costly, lifelong illness.

<comment><USER18> Which is precisely why some of us are opposed to socialised healthcare. Once you have it, every little decision you make becomes everyone else's business.

<comment><USER19> You make "socialised healthcare" sounds like we have a NHS food police tracking us and sending us off to a health re-education camp if we don't eat 5 bits of fruit and veg a day.

<comment><USER20> And even if that were true, they're obviously not doing a very good job of restricting bad food intake given the UK's own obesity problems.

If the NHS were following everyone around, they'd have to be stuffing people full of low quality food to get the present outcome.

</comment></comment></comment></comment></comment><comment><USER21> I would like to see a decline of "Big water". It astonishes me that bottled water is consistently sold for more than sugar water and not one bottler breaks from the pack and sells it for less. That can only happen with industry wide collusion to fix prices and yet no one in the government does anything about it.

<comment><USER22> I think my local supermarket sells bottled water for as little as 19p (29c) for 2 litres, as as much as about £2 ($3) so I think you are mistaken.

<comment><USER23> That stuff is probably more or less tap water:

[http://www.independent.co.uk/news/business/news/tesco-and- as...](http://www.independent.co.uk/news/business/news/tesco-and-asda-deny- claims-that-own-brand-bottled-water-is-in-fact-tap-water-8059703.html)

Not that there's anything wrong with that. Actual spring water is uniformly 45p for two litres:

[http://www.tesco.com/groceries/product/details/?id=258016736](http://www.tesco.com/groceries/product/details/?id=258016736)

[http://www.sainsburys.co.uk/shop/gb/groceries/sparkling- wate...](http://www.sainsburys.co.uk/shop/gb/groceries/sparkling- water/sainsburys-caledonian-scottish-water--sparkling-2l)

[http://groceries.asda.com/product/sparkling-water/asda- eden-...](http://groceries.asda.com/product/sparkling-water/asda-eden-falls- sparkling-natural-mineral-water/19028)

[https://groceries.morrisons.com/webshop/product/Morrisons- Pe...](https://groceries.morrisons.com/webshop/product/Morrisons-Pennine-Vale- Sparkling-Water/111996011)

That said, i imagine kevin_thibedeau was referring to the situation in the US. Perhaps bottled water is more expensive there. Or perhaps sugar water is cheaper.

<comment><USER24> > That stuff is probably more or less tap water

Is that bad? People buy bottled water for the convenience, and because it's filtered tap water which tastes better than regular tap water.

If you want to reduce bottled water purchases you should advocate for a law requiring water fountains to have carbon filters for taste. (Although I kinda like knowing there is chlorine in there when I drink from one.)

</comment></comment></comment></comment><comment><USER25> "Soda companies" are already moving to controlling water supply in many countries. In Brazil, it's already hard to find in some stores water which is not owned by either Coca-Cola or Nestlé.

<comment><USER26> Does the "water supply" of most citizens of most countries come from a grocery store? Or does it still come from things like _municipal plumbing_ or _rural wells_?

<comment><USER27> In Latin America tap water is not drinkable so most people get their drinking water bottled.

<comment><USER28> Not in Peru, Bolivia, Ecuador and probably Colombia. People there massively drink boiled tap water.

<comment><USER27> I just got back from Peru, everybody was drinking bottled water and boiling water for cooking. I'm sure this is a matter of preference.

<comment><USER28> You were probably with middle class people (lived there for 5 years, wife's from there).

</comment></comment></comment></comment></comment></comment><comment><USER29> What are restaurants doing to combat the loss of their highest-margin items? Another bonus of drinking water instead of a soft drink is a cheaper bill at lunch.

<comment><USER30> They're just increasing the prices of everything else. In earlier decades, restaurants used to sell their food at a near-loss because everyone had a couple of high-margin alcoholic drinks with their meal. When that started to go by the wayside, the restaurants had to increase the menu prices of the actual food. I expect we'll see the same thing happen if people stop buying beverages entirely, especially at fast-food restaurants where the soft drink is almost always the highest-margin element of the meal, making up for loss leaders like the "dollar menu".

</comment></comment><comment><USER31> Soda isn't inherently evil. Education on how to track calories and understand the density of some foods is more important. Of course, since sodas don't have really any nutrition, it is _hard_ to drink a lot of them.

You can't single out soda while you still have people eating "healthy" salads that are twice as big as they should be, covered with dressing and cheese and olives and fatty meats.

You can't single out soda while you let people drink "coffee" (which by itself is fine) that includes 1000 calories worth of additives like cream and sweeteners.

And oil is extremely calorie-dense; simply cutting in half the number of fries that you eat would go a long way.

With a proper diet and exercise you can certainly have a couple of sodas a week. There is also a better way to drink soda: don't use a straw. Just _try_ to get through a 32 oz. soda when you sip it a bit at a time; you'll find that you don't really drink it that fast, that it tastes better, and that a small amount is still satisfying. After awhile it'll seem much more natural to order a small soda.

</comment><comment><USER32> pro-tip for anyone trying to kick a soda (or alcohol) habit: get yourself a seltzer maker, and an assortment of bitters (Fee Bros Cranberry, Walnut, and standard Angostura are a good place to start).

This is just my personal experience, but I found that most of the unhealthy drinking I was doing was out of habit and boredom. I'm just as happy with a seltzer + a few dashes of bitters as I was with a beer or a coke.

<comment><USER33> Even if you don't get something like that, just having a glass of water nearby is a great way to reduce consumption. I drink a lot of diet soda and it's mainly out of compulsion to constantly drink liquids. If I keep a nice cold cup of water near where I am, the soda consumption drops super fast. Once I start drinking water during that day, that's usually what sticks.

For me, it's almost all about the habit of constantly drinking. That said, even though I know how to kick the habit, I still tend to drink a lot of diet soda because it's a nice pick-me-up.

</comment></comment><comment><USER18> It's amazing how many people seem to think drugs should be legalised but sugar should be banned.

<comment><USER34> For starters, I don't think there's any conceivable risk of boxes on supermarket shelves being stuffed to the brim with cocaine.

If you want to look at the number of cumulative life-years lost to drugs vs. sugar in the past several decades, my intuition is that it's not even remotely a close contest.

<comment><USER35> > _For starters, I don 't think there's any conceivable risk of boxes on > supermarket shelves being stuffed to the brim with cocaine._

Then again, it might be worth remembering the origin of the brand name Coke.

<comment><USER36> When remembering _that_ , it's also good to remember the regulatory regime present at the time. [0]

From a food and drug regulation perspective, _that_ time is _very_ different from _this_ time. That difference makes your pithy quip not so very clever.

[0] Compare the date at which cocaine was _removed_ from Coca-Cola [1] to the enactment date of the Pure Food and Drug Act. [2]

[1] [https://en.wikipedia.org/wiki/Coca- Cola#Coca_.E2.80.93_cocai...](https://en.wikipedia.org/wiki/Coca- Cola#Coca_.E2.80.93_cocaine)

[2] [https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act](https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act)

</comment></comment></comment></comment><comment><USER37> What we really need is a limit on advertising. Corporations manufacturing packaged sugar like coke, cereal etc. should not be able to manipulate young children to get them addicted to their crap. Sell this stuff in plain white boxes, ban advertising to young children, and in general, limit the free speech of soulless legal fictions when they target children.

<comment><USER38> Young children don't do the grocery shopping. Parents need to take responsibility and not buy unhealthy food for their kids. I'd say children up to a certain age are drinking this stuff simply because their parents bought it for them (How else?).

Plus, "when they target children" would simply become harder to define as companies would modify their ads, but the problem would remain if parents aren't taking an active role in educating their children themselves as to what is unhealthy, and making purchasing decisions that are healthy, instead of depending on the government (or corporate advertising) to do it for them.

<comment><USER39> Kids can buy whatever they want if they have money. And surely, they'll buy the shiny thing thing they saw on TV or the internet when they encounter it in the shop and see it's cheap.

<comment><USER38> Again, kids won't do "whatever they want" if their parents do their job and instill proper discipline, education, and consequences for behavior that _they_ don't want.

If you want the government to do your parenting for you, I don't know what to tell you. Not my idea of good parenting, and I think the results will be disappointing. We can agree to disagree on that.

<comment><USER40> > Again, kids won't do "whatever they want" if their parents do their job and > instill proper discipline, education, and consequences for behavior that > they don't want.

Clearly you aren't a parent, and I suspect you also have never been a child. It would be handy if what you wrote were true. But children aren't robots, and parents aren't the only people who influence them. The reason that food companies target children in their advertising is that it works to manipulate them.

</comment></comment></comment></comment></comment><comment><USER41> > It's clear that soda's calories contribute to weight gain and obesity, but > whether its impact is greater than that of other unhealthy foods has not > been conclusively demonstrated. Nevertheless, the change is already > underway.

I think that if you consume a lot of sugary stuff, like drinking soda all day, you're going to get used to the taste of sugar. If you cut soda, consume a lot less sugar, you may find you have less desire for other really sweet products like ice cream, cake, candies, etc., because they are just _so sweet_ and at the very least won't feel like eating a whole bunch of it any more.

</comment><comment><USER42> Submission from 18 minutes earlier: [https://news.ycombinator.com/item?id=10324311](https://news.ycombinator.com/item?id=10324311)

Submission from 19 hours earlier: [https://news.ycombinator.com/item?id=10321396](https://news.ycombinator.com/item?id=10321396)

<comment><USER43> A small number of reposts is ok if a story hasn't had significant attention yet [1]. After that, we bury reposts as dupes.

This is the main device for migitating the randomness that otherwise dominates which stories make it off /newest [2]. Basically, it's ok to roll the dice a few times.

If we see a good-by-HN's-standard story that fell through the cracks, we've been emailing the submitters and inviting them to repost it. We invited coloneltcb to repost this one. This is the latest in a series of experiments that I've written about at [3] and [4] if anyone's interested.

1\. [https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html)

2\. [https://news.ycombinator.com/item?id=9828818](https://news.ycombinator.com/item?id=9828818)

3\. [https://news.ycombinator.com/item?id=8790134](https://news.ycombinator.com/item?id=8790134)

4\. [https://news.ycombinator.com/item?id=9866140](https://news.ycombinator.com/item?id=9866140)

<comment><USER42> I was aware of that. That's why I didn't flag the duplicates.

My purpose in providing back links on each duplicate to the earlier posts was to try to avoid fragmenting the comments. When there are two submissions 18 minutes apart, there is a decent chance comments will be split between them.

> If we see a good-by-HN's-standard story that fell through the cracks, we've > been emailing the submitters and inviting them to repost it. We invited > coloneltcb to repost this one

That seems pretty inelegant for a site named "Hacker News". Why can't you diddle the database to give the original another shot? The "invite a resubmission" approach makes the submitter have to figure out a different URL, and increases the risk of comment fragmentation.

<comment><USER43> I don't think we need worry about comment fragmentation on posts that have few points and no comments and are more than a few hours old. The odds of new comments appearing there are minuscule. On live threads, yes, but we often move those [1].

We do give originals another shot, by rolling the clock back on them internally, if they're up to a few hours old. Beyond that, it feels weird to have a story on the front page with just a few points and a timestamp that says e.g. "23 hours ago", let alone "163 days ago".

We don't change the user-facing timestamp, because that feels like rewriting history, the sort of thing that HN users wouldn't like. It would also make /newest look weird as neighboring posts could then have different ages.

The repost invites don't require users to come up with a different URL. You click on a link and the software fills that in for you.

Probably our next step will be to add a setting to user profiles that people can turn on if they want the software to do such reposts for them automatically. In that case, no need for an email, plus the software could pick a good time to do the repost. On the other hand, it's already clear that many users like getting these emails. The positive feedback has been striking.

An open question is what to do in cases where the submitter has no email address in their profile, or doesn't check their email.

If you or anyone knows of a more elegant solution, please tell us! We would love that.

1\. [https://hn.algolia.com/?query=by:dang%20comments%20moved&sor...](https://hn.algolia.com/?query=by:dang%20comments%20moved&sort=byPopularity&prefix&page=0&dateRange=all&type=comment)

</comment></comment></comment></comment><comment><USER44> You realize what happens when "Big Soda" declines, right? "Big Water" is next, and with that, privatization. GIVE THEM FAT KIDS.</comment>


Network administrator/Programmer - <AUTHOR> Does a typical Network Administrator utilize programming for automation and network management? How reasonable is it to have a Network Administrator who spends time automating and programming as part of his normal duties? <comment><USER2> _Typical_ netadmin probably writes some minor scripts at most, or at least this is what I have seen in corporations and in a medium-ish ISP.

On the other hand, network administration is not that far away from what sysadmins do (tracking inventory of things, monitoring, collecting logs, creating and distributing some configs), and there are several sysadmins who mainly write code (old school system programmers), so it just makes sense.

</comment><comment><USER3> I don't know about typical, but being able to automate things is a sign of a good network admin.

</comment><comment><USER4> do it; don't limit yourself to what your neighbors are doing.</comment>


Uber lays off 3k more employees in latest round of Covid-19-inspired cuts - <AUTHOR> https://www.theverge.com/2020/5/18/21262337/uber-layoff-3000-employees-covid-19-coronavirus <comment><USER2> Note this is dated May/18th, two weeks ago.</comment>


Show HN: Interview Screening as a Service - <AUTHOR> https://gatekeeper-frontend.herokuapp.com/request <comment><USER2> Isnt that what headhunters are for?</comment>


PyVideo is closing down - <AUTHOR> http://bluesock.org/~willkg/blog/pyvideo/status_20160115.html <comment><USER2> Replaced with a youtube search, I presume?</comment>


The Top Idea in Your Mind (2010) - <AUTHOR> http://www.paulgraham.com/top.html <comment><USER2> "Everyone who's worked on difficult problems is probably familiar with the phenomenon of working hard to figure something out, failing, and then suddenly seeing the answer a bit later while doing something else"

Uncanny how much this happens, I'll be playing Smash or eating cereal when I have sudden moments of clarity on problems I've been at for days

<comment><USER3> Yes it makes me wonder if there's another consciousness in my brain, with its own thoughts and feelings.

<comment><USER4> The most compelling reasons I've seen all come back to the Default Mode Network ( [https://en.wikipedia.org/wiki/Default_mode_network](https://en.wikipedia.org/wiki/Default_mode_network) ).

I'm no neuroscientist but I think there's a lot to uncover about how that influences our creativity and maybe that can influence how we design intelligent systems down the line.

</comment></comment></comment><comment><USER5> "I've found there are two types of thoughts especially worth avoiding...

One I've already mentioned: thoughts about money.

The other is disputes. So avoid disputes if you want to get real work done."

<comment><USER6> _"[3] Corollary: Avoid becoming an administrator, or your job will consist of dealing with money and disputes._ "

<comment><USER7> Is that why lawyers do so well? They just get paid to handle this stuff?

</comment></comment></comment><comment><USER8> This essay really gets better with age. When I first read it I was a kid who thought the advice was pointless -- I already spent all my time thinking about physics problems, what else was there to do? But focus on the good stuff got harder with age.

<comment><USER6> Absolutely. For me it's because of two reasons. When you're young, you feel like you have unlimited time. When you're old you know that's not so true anymore. Also, when you're young you have much more time at your disposal. The older you get, the more your job and your family will take a toll on your time, until you realize one day that the only time you have for yourself is when you take a shower.

<comment><USER9> sometimes not even.

</comment></comment></comment><comment><USER10> This where I would like people to consider the Psychological concepts of Focussed Mind and Diffused mode of thinking. Barbara Oakley's course on Coursera named, "Learning how to learn" talks a lot about it. It is an interesting read and would totally recommend it.

So a parallel to the Top Idea in your mind would be a diffused mode of thinking stuck on a single problem. The course touches a lot on this.

</comment><comment><USER11> An addition to money and disputes could be 'things beyond your control.' If something is bothering your mind, yet there is nothing nothing you can, or will be able to, do about it, then there is no reason to bother with it.

</comment><comment><USER12> I'm really suffering from Covid on the brain lately. I have had to just stop reading about global news beyond a skim. Stay safe out there!

<comment><USER13> Just do what everybody else does: try to make Covid charts, or Python models or whatever. Seems to me half the people I follow are somehow coding something related to Covid. I'm updating [http://covid.410go.net/](http://covid.410go.net/) mostly for myself because I want to track the current dimension of Covid (ie. infectious people, not total cases) and at least it's been an interesting exercise in dealing with the somewhat messy data Johns Hopkins kindly provides.

</comment></comment><comment><USER14> "So avoid disputes if you want to get real work done."

There is an exception: When a dispute forces you to view a topic from a different angle, like in rubber duck debugging.

[https://en.wikipedia.org/wiki/Rubber_duck_debugging](https://en.wikipedia.org/wiki/Rubber_duck_debugging)

</comment><comment><USER15> The phrase is "top of mind"

</comment><comment><USER16> It would appear that this applies to politicians pretty much all the time.

</comment><comment><USER17> I like PG for his personal brand building skills he is like Trump[1], but on smaller scale. He can write about anything, and people will call it gold which opens up gigantic possibilities. It's true master level, not like junior PR copycats from Medium trying to make noise with rewritten articles.

[1] Yes, Trump! You can dislike him as much as you want, but his personal brand building is also on master level.

<comment><USER18> PG doesn't brag incessantly. Both produce a large stream of consciousness that may have nice tidbits and sayings. But the real test is to see if such ideas can be turned into a consistent algorithm or formulas that one can test via historical scenarios or in practice. Just because you have a cool-sounding idea doesn't mean it will usually fly in practice.

T has made some interesting claims about success in business, but they conflict with works such as Dale Carnegie's classic HTWFAIP, which better matches life as I have observed it. But some trolls and demagogues do gain success as counted in money and power. There is more to one strategy in life, but HTWFAIP is probably easier for the average person to use effectively. Plus, trolls and demagogues tend to generate new regulations that prevent later T&D's from pulling the same tricks. There's kind of a first-mover- advantage to jerk-hood, like people selling Purell at huge markups.

</comment></comment><comment><USER19> There's an https error. Here is the archive [https://archive.is/NgXhA](https://archive.is/NgXhA)

</comment><comment><USER20> This is one reason why we need a universal basic income for all human beings. See also: [https://www.economicsecurityproject.org/guaranteedincome/](https://www.economicsecurityproject.org/guaranteedincome/)

</comment><comment><USER21> PG's best writings are the ones where he writes about what he knows. He obviously knows a lot about ideas, how to execute on them, raising money, disputes that come with it. This is a very good example.

It's when he writes about things he doesn't know much about that his essays become incoherent and confused.

<comment><USER3> That's not why. From now on PG should only write about things he thought about in the shower.

<comment><USER22> Brilliant.

<comment><USER23> Or bath. Douglas Adams endorsed baths for such purposes!

I'm just glad PG is writing them. I don't mind if they're not always in his sphere. That's the point of essays: to try to sort out our thinking in new areas (from the French word _essayer_ ).

</comment></comment></comment></comment><comment><USER24> No https PG?</comment>


A whole new GitHub Universe: announcing new tools, forums, and features - <AUTHOR> https://github.com/blog/2256-a-whole-new-github-universe-announcing-new-tools-forums-and-features <comment><USER2> Comments moved to [https://news.ycombinator.com/item?id=12499642](https://news.ycombinator.com/item?id=12499642).

</comment><comment><USER3> So this doesn't apply to the github website? Just their standalone offering?

<comment><USER4> This all applies to github.com the website, and will ship separately in upcoming Enterprise releases per our normal Enterprise release schedule.

<comment><USER5> What is the Enterprise release schedule, is that available somewhere to view?</comment></comment></comment>


Exercise and Weight Loss: Shouldn't somebody see if there's a relationship? - <AUTHOR> http://www.stat.columbia.edu/~cook/movabletype/archives/2010/02/exercise_and_we.html

<comment><USER2> "studies that claim that exercise doesn't lead to weight loss, and noticed that the exercise intensity is quite low. For instance, one of the studies had people exercise for an hour at "55% of aerobic capacity"; a different study had people exercise at "70% of maximum heart rate."

</comment><comment><AUTHOR> One possible reason people are reluctant to study this: high-intensity exercise is, or seems to be, much more dangerous.

I sent a marathon-runner this:

<http://lupoleboucher.livejournal.com/30217.html>

She tried it once, threw up, and could do any serious exercise for about a month and a half.

<comment><USER3> That means she had other issues. Tabata is difficult but it exhausts you, it does not make you bed-ridden.</comment></comment>


Show HN: Loc2country – Location coordinates to country in microseconds - <AUTHOR> https://github.com/soorajb/loc2country <comment><USER2> The typical user might not need [https://en.wikipedia.org/wiki/Enclave_and_exclave](https://en.wikipedia.org/wiki/Enclave_and_exclave) support. I think understanding the limitations is important though. Being fast vs being accurate.

Country boundaries change over time and there's many disputed areas (let's say the Kashmir region. Are coastline or maritime borders used (useful for island states)? Explaining how and when the master data was created would help. Ideally a repeatable process. If the master data came from a different source then the possible license should be named.

How are coordinates rounded? I see geohash is used.

Simplified boundaries as input also have an effect.

<comment><USER3> I had the same thought, while geohashes provide a computational speedup this project doesn't seem to be more than a glorified spreadsheet rather than a point-in-polygon search

</comment></comment><comment><USER4> The project includes a data file. I couldn't tell where this data came from, or if there's any way to update it so it's for immediate or approximate use only.

<comment><USER3> It's a CSV file with geohashes and the iso-3 codes that those geohashes belong to

</comment></comment><comment><USER5> I wonder what happens if you give it the coordinates for Bir Tawil.

</comment><comment><USER6> Typically this is done with reverse geocoding. What is the advantage of Loc2country?

<comment><USER7> Isn't loc2country an implementation of a reverse geocoder?

</comment></comment><comment><USER8> How does one refresh the data?

</comment><comment><AUTHOR> Have added details regarding data generation to the README.</comment>


Einstein's 'spooky action at a distance': 10,000 times faster than light - <AUTHOR> http://www.extremetech.com/extreme/150207-chinese-physicists-measure-speed-of-einsteins-spooky-action-at-a-distance-at-least-10000-times-faster-than-light

<comment><USER3> "If it turns out that we actually can communicate data via quantum entanglement, we now know that it'll be much faster than the speed of light."

No, no, no, no, no. No. No. No. Quantum entanglement can not be used to communicate in that manner, on its own. This is well-established, and given the how mathematical the establishment is, it's not going to change anytime soon.

No. No. No. Stop that. No.

<comment><USER4> Yes, they shouldn't get their hopes up too much. Since the Many Worlds Interpretation is the simplest one, it is also the most likely one. With MWI there is no spooky action at a distance, and nothing is actually transmitted faster than the speed of light.

<comment><USER5> _> Since the Many Worlds Interpretation is the simplest one, it is also the most likely one_

The simplest defeasible claim in 2013, is still: "we don't know".

Consider, 'How does the sun come up every day?'

 _Answer A:_ God makes it happen.

 _Answer B:_ The earth is a spinning geoid that rotates eastwards, once about it's axis, roughly every 24 hours. However most of the effect is not related to the orbits of either the sun or earth, with respect to each other. The sun "rises" is more an observational effect of a stationary observer on the surface of the earth, having their field of view rotated.

(For the record, answer B can be improved of course, though that's not the point here)

"Simplicity" does not mean a catchall (Answer A). It also does not necessarily mean linguistic brevity either. There is debate around what it means and it's beyond the scope of my comment[1]. This is the problem with string theory, that also tends to be popular among the MWI crowd too. It's overly broad and the math looks pretty, but those are not sufficient conditions for a good scientific theory. (Oblig xkcd [1])

MWI, like string theory, makes little if any falsifiable and testable claims.

 _[1]<http://plato.stanford.edu/entries/simplicity/> _

_[2]<http://xkcd.com/171/> _

<comment><USER6> Um, no. MWI is simplest in the strict, formal, information-theoretic, Kolmogorov-complexity sense, while fitting the data to equal precision with any competitor. That's kinda the whole point of MWI.

<http://lesswrong.com/lw/q3/decoherence_is_simple/>

[http://lesswrong.com/lw/q4/decoherence_is_falsifiable_and_te...](http://lesswrong.com/lw/q4/decoherence_is_falsifiable_and_testable/)

<comment><USER5> From your link: _"This is Bayes's Theorem. I own at least two distinct items of clothing printed with this theorem, so it must be important." - <USER2>

 _cringe_. "Here's a bunch of _math_ so I must be really smart. Trust me guys." Math is awesome, but waving it around as if to automatically lends credibility to your argument is a poor showing IMHO.

Specifically, quantum decoherence is simply not accepted as a thing that exists on a macro scale. You're post presents no claim that is testable, despite the bold headline to the contrary.

Lets look at an example of a what physicists consider to be a falsifiable claim: The Standard Model predicts the existence of a new elementary particle, the Higgs Boson. This could be tested experimentally in theory, and later it was tested in practice, once the capability existed (LHC).

Or another example: orbital mechanics predicts the location of the asteroid Apophis will pass unusually close (<100 000KM) to earth in 2029.[1]

Now of course if Apophis fails to show up, or shows up overly late or in the wrong place etc, that doesn't necessarily invalidate the theory, but there does eventually need to be a credible explanation.

The simplest is usually measurement error, or silly human error etc. But every now and then, there are discrepancies that just don't go away, and those are often the ones that lead interesting new theories pushing science forward.

TBH I don't think your post does that.

The failing of physicists to take your above link seriously, does not epistemically speaking invalidate the claim of course, but it should be pause for concern.

 _[1]<http://en.wikipedia.org/wiki/99942_Apophis> _

<comment><USER4> MWI certainly _does_ make falsifiable claims. It's just that different interpretations of QM are _extremely_ difficult to experimentally distinguish with any technology we are likely to have in the foreseeable future.

The only well-known interpretation of QM that is experimentally indistinguishable--in theory--from MWI is Bohm's Interpretation. And I think it would be a difficult row to hoe that Bohm's Interpretation is simpler than MWI, as Bohm's universe contains the MWI universe as a subset in the form of pilot waves that never collapse. I.e., the only difference between MWI and Bohm is that in Bohm, one of the "worlds" is granted reality, while the others are not.

Of course, there's nothing in MWI that grants all the worlds reality either. Someone can certainly remain agnostic about that. E.g., Stehphen Hawking has said that MWI is obviously true, though he is agnostic on the reality of the other worlds.

For more information on this, including a tutorial on all the linear algebra that you need to understand, see the book _Quantum Mechanics and Experience_ by David Albert.

<comment><USER5> _> MWI certainly does make falsifiable claims._

Would you mind stating an example of such a claim?

<comment><USER4> Another way in which MWI is falsifiable is that it would be proven wrong if those Chinese physicists were able to show a speed limit on the spooky action at a distance. MWI says that there is no such speed limit, since there is no spooky action at a distance.

Additionally, there are collapse theories that are much easier to test than consciousness-based ones. By "much easier", I still mean fantastically difficult, but should the day come that one of these collapse theories is proven correct, then MWI will clearly have been falsified.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Contrary to the claims of the extremetech article, they have not closed all loopholes in this experiment. Specifically, the detection loophole has not been closed. This loophole is caused by not being able to detect a high enough portion of the photons your source emits. A paranoid person might argue that the ones you do detect are not a representative sample. For example, some devious third party could be choosing which photons you detect in such a way that your results are skewed. Yes... This is a paranoid suggestion, but closing loopholes is all about satisfying the paranoid.

In this paper they invoke the assumption of "fair sampling". i.e. They assume what they do detect is representative. Ergo, they have _not_ closed the detection loophole in this experiment. Fair sampling is a perfectly reasonable assumption to make and is very commonly used, so this should _not_ be interpreted as invalidating their results. It just leaves room for improvement in the future, since single photon detectors with sufficient efficiency to close the detection loophole do now exist (They are very bleeding edge).

Note that the paper is correct and does not make the claim that "all loopholes" have been closed. The extremetech article got it wrong.

<comment><USER8> _the detection loophole has not been closed._

Not completely, but if I'm reading the paper correctly, they claim a detection rate of 91%, which is pretty high. Previous experiments had much lower detection rates. So while it may not completely close the loophole, it does make the detection loophole argument ("the photons you detected weren't a representative sample") much more "paranoid", to use your word.

</comment></comment><comment><USER9> Maybe I'm mistaken, but I thought Einstein coined 'spooky action' sarcastically, to try and point out that it was absurd and violated his theories. I guess there aren't any better, sexy physicist names to put in the headline to attract clicks.

<comment><USER10> I think that's what happend. Schrodinger's cat was also initially conceived to ridicule the idea of quantum superposition.

<comment><USER11> The name Big Bang was also coined to ridicule the idea.

<comment><USER12> I had never heard of that so I tried to find a reference and it appears to be true. <http://en.wikipedia.org/wiki/Big_Bang#Etymology>

</comment></comment></comment></comment><comment><USER13> It's even better in German: "spukhafte Fernwirkung"

</comment><comment><USER14> > One half of the pair was then observed, and the time for the other half to > assume the same state is measured.

How??

How is it possible to observe whether or not an entangled particle has had its state resolved by the other particle being observed?

<comment><USER15> In a pair of quantum particles [1], when one is spinning-up, other is spinning down even when they are separated. Therefore it is possible to find out about the state of one by looking at the other.

[1] <http://en.wikipedia.org/wiki/Quantum_entanglement#Concept>

<comment><USER16> Yes but doing so _collapses the wavefunction_. Hence there's no way to observe _when_ the wavefunction collapses.

<comment><USER17> Many (most?) interpretations of quantum mechanics do not involve a "collapse" of the wavefunction. The Copenhagen interpretation, which is the oldest and better known, does, but the many-worlds interpretation (which I dislike), which is favoured by many working in cosmology and quantum gravity, does not involve a collapse of the wavefunction.

</comment></comment></comment></comment><comment><USER18> The speed of light ought to be enough for anyone.

<comment><USER19> Well, actually it poses a lot of problems. Communications that have back-and- forth transfers can be very slow.

For instance, if I wanted to do a distributed computation with a server on the other side of the world, it would be limited by the speed of light. From where I am to Australia is about 48 ms. So if I want to communicate there and receive a reply, that's about 1/10 of a second right there. I can only do that 10 times a second then. Not so good.

Not to mention the Mars mission has to wait anywhere from 8-19 minutes to get data between earth and the rover.

We could certainly use > c if it was possible.

<comment><USER20> I believe the poster of this comment was being sarcastic, in the spirit of Bill Gates' [debatably misattributed] quote: "640K ought to be enough for anybody."

</comment></comment></comment><comment><USER11> How does this rule out hidden variables?

I am aware of Bells inequality, but how do they show that for this specific experiment there were no hidden variables in play?

<comment><USER7> They test a CHSH Bell inequality in this paper. This rules out local hidden variable theories. Non-local hidden variable (NLHV) theories are another matter entirely. Indeed, some NLHV theories may reproduce the predictions of quantum theory exactly.

</comment></comment><comment><USER21> Screw that, faster than the speed of light should be _back_ in time. I want the information _before_ it was sent!

</comment><comment><USER22> It is so simple, this is the same object seen over 2 places. Think of a fishing tank, the side glass wall and the front glass wall show the same fish. And that's i hope an easy to understand example of an extra dimension, it solves many physics problems add a dimension.. As the fish swims in the tank, both walls show change in position to and no need for communication at all, its just one fish.

you need proof for extra-dimensional existence just take all those experiments that could be solved by it .. double split... spooky action .... and many more.. (even quantum computers inner working)

</comment><comment><USER23> Quantum internet is "starting to become feasible"? It will be in their daily lives so much faster than that.</comment>


I Didn't Hate the English – Until Now (2018) - <AUTHOR> https://www.nytimes.com/2018/10/18/opinion/england-ireland-border-brexit.html#click=https://t.co/Elmh9EXqNc <comment><USER2> Her issues with the English are founded and interesting. But we all feel like that. Our history is complex and bloody, you go to any part of the UK and the people there feel the same way, why are we the butt of jokes? Why do they not care about our history and oppression (the North, Scotland, Cornwall, Wales, any rural part of the UK). While Londoners drone on about London.

Brexit and the Irish border is almost impossible to solve. The talk has been endless, with what? An hour a day on Radio 4 for the past 3 or 4 years. It is complex. It is miles and miles away in a land 99% of us have never been to and can't help with a coherent answer. A hard border is madness. A soft border leaves a gaping hole in the EUs borders. A backstop means Northern Ireland are running a separate legal system to the rest of the UK, and we still have the issue of some sort of border somewhere ... in the UK ports? How many harbours and ports are there that are capable of getting a boat to Ireland? Hundreds?

<comment><USER3> > Brexit and the Irish border is almost impossible to solve.

the solution would be to kick the illegal English occupants off the island of Ireland. also what gets labeled as _terrorist_ in one country is called a freedom fighter in another. The Kurds are a great example of that too.

<comment><USER4> What about those of Norwegian descent? Since they've only been there half as long as the ones of Spanish descent, do you get to 'kick' those out too?

Also, have you read The Terrorism Act 2006, lately?

<comment><USER3> see my other comment. I'm speaking of English occupants who as a minority identify as English and want NI to remain English (e.g. a divided Ireland).

Sorry I couldn't give a rats arse about hypocrite laws that the US enforces on ther countries.

Unlike others here I've lived in conflict zones for 20+ years, and am not an armchair philosopher like most of you who think just because they have an Internet connection it makes them an authority on all sorts of subjects.

</comment></comment></comment></comment><comment><USER5> > Crucial questions about how to avoid a hard border between Northern Ireland > and the Republic — a border abolished in the Good Friday Agreement, the > reintroduction of which would be inextricably associated with the preceding > decades of violence and unrest — remain unresolved, months before Brexit is > slated to become official.

It's interesting how this has taken hold and is not really questioned. There has in fact never been (apart from a brief period in World War II) a border requiring a passport between Ireland and Northern Ireland.

Neither did the Good Friday agreement 'abolish' the customs border in any meaningful way. The Good Friday argument has very little to say on the subject of borders in fact - the customs barriers that there were on the island were in fact abolished in 1993 as part of the common market reforms.

Given that all sides have committed not to re-establish a hard border even in the event of a no deal, it's difficult to see how it would threaten the peace process to the extent that is being made out.

<comment><USER6> > Given that all sides have committed not to re-establish a hard border even > in the event of a no deal, it's difficult to see how it would threaten the > peace process to the extent that is being made out.

I'm rather ignorant of the situation so I ask sincerely: Theresa May might have made a "commitment" and stated that she's "determined" not to see a hard border but how is she going to do it? I'm guessing the EU's & the UK's red lines plus Parliament rejecting her deal are a bit of a spanner in the works? If she doesn't have a credible plan, you'd forgive people for being worried about a hard border, surely?

<comment><USER5> The EU have committed to Ireland, and the Irish government themselves have committed that there wouldn't be a hard border. The U.K. government has also committed not to build one. It's not just the U.K. that has committed not to build one - it's all parties involved in this that have committed not to build one.

In addition, there are other borders between EU and non-EU countries that don't require immigration controls. It's hard to see why one is necessary - other than if the EU was deliberately setting out to make leaving look as difficult as possible.

</comment></comment></comment><comment><USER7> As an American of Irish heritage - my family until very recently lived in an Irish American community in South Boston for generations - I can't stress enough how little I share these feelings. I know I'm not "Irish", but my life is presumably a consequence of the same decisions that the author is upset about. It seems that the author is using historical events to justify political resentments that are very current and very local. To me that is unhealthy.

<comment><USER5> > It seems that the author is using historical events to justify political > resentments that are very current and very local.

There have also been very real consequences of that lingering political resentment. For years, Irish Americans from Boston and NY were the primary source of funding for the Irish Republican bombing campaign in Britain.

[1] [http://www.washingtonpost.com/wp- dyn/content/article/2005/08...](http://www.washingtonpost.com/wp- dyn/content/article/2005/08/02/AR2005080201943.html)</comment></comment>


How does Twitter make money? - <AUTHOR> http://www.quora.com/Twitter-1/How-does-Twitter-make-money

<comment><AUTHOR> What they should do is offer two options: either user-pays at some minimal amount per year (e.g. $5-$10) or advertising-funded where the user can choose to follow certain companies or brands.

<comment><USER2> The trouble is, the advertisers want exactly those users who would be willing to pay money; they're high-value eyeballs. By allowing their most valuable users to opt-out, they devalue themselves as an advertising venue. Of course, they also decrease the available impressions if a significant number of people opt out, making them a less attractive venue again.</comment></comment>


Show HN: Paper by FiftyThree – 33% more installs with screenshots A/B testing - <AUTHOR> https://splitmetrics.com/blog/paper-by-fiftythree-how-to-get-33-more-installs-with-screenshots-ab-testing/ <comment><USER2> Show HN is reserved for usable things and specifically not for content.

[https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html)</comment>


Breaking the trillion-rows-per-second barrier with MemSQL - <AUTHOR> http://blog.memsql.com/shattering-the-trillion-rows-per-second-barrier-with-memsql <comment><USER2> I have doubts about the "We're actually spending some time on every row" claim. Saving data using columnstore often comes with meta-data saved at the page level, such as min, max and count values for the data in that page. These values are used to filter and optimize the flow of data processed for a query (I mean 'skipped' here). If you run a 'count' query 10 times, it's very unlikely the DB will count rows 10 times. It will rely on the page's existing meta-data when available (i.e., already computed). The tests described in the post are misleading IMHO.

EDIT: This comes on top of the fact that DBs can store queries results too. Moreover the post does not tell whether they have implemented clustered or filtered indexes on the considered columns. It does not explain how partition has been performed too. All this has a big impact on execution time.

<comment><USER3> The query in the example is not "count" but a count + group by query. While it is possible to precompute group by results on every possible columns per page we don't do that and this query does touch every row. The article touches on how this is possible: operations on encoded data, AVX2, not using a hash table for the intermediate result of the group by. And we certainly don't fake it with storing query results.

We guarantee that the result is legit.

<comment><USER4> What does the columnar format look like? Particularly, is the group by column compressed with RLE? That's kind of a pre-computed group-by + count that would make this kind of query very very fast :)

<comment><USER3> This is an astute observation. Here is the accurate answer from the author of the feature: "We do have special handling for RLE for filters and group by column but in both cases there would be per row component (updating selection vector for filter on RLE, updating aggregates for group by on RLE) and so far I saw RLE encoding usually slower than dictionary encoding for both these cases"

</comment></comment></comment></comment><comment><USER5> Impressive although reading that they used 448 cores on super-expensive Intel lab machines takes the edge off a little

<comment><USER6> 1,280,625,752,550 rows per second is pretty impressive even if it was done on a million dollar's worth of CPU.

<comment><USER7> And do you have a business use case that requires that?

<comment><USER8> I've worked on a ton of businesses cases that require it. This is what I worked on for years.

It's generally a large company, large issue sort of problem. You can throw millions in hardware/software into a problem that saves you 100's of millions.

<comment><USER3> We can solve it quite efficiently. You can define an index on a columnstore table and we will maintain order on a per node basis

</comment></comment></comment></comment></comment><comment><USER9> So some questions::

1\. Isn't (3)vectorization and (4)SIMD the same thing ?

2\. I don't see the data-size before-after compression ?

3\. How much RAM has each server ?

4\. How do all cores work for all queries ? Is the data sharded by core on each machine or each core can work on whatever data ?

5\. What's a comparison open-source tool to this ? Only I can think about is snappydata.

<comment><USER10> SnappyData employee here. In general this is called the "HTAP" industry (Gartner's phrase: Hybrid Transactional/Analytical Processing).

SnappyData: [https://www.snappydata.io](https://www.snappydata.io), MemSQL: [https://www.memsql.com/](https://www.memsql.com/), Splice Machine: [https://www.splicemachine.com/](https://www.splicemachine.com/), SAP Hana: [https://www.sap.com/products/hana.html](https://www.sap.com/products/hana.html), GridGain: [https://www.gridgain.com/](https://www.gridgain.com/)

are some of the technologies within it

</comment></comment><comment><USER11> Meh. They used 448 cores to count the frequency of bit patterns of some small length in a probably more or less continuous block of memory. They had 57,756,221,440 total rows, that are 128,920,138 rows per core. If the data set contained 256 or less different stock symbols, then the task boils down to finding the byte histogram of a 123 MiB block of memory. My several years old laptop does this with the most straight forward C# implementation in 170 ms. That is less than a factor of 4 away from their 45.1 ms and given that AVX-512 can probably process 64 bytes at a time, we should have quite a bit room to spare for all the other steps involved in processing the query.

Don't get me wrong, in some sense it is really impressive that we reached that level of processing power and that this database engine can optimize that query down to counting bytes and generating highly performant code to do so, but as an indicator that this database can process trillions of rows per second it is just a publicity stunt. Sure, it can do it with this setup and this query, but don't be to surprised if you don't get anywhere near that with other queries.

<comment><USER12> > My several years old laptop does this with the most straight forward C# > implementation in 170 ms.

Sure, but writing a custom C# program for each query on data that has been pre-formatted in the most optimal manner for that program is not really comparable to writing a bog standard SQL query.

<comment><USER11> I am not sure I understand what you want to say but I assume you want to say that what I did was kind of cheating because I wrote code tailored to that problem and to work on data in a format that makes that problem rather easy? Correct me if I am wrong.

But they did exactly the same just that the query planer, optimizer, and compiler generated the code from the SQL query. They still picked a data set, a physical layout of that data set, and a query that would result in maximum throughput. That was not any random SQL query, it was a carefully picked one, chosen because it would be able to take advantage of all possible optimizations.

</comment></comment></comment><comment><USER13> And of course, how does it compare to kdb? So it seems less expensive, but also lacks the advanced query language.

The last tests I saw for kdb was the 1.1 billion taxi ride.

[http://tech.marksblogg.com/billion-nyc-taxi- kdb.html](http://tech.marksblogg.com/billion-nyc-taxi-kdb.html)

Where it basically outperformed every other CPU based system with slightly more complex queries.

Any comparisons planned?

<comment><USER14> It'll be faster than Redshift and about the same as ClickHouse, +/\- depending on hardware and setup.

It's a great system, we used it for 2 years and it's one of the most polished databases out there with a simple MySQL interface. It's more general purpose than kdb, with a nice rowstore + columnstore architecture. I believe they're adding full-text search indexes in the latest version too.

If you need the query language, the advanced/asof joins, or the tightly integrated query/process environment, then there's no match to kdb though.

</comment></comment><comment><USER15> "When you deliver response time that drops down to about a quarter of a second, results seem to be instantaneous to users."

I don't think everybody agrees with this statement.

<comment><USER16> The threshold for most people is slightly lower than a quarter second, but it's certainly the case that above a quarter-second users will notice the delay[0].

However, people may be primed for longer delays still seeming instant. e.g. Smartphones for many years had a built-in 300ms delay on any click event, and even without that event typically still have delays on many 'instant' actions.

So while the delay will be registered as being present, it may not be registered as "this site is slow" but "it's just a natural delay".

[0] [https://psychology.stackexchange.com/questions/1664/what- is-...](https://psychology.stackexchange.com/questions/1664/what-is-the- threshold-where-actions-are-perceived-as-instant)

<comment><USER17> The threshold for noticeable touch "rubberbanding" is 1 ms:

[https://www.youtube.com/watch?time_continue=52&v=vOvQCPLkPt4](https://www.youtube.com/watch?time_continue=52&v=vOvQCPLkPt4)

</comment></comment></comment><comment><USER18> If all this data just fits in memory then what is surprising about the speed?

<comment><USER19> Well, you can try and run MySQL or Postgre off a RAM drive, but I suspect you won't get close to this result. And then your data is not on a reliable storage media.

<comment><USER18> Can't talk about MySQL or PostgreSQL, but with Oracle I am pretty sure you can get very much close to this result with cached data. And this is without using the in-memory option. However, talking about it withouth any experiments to prove a point is just speculation. Memory vs Disks changes a lot performance- wise.

<comment><USER20> It was expressly forbidden in the software contract to benchmark Oracle back in the 80s.

[https://danluu.com/anon-benchmark/](https://danluu.com/anon-benchmark/)

<comment><USER21> It's forbidden to publish the benchmarks, not to make them. And it's because most users are rubes who can't tune 2 databases at the same time.

</comment></comment></comment></comment></comment><comment><USER22> How fast is data import? Loading into RAM? (for example booting up a cluster for an existing imported database on AWS)

Working with some datasets with 100s of billions of short rows, curious to give it a try.

<comment><USER3> We have seen 1Gb/s streaming ingest into MemSQL. Also feel free to join [https://memsql-public.slack.com](https://memsql-public.slack.com) where people help with developer edition.

<comment><USER22> With how many nodes? Does it scale linearly with nodes? (1Gb/s by itself doesn't help me estimate the scale of the project), Thanks!

<comment><USER3> Data ingest scales linearly assuming the source is scalable (S3, HDFS, or Kafka). There are other things that matter: how wide is the table, what data types, etc. We achieve 1GB/s on a 16 nodes cluster for some combination of the above. What is your target?

</comment></comment></comment></comment><comment><USER23> The speed of light is a hard limit. I don't believe there is any free lunch[1], but trade-offs to manage. I'm skeptical of any claim that implies free or easy speed without potentially significant trade-offs.

If you can live with somewhat out-of-date and/or out-of-sync data, you can throw mass parallelism at big read-only queries to get speed. The trade-offs often are best tuned from a domain perspective such that it's not really a technology problem, although technology may make certain tunings/tradeoffs easier to manage.

[1] (Faster hardware may give us incremental improvements, but the speed of light probably prevents any tradeoff-free breakthroughs.)

<comment><USER24> What are you on about? The author is counting rows in a database and checks how much time it takes the database to process the query. Counting rows is easily parallelisable.

So are many problems, and we still are at a point where we are creating languages and frameworks that allow us to work with the data at a higher level: e.g. theano, numpy or TensorFlow.

Speed of light is only an issue for latency-sensitive operations: e.g. if every sub-operation is non-commutative in a given transaction then it would be expected to be completed before any other sub-operation.

<comment><USER23> Re: _Speed of light is only an issue for latency-sensitive operations_

That was my point: if you don't care about data being "fresh" and consistent (per transaction coordination) you CAN throw mass parallelism at the problem. If you do care, then the speed of light is the bottleneck.

<comment><USER23> Clarification: "mass parallelism" may also involve distributed and duplicated data sets.

</comment></comment></comment></comment><comment><USER25> How fast can it sort those rows?

<comment><USER3> Can you share the whole use case as well as data size?

</comment></comment><comment><USER26> Who from HN would need this and why?

Serious question. I would like to know different real use cases from people on HN, given our backgrounds.

<comment><USER27> We actually use memsql for almost exactly this use case. Realtime arbitrary grouping and filtering on multiple (30+) columns on a billion+ rows with substantial concurrency (reporting tools and api's of them). Previously with pgsql this required a lot of fine tuning of indexes and lots of etl scripting to break things into separate dimension tables, and then large join queries to pull them back. This is expensive at the development and operational level, and data was batched in which involves delays. It was also extremely resource intensive to query and moderate levels of concurrency required a master and several slaves, with response times for anything more than a week of data requiring multi-second waits, with the worse cases approaching the minute mark.

The hardware in this example is overkill and impractical for most use cases - to say the least. For our setup, Memsql does this for us on a single node with 256Gb of ram, 40 cores (1 aggregator and 4 leafs) and a modest enterprise nvme ssd. The machine cost $4,500 over a year ago. Adding more machines to mem is pretty trivial should we ever need to partition this across machines, despite this not being necessary.

There are some gotchas and it should not be consisted a drop-in replacement for MySQL.

<comment><USER28> So, you're still shipping data between your primary datastore and memsql, or you've switched entirely on memsql ?

<comment><USER27> Originally it was just a port, but now the inserts go straight into mem. This used to be a big no-no on mysql (with inno and myisam anyway) as it would invalidate a lot of query cache on every insert. Here you can refresh the query every second and see the counts go up.</comment></comment></comment></comment>


Firefox 15 arrives, supports compressed textures for impressive 3D gaming - <AUTHOR> http://arstechnica.com/information-technology/2012/08/firefox-15-arrives-supports-compressed-textures-for-impressive-3d-gaming/ <comment><USER2> I had to do some testing and installed latest versions of Firefox, Chrome, IE and Opera. To my surprise, Firefox seems to have improved a lot in the last few versions. It feels much snappier and more responsive than Chrome, which was not the case for a long time. Looking at other stuff, like download page experience, anti-aliased fonts on pages by default and overall look and feel, one can see that Mozilla has put a lot of effort into fixing the problems and trying to get back to #1 spot in terms of being the most advanced browser.

Competition from Chrome has awaken them from sleep and finally we are starting to see a very fast, slick browser. Given all the plugin ecosystem, Firefox could be back on top. The only problem is that they might be a little bit too late with these improvements, and it would be hard to win back the users that switched to Chrome. Most of my friends who switched won't even be bothered to try new Firefox. They just discard it, the same way majority of ex-IE users discards newer versions of IE without even trying once.

<comment><USER3> I don't think Firefox was ever asleep. They just had different priorities during the time when Chrome was advancing primarily in performance.

<comment><USER4> I'm a Mozilla employeed, and I think it's fair to say that Chrome's presence caused us to increase the focus on Firefox performance. E.g. <https://wiki.mozilla.org/Performance/MemShrink> and <https://wiki.mozilla.org/Performance/Snappy>.

</comment></comment></comment><comment><USER5> I switched back from chrome to FF few months ago, well I never left really due to firebug and client certs, but now I really don't use chrome anymore. FF is as fast as chrome to start, ie turning off tabs autoloading. I never lost my history with FF while it happened too many times with Chrome. Extensions integration is a lot better in FF, try LastPass for example. FF can have multiple rows of tabs with an extension, I don't think that's possible in Chrome. chrome text search is light years back from FF. And finally, Chrome way of hiding privacy settings, ie delete cookies, is annoying, obviously Google doesn't have any interests on my privacy. FF has been terrible and slow for at least 4 years, but it's back!

<comment><USER4> Try the "Tree Style Tab" extension. It puts tabs on the side, so you can fit many more of them, and you can even collapse and expand sub-trees.

<comment><USER6> Yes Tree Style Tab is a must. Chrome and Chromium do not offer vertical tab support on Linux.

</comment></comment></comment><comment><USER7> I'm actually excited about the second part: The new version also significantly reduces the memory overhead of leaky add-ons.

I'm going to download a bunch of downloads and see how it feels now.

<comment><USER8> Don't they say that EVERY firefox release?

<comment><USER9> Yes, they do.

Whenever the topics of Firefox's memory leaks or performance problems come up, there are always people who claim that it has none. They hold this position even when presented with mountains of evidence to show that there are in fact problems. Then the next version of Firefox is released, and the release notes go on and on about how memory leaks and performance problems have been fixed. Strange, how could they fix these problems that supposedly never existed in the first place?

<comment><USER10> The MemShrink project has been in progress for over a year now, with measurable results shipping in many updates. The specific bullet point here is a fix to a very common addon problem that has been shown to have real-world benefits.

Nobody here is denying there's a problem, things are getting fixed.

</comment></comment></comment></comment><comment><USER11> i have always stuck with FF for firebug and because of how much i can customize every single aspect to my needs. i think opera fits the bill as well, but something about its UI doesn't work for me. ff is plenty fast and i dont need to deal with untweakable interface decisions as i do in chrome. i have said for a long time that i would give chrome another try when they give me an about:config equivalent and a fully customizable ui, but that's not in the cards.

</comment><comment><USER12> Does anyone know what the status of the new Australis theme is? I think FF is really in need of an update and the mockups of Australis that I saw looked really nice to me.

</comment><comment><USER13> No HiDPI support yet. They are slacking!

<comment><USER14> Is it really a priority? The vast majority of Firefox users do not own the new MBP.

<comment><USER15> More of them are being sold every day, and the pace is only set to increase as it's pretty much inevitable that Apple will release more high-resolution devices as time goes on. Might as well get it over with now before Firefox gets labeled as "the ugly one on high-res devices".

<comment><USER3> Are we ever going to see a high resolution Windows implementation?

<comment><USER14> Windows 8 will support multiple DPI values, at least for Metro.

</comment></comment></comment></comment></comment><comment><USER16> I wish the gui interface was more like chrome. The tab management seems clunky.

<comment><USER17> The way Chrome handles a large number of open tabs is one of the reasons I don't use it. In fact, out of all the things that could possibly bother me about Firefox, the interface isn't one.

<comment><USER18> The same for me. My wife uses Chrome and when I sometimes have to use it on her laptop it really makes me mad when I have to find a tab when the only thing I see is the favicon.

I tried Chrome on my linux and I much more prefer Firefox (aurora channel)

</comment></comment></comment><comment><USER19> Mozilla is shining these days! Introducing awe techs (Not to mention, servo!!).

</comment><comment><USER19> Mozilla is shining these days! Not to mention servo!!

</comment><comment><USER20> can someone just email me when it's faster than chrome? thx

<comment><USER21> Performance and stability wise based on some use today it's a step forward.

It's still fugly sadly.

<comment><USER22> Finally, someone who understands! Prettiness really is the most important feature of any browser.

<comment><USER21> You need to compete on all fronts and decent UI is one of them. Firefox, for me at least, is still behind in this area (though I loved the Australis stuff that was being put forward).</comment></comment></comment></comment>


The Death of App Attribution - <AUTHOR> https://hackernoon.com/the-death-of-app-attribution-abb6f370d0c7 <comment><USER2> Death of attribution? That sounds good to me. Personally, I do everything that I can to deny marketers as much data as possible, including attribution. If they don't like that, they only have themselves to blame. They started this war against people, after all.</comment>


The phrase "social proof" seems to have originated online in 2007 - <AUTHOR> http://www.google.com/trends/explore#q=%22social%20proof%22

<comment><USER2> It appears to come from much earlier than 2007. See for example: Cialdini, Robert B.; Wosinska, W.; Barett, D. W.; Gornik-Durose, M. (October 1999). "Compliance with a request in two cultures: The differential influence of social proof and commitment/consistency on collectivists and individualists". Personality and Social Psychology Bulletin 25 (10): 1242–1253. doi:10.1177/0146167299258006

Robert Cialdini may be the one who is most associated with this phrase. A few other items which reference it are:

Cialdini, Robert B. (October 2001). "Harnessing the science of persuasion". Harvard Business Review 79 (9): 72–79.

Cialdini, Robert (1993). Influence (3rd ed.). New York: HarperCollins.

So, I suppose the real question is, why does Google trends show a beginning in 2007?

Edit: There appear to be a number of articles about Cialdini's book in 2007. It appears that there was a Washington Post article in 2007 that spurred the interest spike: [http://www.washingtonpost.com/lifestyle/magazine/pearls- befo...](http://www.washingtonpost.com/lifestyle/magazine/pearls-before- breakfast-can-one-of-the-nations-great-musicians-cut-through-the-fog-of-a-dc- rush-hour-lets-find- out/2014/09/23/8a6d46da-4331-11e4-b47c-f5889e061e5f_story.html)

<comment><USER3> Not knowing anything about "social proof" or why it's important to know when it appears online, but the term appeared online well before 2007. For example, [https://web.archive.org/web/20000511115153/http://www.isoc.o...](https://web.archive.org/web/20000511115153/http://www.isoc.org/inet99/proceedings/3g/3g_2.htm) (from 2000) says:

> Social proof. In many social situations, one of the mental shortcuts on > which we rely, in determining what course of action is most appropriate, is > to look to see what other people in the vicinity are doing or saying. This > phenomenon, known as social proof, can prompt us to take actions that may be > against our self-interest without taking the time to consider them more > deeply. Cults from the Jonestown Temple to Heaven's Gate, for example, > provide cogent evidence of how strong the effects of that phenomenon can be > in the right circumstances. [19]

Therefore, it's likely an artifact of Google trends.

Various sources cite: White, S. H. (1977). Social proof structures: The dialectic of method and theory in the work of psychology. Life-span developmental psychology: Dialectical perspectives on experimental research, 59-92 as the source of the term.

</comment></comment><comment><AUTHOR> Posted this out of curiosity -- does anyone know what "patient zero" is for this meme?

It's a meme that I find possibly harmful, since it amounts to the idea that herd effects and other forms of social confirmation bias are a good thing and that these biases should be intentionally built into the decision making process. It also carries some other questionable baggage / corollaries:

\- Secrets do not exist, hence trade secrets and contrarian business models or scientific ideas are rare or nonexistent.

\- Markets are almost perfectly efficient -- including the marketplace of ideas.

\- Conceptual thought and first-principles reasoning are very unreliable.

\- In addition to intellectual biases toward cargo cultism and herd effects, it also introduces a powerful bias toward cronyism and "old boy networks."

I'm also curious if the original idea was possibly more nuanced than that. It's very common for a nuanced idea (e.g. disruption) to get very dumbed down by the game of telephone.

Related: [http://fivethirtyeight.com/features/the-conventional- wisdom-...](http://fivethirtyeight.com/features/the-conventional-wisdom-on- oil-is-always-wrong/)

Social proof would seem to be an intrinsically _bad_ thing in investing -- so why is this concept so popular among investors?</comment>


Show HN: RSA Public Key Encryption in Python - <AUTHOR> https://github.com/ridwanmsharif/prsa <comment><USER2> Congrats on implementing Textbook RSA. Unfortunately as implemented this isn't really secure in any sense.

I'd encourage you to start reading here: [http://crypto.stackexchange.com/questions/1448/definition- of...](http://crypto.stackexchange.com/questions/1448/definition-of-textbook- rsa)

And here is a full explanation: [http://www.di.ens.fr/~pnguyen/pub_BoJoNg00.htm](http://www.di.ens.fr/~pnguyen/pub_BoJoNg00.htm)

<comment><AUTHOR> agreed, this was purely experimental and intended for learning purposes, not production use. I've just added a disclaimer, thanks for providing those links, I'll definitely be going through them

<comment><USER3> I think you should put the disclaimer at the top of the github page, and clearly state that it is not secure.

The disclaimer currently is near the end of the document and says only "Purely experimental project. Designed for learning purposes not production use.".

Some people will miss that at the bottom of the page (after how to install, how to use, etc.), or might not realize that textbook RSA is insecure ([https://crypto.stanford.edu/~dabo/courses/cs255_winter00/RSA...](https://crypto.stanford.edu/~dabo/courses/cs255_winter00/RSA.pdf)).

</comment></comment></comment><comment><USER4> Yeah, don't actually use this. It's vulnerable to stuff that's been out of style for 20 years.

Strong suggestion: try [https://nacl.cr.yp.to/](https://nacl.cr.yp.to/) instead.

<comment><AUTHOR> for sure, just added a disclaimer. worked on this merely to learn and get my hands dirty with python and building Clis</comment></comment>