Leaving Chef - <AUTHOR> https://sethvargo.com/leaving-chef/

<comment><USER2> Wow, death threats. What is this world coming to? How does something like evangelism get escalated like this?

Seth, I hope your sabbatical gives you a buffer from this abuse.

<comment><USER3> Online death threats are very common.

The numbers are a bit confusing but English police recieve a lot of calls about this kind of stuff: [https://fullfact.org/crime/college_policing_bbc_correction_a...](https://fullfact.org/crime/college_policing_bbc_correction_article_social_media_half-33333)

[http://m.bbc.co.uk/news/uk-27949674](http://m.bbc.co.uk/news/uk-27949674) "social media 'at least half' of calls passed to frontline police".

I'm not trying to suggest that toxic online comments are acceptable; or that people who are the targets of such comments should just ignore them.

<comment><AUTHOR> Those numbers (while confusing) definitely sadden me. While I don't fathom the absolute values, it is clear that the trend is upward :(. This is especially concerning because it increases the signal-to-noise ratio and makes it difficult for police to response. It's like the boy who cried wolf.

</comment></comment></comment><comment><USER4> Seth, I've been using Chef for a few years and your tools and those I see you contributing to have been a massive help. Your all around enthusiasm for the community (and product) is inspirational.

Thank you so much for all your efforts.

Thanks also to all the like minded people maintaining and contributing to other projects that make the Chef community so rich.

<comment><AUTHOR> Thank you!

</comment></comment><comment><USER5> Death threats? What is going on here?

<comment><AUTHOR> Unfortunately some people are just mean and hateful. I think that is really what is going on. But without a way to combat that (like when you are on a lonely island), it sucks.

<comment><USER6> It doesn't help that criticism of crappy behavior and abuse is met with cries of "Free Speech" and accusations of being too sensitive. We as a community/industry need to start calling out abusive behavior for what it is.

<comment><AUTHOR> Yea. This comment was especially hurtful to me: [http://www.reddit.com/r/sysadmin/comments/2equq9/leaving_che...](http://www.reddit.com/r/sysadmin/comments/2equq9/leaving_chef/). The problem with most FOSS is that there's no mechanism for calling out said people, and there's little or no punitive action that can be taken.

Sure, you can block someone on GitHub, but that only causes more drama (trust me, I know).

I would love to find a solution. I'm all ears.</comment></comment></comment></comment>


Multiplying two 3-digit numbers together mentally in just one line - <AUTHOR>

http://mathema-tricks.blogspot.com/2012/04/multiplying-two-3-digit-numbers.html <comment><USER2> Welcome back.

Flagged. Again.

</comment><comment><AUTHOR> [http://mathema-tricks.blogspot.com/2012/04/multiplying- two-3...](http://mathema-tricks.blogspot.com/2012/04/multiplying-two-3-digit- numbers.html)</comment>


Scientists say dolphins should be treated as 'non-human persons' - <AUTHOR> http://www.timesonline.co.uk/tol/news/science/article6973994.ece

<comment><USER2> _Marino and Reiss will present their findings at a conference in San Diego, California, next month, concluding that the new evidence about dolphin intelligence makes it morally repugnant to mistreat them._

It is morally repugnant to mistreat animals regardless of the intelligence of the animal and being a scientist has nothing to do with making such a judgment, since it is not a scientific judgment but an ethical one. Thus, the new-found evidence serves only to dramatize something these scientists want because of their philosophical predispositions, not because of their knowledge as scientists. If the logic suggested in the article were true, the converse might be up for consideration as well (e.g., that we can be cruel to deer because they are so dumb).

Science and advocacy are distinct and should be kept that way. Scientists _can_ be advocates but, when they are, they are not acting as scientists but as everyone else who seeks to further an ethical position they believe to be right.</comment>


Extending Patents for Antibiotics Creates Perverse Incentives - <AUTHOR> http://www.citizenvox.org/2012/06/12/antibiotic-resistance-gain-ac/

<comment><AUTHOR> Summary: extending patent duration will increase illness and death.

1\. Antibiotics lose effectiveness when more frequently used.

2\. New antibiotics need to be invented or common infections will become fatal.

3\. Extending patent duration encourage drug makers to sell higher volumes of older antibiotics.

4\. This makes old antibiotics less effective and reduces incentives to invest R&D in new antibiotics.

<comment><USER2> Good summary of the article, but I'm not sure I agree with the logic.

If I have a 5 year patent, I have 5 years to make my high-markup profit. If I have a 10 year patent, I have 10 years to make a high-markup profit.

My R&D costs are constant. I need to make at least, say, 10x my R&D profits to stay in business (a lot of products don't pass trials).

So if I have 5 years to amortize the R&D costs instead of 10 years to amortize the R&D costs, I would have to push the antibiotics a lot harder and convince doctors to prescribe them in a lot more instances in order to make the money back.

Whereas if I have 10 years, I can potentially recoup the investment and make a decent profit at half the rate of prescription in the wild. If it were even longer, I'd be able to afford to amortize the R&D costs over an even longer period.

How about instead the patents on lifesaving drugs could actually be extended a LOT more as long as the company agrees to specific capped/regulated licensing costs after an initial period. In other words, after a short patent monopoly expires (3 years, say), in order to keep a monopoly they agree to be regulated in their sub-licensing terms so that the drug can become affordable, but the discovering company still gets an income stream from it (maybe for 20 years from discovery? longer?).

Or give the company options: 5 years total monopoly, 3 years monopoly+17 years regulated sub-licensing, 2 years + 40 years regulated sub-licensing, 1 year + 60 years regulated sub-licensing. Depending on the product, having a long term income stream might end up more valuable than trying to gouge people for 5 years and then making practically nothing.

Of course that assumes that companies are willing to think in terms of long- term profit as opposed to maximizing the current quarter, which is probably the reason this suggestion would never work. Oh well.

<comment><AUTHOR> Let's say I invented a new antibiotic 12 years ago, and I'm collecting monopoly profits due to 17-year patent protection.

Since the patent term expires in 5 years, I should invest $1 billion in R&D right now, and work around the clock to get an innovative new drug on the market before my monopoly turns into commodity.

But if the patent term is extended for an additional 17 years, I'm no longer in such a hurry to put $1 billion at risk.

Instead, I'll milk every last drop of monopoly profits from this drug. Even though superbugs are becoming more resistant every day, my product is still one of the best on the market.

Maybe I'll invest $100 million a year in R&D, but I can certainly afford to take my time, and more importantly, to take fewer risks.

It's good news for my bank balance, and bad news for patients suffering from drug-resistant bacteria.

<comment><USER2> OK, that's reasonable, but the company is ALSO motivated to dump as many pills on the market as possible, since their profits go to almost zero after generics are available.

</comment></comment></comment></comment><comment><USER3> Not a summary. But, relevant questions:

1\. How do we reduce the costs of drug R&D? Can we crowdsource the process?

2\. How can we change the system for getting credit(money) for creating a new drug?

3\. Given that antibiotics continually require to be made in stronger/different strains, is the use of antibiotics a flawed way of treating patients?

4\. Why are people falling sick?

</comment><comment><USER4> >However, monopoly protections conflict with the need for preservation by encouraging companies to sell as much of the new drug as possible.

That is completely nonsensical economic reasoning. A monopoly _reduces_ availability. Less competition => higher prices => lower quantity demanded.

<comment><USER5> My interpretation of that sentence was that, with limited-time monopoly protection, it's in the producer's interest to sell as much of the stuff as possible before the monopoly ends, after which point it won't be possible to make significant profits from the same product. I haven't yet figured out whether this makes sense.

<comment><USER4> Hmm, so the idea is that a company will actually sell _below_ the price you'd later see in a competitive market? That's an interesting proposition, but it doesn't hold water.

Is it possible in theory? I doubt it, but I can't disprove it off the top of my head. But it just doesn't happen in the pharma industry. When a drug patent runs out, generics come out and the price drops dramatically, every single time. It is true that name brand prices do actually increase when patents expire, but not the generic prices, and that means that quantity supplied will certainly increase once generics are introduced.</comment></comment></comment>


HP Grooming webOS For Assault on Android, Apple iOS - <AUTHOR> http://www.eweek.com/c/a/Mobile-and-Wireless/HP-Grooming-WebOS-for-Assault-on-Android-Apple-iOS-707919/

<comment><USER2> Color me skeptical. The whole idea of placing webOS on PC's strikes me as a "well what else are we going to do with it" moment rather than a well thought out plan.

They're going to have to show me a lot more before I take their plans seriously.</comment>


How some animals use the Earth's magnetic field to navigate - <AUTHOR> https://www.economist.com/the-economist-explains/2018/09/25/how-some-animals-use-the-earths-magnetic-field-to-navigate <comment><USER2> If the magnetic field flipped 780k years ago [0] and animals do rely on it for survival (like migrations), wouldn't we see mass extinctions around those magnetic flips? Or maybe magnetic field navigation is a weak signal out of many?

[0] [https://www.livescience.com/18426-earth-magnetic-poles- flip....](https://www.livescience.com/18426-earth-magnetic-poles-flip.html)

<comment><USER3> If animals learn where to migrate from their parents/flock, then it would be reasonable to assume they learn the relation between where they should be going and what their magnetosense is telling them. The article implies that even during transition, the magnetic field is pretty much constant within a lifetime of any individual animal.

Now a tangential question I have is: how the hell do animals know where they should migrate? Is this learned from observation, or do they have some innate idea of where they should go?

<comment><USER4> I recently learned that salmon find their way back to their birth stream partly by smell: [https://www.scientificamerican.com/article/how-do- spawning-f...](https://www.scientificamerican.com/article/how-do-spawning- fish-navigate-back/)

</comment></comment></comment><comment><USER5> Cows face north-south when grazing.

<comment><USER6> That was one of the most interesting applications of Google maps that I know of.

</comment></comment><comment><USER7> So this is pretty incredible. We know that animals can detect magnetic fields, but we don't know how they sense it!

I hope one day humans can have the same sense.

<comment><USER8> We actually kind of have a few putative mechanisms.

Certain proteins in eyes of some birds are magnetosensitive.

[https://www.sciencealert.com/birds-see-magnetic-fields- crypt...](https://www.sciencealert.com/birds-see-magnetic-fields-cryptochrome- cry4-photoreceptor-2018)

Pigeons apparently may have magnetosensitive balance sense for some reason.

[https://www.nature.com/news/pigeons-may-hear-magnetic- fields...](https://www.nature.com/news/pigeons-may-hear-magnetic- fields-1.10540)

</comment></comment><comment><USER9> Answer: We don't know yet.

<comment><USER10> Do we really not know, though? "Birds can see Earth's magnetic fields because of cryptochromes in their eyes" [https://news.ycombinator.com/item?id=16823201](https://news.ycombinator.com/item?id=16823201)

<comment><USER8> It's a putative mechanism not a fully explored one.</comment></comment></comment>


Basic Tax Law That Every Start-Up Must Know: the 83b Election - <AUTHOR> http://www.fairmark.com/execcomp/sec83b.htm

<comment><USER2> Doesn't everyone know this? That's the best deal I know for getting stock in startups. Don't get options, get stock with a vesting schedule. Get it at fair price when it's super-cheap (a few cents per share). Buy the stock outright for fair market value, therefore you pay $0 in taxes.

<comment><USER3> Purchase also starts the clock for determining the difference between long- and short-term capital gains. This is (currently) a big deal for federal income tax.

Taxes on the difference between the purchase price and the "value" on the purchase date is what caused a lot of people problems during the dot-bomb. They exercised in the money options, owed taxes, and couldn't pay them when the stock price dropped. (Many of them were trying to hold out for long-term capital gains rates. Oops.)

Note that CA taxes both long- and short-term capital gains at the same rate. Jim Clark says that's why he no longer lives in CA.

</comment></comment><comment><USER4> Please don't use all-caps in post titles...</comment>


Docker 0.5.2 Released - Security Update - <AUTHOR> https://groups.google.com/forum/?fromgroups#!topic/docker-club/7j9quGgOtZQ

<comment><USER2> is this chroot or root, because, in my experience, don't you NOT want apps running as root? I suppose with containers having sub-level access it would be a difficult decision to choose which was more secure, giving docker full access to everything, or it not having control. It's a shame that its such a difficult place to change it, or that you can't select it at install.

<comment><USER3> Docker itself requires root privileges to place processes in different namespaces, manipulate their control groups etc.

However, you can very easily drop the privileges of the application running inside each container, with "docker run -u <USER>". This is definitely recommended for several use cases, for example running untrusted code on shared hardware.</comment></comment>


The relatively unknown immigrant credit problem - <AUTHOR> https://medium.com/@Stilt/the-relatively-unknown-immigrant-credit-problem-1ceaf1aa7a65#.up4sdm4l8 <comment><USER2> It is well known if you are an immigrant. Daily conversation even. Also applying for jobs in a Country X, and 'Country's X work experience requirement'.</comment>


1960s Braun Products Hold the Secrets to Apple's Future (2008) - <AUTHOR> http://gizmodo.com/343641/1960s-braun-products-hold-the-secrets-to-apples-future <comment><USER2> This is the clearest example I've seen of the Dieter Rams influence:

[http://www.air-port.com/blog2/07/19/2007/iphone-calc- braun-e...](http://www.air-port.com/blog2/07/19/2007/iphone-calc-braun-et66/)

<comment><USER3> wow...he's new as a designer for me, but the influence is amazing. Is everything we think modern, just retro?

<comment><USER4> Yes, and it always has been. There's nothing new under the sun, etc... etc... At least not when it comes to things as easily explored as form.

</comment></comment></comment><comment> onreact-com Also make sure to read the story about the original iPhone, the Linux based open source deeda Pi (now RIP): [http://applephones.blogspot.com/2007/04/is- apple-iphone-copy...](http://applephones.blogspot.com/2007/04/is-apple-iphone- copy-of-deeda-pi.html)</comment>


Show HN: Screentop.gg – a platform for tabletop games - <AUTHOR> https://screentop.gg <comment><AUTHOR> Author here.

Screentop is a tabletop sandbox that supports cards, dice, boards, counters, and hidden player state. In the future I'd like to add scripting support so that more complex game mechanics and rules can be implemented.

Client stack: Typescript, React, Apollo, and a custom react-reconciler renderer that targets canvas

Server stack: Elixir, Phoenix, Absinthe

Happy to answer any questions and would love your feedback! Also I have two asks for HN:

1) Content - if you know any designers or publishers that would be interested in creating digital versions of their games please direct them my way!

2) Individual(s) to team up with - this is a solo side project right now but I'm interested in working with others, technical or non-technical, to see where this goes.

Discord: [https://discordapp.com/invite/wva8ebh](https://discordapp.com/invite/wva8ebh) Email: email@example.com

<comment><USER2> I really need a word for when I'm pissed someone beat me to an idea but delighted that it exists because of them.

I'd really like to know more on how to specify a game for your platform. A scripting language is a good idea, but I'd like to see it limited in scope. You don't want your board games to just become video games. I never could figure out where exactly those limits should be.

<comment><USER3> I'm planning a startup around a programming language just for board games.

<comment><USER4> Do you have a design document or wishlist or blog post for such a language?

I think you are on to something here.

<comment><USER3> I have several documents, but nothing coherent yet. On the bright side, I'm about to finish implementing hearts in my own language, so that's exciting.

</comment></comment></comment></comment></comment><comment><USER5> I would also like to see a screenshot/video gallery to get a feel for this platform, but I think you've probably gotten the message by now. Instead I'll throw a couple more questions your way:

Are you prepared for the incoming wave of DMCA takedown requests if this proves to be popular?

If someone spends the time to create content for this platform, how can they be sure that it won't shut down next week?

Can players create private content, such as for a D&D campaign?

If a player starts using content created by someone else, and it gets taken down for either copyright or because the author removed it, can that player continue to use that content?

In fact, can an author remove content at all?

Is there a system in place where the legitimate owner of an IP can create content for this platform and monetize it in some way? Tabletop Simulator may be a den of piracy[0] but there are some games for sale there as well: [https://www.tabletopsimulator.com/games](https://www.tabletopsimulator.com/games)

[0]: Ignoring that some number of users own the physical game and are merely playing with friends who are not in the area. Legally it might be piracy (IANAL), but ethically there's some discussion to be had there.

<comment><AUTHOR> Thanks for the questions!

 _Are you prepared for the incoming wave of DMCA takedown requests if this proves to be popular?_

I don't expect this to actually happen but if it does, I would comply.

 _If someone spends the time to create content for this platform, how can they be sure that it won 't shut down next week?_

This currently runs on just one server and even with the incoming HN traffic, the server utilization is pretty low. This won't cost much to keep running.

 _Can players create private content, such as for a D &D campaign?_

Authors can create private games. This means only they are allowed to create a room. Rooms themselves are only semi private (the 4 digit code is guessable) and will expire after a period of inactivity. Not sure if the current setup would work for private D&D sessions but would love to support that use case.

 _If a player starts using content created by someone else, and it gets taken down for either copyright or because the author removed it, can that player continue to use that content?_

If the game itself is deleted, all rooms of it are also deleted.

 _In fact, can an author remove content at all?_

Yes, authors can delete their own games.

 _Is there a system in place where the legitimate owner of an IP can create content for this platform and monetize it in some way? Tabletop Simulator may be a den of piracy[0] but there are some games for sale there as well:[https://www.tabletopsimulator.com/games*](https://www.tabletopsimulator.com/games*)

At this stage I expect most games to be from hobbyists who want others to playtest their games or kickstarter creators who want to offer a digital game copy to backers. Would be open to creating opportunities for monetization if this gets more traction.

</comment></comment><comment><USER6> You didn't ask but, consider making a "highlight reel" or screenshots and showcasing them on your page, otherwise I don't really understand what this is or what it looks like. Is it like Tabletop Simulator?

<comment><AUTHOR> Yes this is like a 2D web based Tabletop Simulator that you can play with friends without having everyone purchase and install a copy of TTS.

</comment></comment><comment><USER7> I couldn't tell if this was different to [https://boardgamearena.com/](https://boardgamearena.com/). But that site does license the games it uses.

<comment><AUTHOR> The idea is similar but I wanted to build something a bit more modern than boardgamearea.

Focusing on adding more features and growing the site at the moment. Hope to license official games eventually.

<comment><USER8> I tried to build a game once with their stack, but it's just too old school and custom for my taste. I really appreciate your platform and hope it will attract some people.

The licensing thing is a big thing though. But I think the publishers don't mind as long as the games are well presented and don't have a digital version yet.

</comment></comment></comment><comment><USER9> I'll have to dig deeper into the creation mechanic but this would fill a need that I've got to recreate table-top games but for distributed groups of friends that haven't been able to devote the same amount of time as before to gaming.

We've been trying to work through Risk: Legacy, Pandemic, or other, longer-run games (I would think Axis & Allies and other longer-run strategy games as good candidates for this thing) in-person for years and it's hard to find the time.

It'd be awesome to be able to continue things on a platform like this, but would obviously necessitate creating a version of the game on the platform first. I like the concept!

<comment><AUTHOR> Yeah I built this because it's painful to coordinate playing tabletop games online - with Tabletop Simulator or individual apps it's time consuming to get everyone to purchase/install/setup a copy just to get started.

Hopefully one day publishers will put some of those games up on Screentop!

</comment></comment><comment><USER10> I wish there were an open source, well defined DSL for how to describe board games. I suppose there is for things like chess, and custom games... Anyone know of such a language which aims to describe _any_ such game?

</comment><comment><USER11> I found this platform really easy to play around with when building a game. I put together a version of Hanabi, and the system was flexible enough to let me make player hands that are visible to everyone _except_ the player. Reconstructing this relatively simple game from scratch only took ~3h, and that is everything from finding royalty-free assets to writing the manual.

</comment><comment><USER12> Here is also a tutorial app for board games - Dizie tries to make board games more approachable by giving them similar tutorials as in video games.

[https://dized.com/](https://dized.com/)

</comment><comment><USER12> Another well know platform: [https://yucata.de/en](https://yucata.de/en)

Also you can shop digital version of board games like Terraforming Mars in Steam.

</comment><comment><USER13> At least one of these games is using copyright of uncertain provenance... have you really purchased the rights to "Code Geass"?

<comment><AUTHOR> I don't own the rights to any of the content. The Code Geass game is a fan created board game by my friend from: [https://brotherminggames.com](https://brotherminggames.com)

</comment></comment><comment><USER3> This is really cool, and I like this space. I'm currently looking at starting a company around implementing board games online.

</comment><comment><USER14> Question: What's the difference between this and Tabletop Simulator?

<comment><AUTHOR> This is web based and free to use. No need for everyone to purchase and install a copy - just send a link and start playing.

Another major difference is that this is 2D, which should work for most tabletop games but not all.</comment></comment>


How social networks can be used to bias votes - <AUTHOR> https://www.nature.com/articles/d41586-019-02616-2 <comment><USER3> If only this level of scrutiny was applied to traditional mass media over the last century. All of these techniques are just updated versions of handing the anchorman a biased script.

<comment><USER4> My thoughts exactly. You know what else can bias voters? Expert testimony, even in fields outside their expertise. Religious leaders. Teachers. Flyers. Advertisements. In fact, just about any form of communication by anybody. Without comparing the size of the bias to other forms of influence, this is a nearly contentless statement.

<comment><USER5> Know how you get a human to believe something? Make them work for the information. The more someone works for information the more likely they are to believe it is valid. I think this is why you see modern educated people with such deference to the "experts". They know the hard work and the costs involved in becoming a academic and as such they value that experience with an almost religious fervor.

<comment><USER6> Wow. Are you really equating expertise with religious fervor?

The anti-expertise movement is what I would describe as religious fervor, not the other way round.

<comment><USER7> here, have some economics experts:

[https://www.americanprogress.org/issues/economy/view/](https://www.americanprogress.org/issues/economy/view/)

[https://www.cato.org/research/growth- development](https://www.cato.org/research/growth-development)

how big is the middle part of the venn diagram here?

being skeptical of domain-experts is only 'religious fervor' if the field is cut-and-dried. many aren't

<comment><USER6> Let me guess. You cherry-picked some cases where experts were wrong. Am I right? Here, read this: [https://www.amazon.com/Death-Expertise-Campaign- Established-...](https://www.amazon.com/Death-Expertise-Campaign-Established- Knowledge-dp-0190865970/dp/0190865970/ref=mt_paperback)

I don't have time for this.

<comment><USER7> cherry picked? yes and no.

These are both hugely influential policy think tanks which hire experts. They both have hugely differing views on the economy.

Or is providing evidence to support a claim now 'cherry picking' to be dismissed? And if so, aren't you proving my point?

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> > We initially assume the payoff to the losing team is positive, b > 0, which > reflects a 'compromise worldview': it is preferable to reach some decision > than to end in deadlock. This payoff scheme captures the common practical > value of broad consensus in collective decisions, even as individuals pursue > partisan preferences12,13. There is ample evidence that large majorities of > Americans, for example, adopt a compromise worldview in their attitudes > towards political decisions14. However, others adopt a 'zero-sum worldview' > in which they prefer deadlock to their party losing, that is, b < 0\. The > behaviour of people with a zero-sum worldview is simple: they act as > zealots, meaning that they always vote for their preferred party regardless > of the poll that they see. We begin by focusing on players with a compromise > worldview, and then we study groups with a mixture of compromise and zero- > sum players, including zero-sum bots.

In the context of voters in our modern politics, does the "compromise" viewpoint even exist?

<comment><USER9> They do, but they're decreasing in number. They're also kind of boring so they don't get coverage the way highly vocal 'zealots' do.

</comment></comment><comment><USER10> It's an interesting study in terms of how they conducted it, but isn't it obvious? If political ads didn't swing votes, no one would run them.

<comment><USER8> This study would also indicate that political polling has the ability to influence elections?

<comment><USER11> Of course, anything that brings focus potentially changes the final metric.

<comment><USER8> I would agree, but I think most political reporters and pollsters view themselves as impartial observers rather than active participants...

<comment><USER11> What they view themselves as does not change the fact that they skew things in one way or another. The only observer that is neutral is the one that is not seen and not heard of.

</comment></comment></comment></comment></comment><comment><USER9> >All data necessary to reproduce the results are available at [https://github.com/jplotkin/InformationGerrymandering](https://github.com/jplotkin/InformationGerrymandering)

>For example, when just a few yellow-party zealots were deployed strategically among a larger number of undecided players in the purple party, these bots were able to sway the majority opinion towards the yellow party. This was true even when the parties had exactly the same number of members, and when each player had the same amount of influence.

I'd like to know a little more about the experiment, specifically how much information was available to individual voters. Did they know the votes of specific other voters? Of their party specifically? Or just the overall polling situation? Were they allowed to communicate, like through a chat box or something?

Without knowing these details I'm finding it a little hard to understand how these few 'planted' zealots were able to influence the result that strongly.

</comment><comment><USER12> There is an ongoing controversy in Norway right now, where a TV show during one month tried to manipulate the results of "school elections" (basically mocks of the real elections, where one votes for the real parties - <USER2> results are often used as an indicator for the real election results some weeks later).

The goal of the show was to show how election results can be manipulated, which created some outrage: some of the students are old enough to vote in the real elections, so they might have manipulated real results.

Norwegian news source: [https://www.nrk.no/norge/folkeopplysningen_-slik- forsokte-de...](https://www.nrk.no/norge/folkeopplysningen_-slik-forsokte-de- a-manipulere-skolevalget-1.14688590)

</comment><comment><USER13> Yes, it all comes down to money. Money = the means to influence = power. It is interesting to note that citizens have been self-inflicting political bubbles upon themselves this for hundreds of years by surrounding themselves with similar-minded people (i.e., people who aren't going to argue with their politics).

</comment><comment><USER14> Votes are biased by worldview and worldview is tied to group identification.

</comment><comment><USER15> i think this might cross over to something like: Election campaigns can be used to bias votes.

That being said social media has a great potential to be used for campaign violations.

<comment><USER16> Elections considered harmful.

</comment></comment><comment><USER17> It's impossible to tell from TFA or the abstract what precisely is meant by "information gerrymandering". One imagines some sort of graph theory or "six degrees of Bacon" argument. "This node should only influence some number of other nodes, but in fact if you count second- and third-order connections it influences some _higher_ number of other nodes!" Presumably they've picked out some particular criteria for differentiating justifiable influence from unjustifiable influence. The criteria are probably fine; since we don't know them the point is moot.

Even if one accepts all of that, one still can't follow TFA to its conclusion of "...voters really can be manipulated in the digital age. Legislators and regulators must take note." We've been working on this "giant graph of nodes influencing each other" problem for decades now. That's just SEO. That's the basic thing that Google does. Any network has these properties, even the "snail" networks of pamphlets and broadsheets that kept our founding fathers informed.

It was completely appropriate for late-18C pamphleteers to publish and distribute according to the dictates of their own best judgment. It's completely appropriate for today's social network operators to do the same. It's completely appropriate for _users_ to do just exactly as they please, whatever the TOS say. What is not appropriate is for "legislators and regulators" to imagine they have a damned thing to say about this process.

1A is not ambiguous on this question. The government don't get to forbid the public from hearing Trump describing to Billy Bush his techniques of seduction. They don't get to forbid the public from hearing Clinton tell bankers they should be able to regulate themselves. [0] It's a conversation because they _don 't_ get to decide everything that gets said. For three years now, a segment of the "intelligentsia" have wallowed in a mode of magical thinking about networks and bots that has left them completely uninterested in real things. If this research can help them break out of this rut, one welcomes it. However, that seems unlikely...

[0] [https://www.thedailybeast.com/leaked-podesta-emails-show- ber...](https://www.thedailybeast.com/leaked-podesta-emails-show-bernie-was- right)

</comment><comment><USER18> Advertising works. News at 11...

Here, all the polls are skewes towards whoever the polsters support. And have been for many years..

At least they can't blame Trumps victory on this as the polls showed Hillary in the lead until after the election.

</comment><comment><USER19> Why Social Media (Facebook, Twitter, etc.) isn't willing to work with local media and communities? Maybe off topic but genuine question.

We have traditional media that failed to transition to Web. Same stuff in different package, instead of collaborative approach.

And we have social media that fails to deliver the promise of communities. Fails in the transition from virtual to real world.

From my experience, success in Facebook is closed groups. They are the ones with most activity and the ones that achieves real social events.

Both players (traditional media and social media) have what the other one lacks. I'm really scratching my head on why they aren't working together.

<comment><USER20> Work with how? What would that entails? What is stopping them from getting their own right now?

<comment><USER19> Contributing with collaborative tools, billing, linking communities, managing users, comments, events, etc.

Local media adds content, moderation and insite presence.

A lot of communities can benefit from this.

<comment><USER19> Events like cleaning trash, planting trees, sport events, improving schools.</comment></comment></comment></comment>


Radioactive Hulk of Carrier USS Independence Found Off San Francisco (2015) - <AUTHOR> https://telstarlogistics.typepad.com/telstarlogistics/2015/04/radioactive-wreck-of-aircraft-carrier-uss-independence-located-off-san-francisco-coast.html <comment><USER2> I used to live on Treasure Island. Many years prior, some of these radioactive ships had been docked there. Recently I googled my old Treasure Island address & discovered the living room of the house in which I had been living had been flagged for higher than normal levels of radiation.

I'm still waiting for my superpowers to manifest.

<comment><USER3> Maybe you already have them - the proportional strength and agility of... a living room.

<comment><USER2> I am exceptionally good at being a couch potato.

</comment></comment></comment><comment><USER4> According to Wikipedia the wreck was found in 2009 and this article was about when it was first imaged. Although it isn't clear to me why exactly the Independence needed to be found in the first place since it was intentionally scuttled. Was the location of the ship classified? Or was the record keeping from the 1940s so bad they the Navy didn't record the location of a scuttled ship or listed it as something vague like "off the Farallon Islands"?

<comment><USER5> Ships don't sink in a straight line, knowing where it sank does not give an accurate position for where it rests.

</comment></comment><comment><USER6> Humans are pretty stupid.

So, we exploded a nuclear bomb, just to see what would happen to a bunch of ships (I guess since, it was quite the mystery), then towed one the ships back to a major metropolitan city, sandblasted it to try to decontaminate it (which didn't work), stored the now contaminated sand onto the ship, then towed the ship back out to see, just to use it as target practice until it sank, thus sinking a radioactive ship and it's contaminated sand.

Living a few miles from Rocky Flats, where are sorts of crazy stuff _almost_ happened, it's no wonder we survived the 20th century.

<comment><USER7> Nope, humans were trying to figure out how to manage the consequences of nuclear weapons, which were judged to be a necessary strategy to end World War II to attempt to defeat fascism and save democracy.

Not "just to see what would happen" but to know how vulnerable our Navy would be to the very real threat of atomic attack from the Russians, and thus to responsibly inform strategy and tactics should that nightmare ever come to pass.

Testing decontamination also necessary to inform e.g. necessary fleet sizes in case of nuclear war -- could ships ever be reused or not? An existing shipyard is the logical place to do that, and unfortunately widespread environmental consciousness just didn't happen until the landmark publication of _Silent Spring_ in 1962, more than a decade later.

Storing the contaminated sand on the ship and sinking it was a pretty reasonable thing to do at the time, not "just to use it as target practice" but to kill two birds with one stone.

Nobody's perfect, but you seem to _really_ be judging people from several generations ago according to current societal standards, and completely misportraying their motives.

<comment><USER8> There has also been an awful lot of humans doing stuff "just to see what would happen" across history. From ice-picks for faster lobotomies, through widely available quack remedies to project A119 - "I wonder what would happen if we nuked the moon".

Let's not presume high moral motives in everything our forefathers did, nor to all of our current generation either.

[https://en.wikipedia.org/wiki/Project_A119](https://en.wikipedia.org/wiki/Project_A119)

<comment><USER9> As 'paulddraper notes, doing things "to see what would happen" is a recognized research strategy that created a lot of science and technologies we enjoy today. This thinking, a childlike curiosity of sorts, is something you seek in good researchers.

As for A119, the issues with the project were entirely political. The idea of bombing the moon is a _good_ one; it's a legitimate way of collecting scientific data in lieu of being able to deploy a drilling team on the surface, and something that's been done with non-nuclear impactors. See [0], [1], [2], and probably a bunch of other cases I don't remember today.

It's easy to say that "nukes bad humans stupid", but there are aspects to this idea - and a bunch of other so-called "peaceful uses for nuclear weapons" \- that illustrated human smarts, not stupidity.

\--

[0] - [https://en.wikipedia.org/wiki/LCROSS](https://en.wikipedia.org/wiki/LCROSS)

[1] - [https://en.wikipedia.org/wiki/SELENE](https://en.wikipedia.org/wiki/SELENE)

[2] - [https://en.wikipedia.org/wiki/Moon_Impact_Probe](https://en.wikipedia.org/wiki/Moon_Impact_Probe)

<comment><USER10> Does that type of _aggressive_ research just assume that the systems being impacted are durable enough to withstand the destruction (2tons of tnt, in one of your links), or has that really been studied? I don't know the context of this, but I can't help but think of how many of the ocean's submersibles damage the ecosystem.

<comment><USER9> The Moon has some very convenient features for such research - namely, it's a big, dead rock. Hard to think up a bad thing that would happen from hitting it with a satellite, or even a bomb. We already know it receives a regular beating from the solar system, in form of meteor impacts.

<comment><USER10> I don't see one of the impact probes having as much force as a meteor, but I could see it being enough to throw off a fragile system. Yeah, its not like bombing ganymede or mars' ice-caps. I'm just curious if they are going to accidentally collapse some cave network, or give it a temporary atmosphere of dust.

Edit: thanks for the response btw

<comment><USER9> Who knows? There's always uncertainity, and a non-zero probability that the bombing site happened to have a geologically interesting cave system underneath, or a hidden cache of alien artifacts. The probability of such a thing, while non-zero, is _really_ low, and we also don't observe fragile systems on the Moon. It's simply a dead rock.

As for bombing Mars ice caps, that was a nice idea too, IMO (even though AFAIK it turned out to not be useful). The goal here was to densify the atmosphere with water vapor and CO₂. Assuming you consider terraforming Mars a good idea, doing it in reasonable timescales will require some preparations involving direct applications of lots of energy - whether it's melting the ice caps, or redirecting comets to hit the planet. At some point we might be able to accelerate the process by deploying bacteria and plants (read: self- replicating chemical processing nanobots), but the conditions are not right for this yet - hence the ideas like using nukes or comets to bootstrap those conditions.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Hmm. As a way of disposing of nuclear waste, how bad is sinking a ship at sea?

<comment><USER5> Nuclear waste is like asbestos, it's fine if you leave it alone. If the depth is sufficient that the water is fairly anoxic and the area is geologically stable it's pretty safe.

However if the area is unstable or the ship can rust you can easily end up with a huge distributor of radioactive contamination. The effects depend on the substance and type of radiation.

Dumping of radioactive waste in the ocean has been banned since 1993. [https://en.m.wikipedia.org/wiki/Ocean_disposal_of_radioactiv...](https://en.m.wikipedia.org/wiki/Ocean_disposal_of_radioactive_waste)

</comment></comment><comment><USER12> Remember Blueseed, the "seasteading startup"? Their planned location was west of Half Moon Bay, roughly where this is.

<comment><USER13> Cool, a piece of history lurking under your workplace. Far enough down that there's no problem with floating above that particular spot.

</comment></comment><comment><USER14> So at least we have a good idea where Godzilla will likely rise from the ocean and decimate San Francisco!</comment>


Android Q: The Bad and the Ugly - <AUTHOR> https://wearespindle.com/androidq-the-bad-and-the-ugly/ <comment><USER2> Removing the option to turn off Wi-Fi seems like just another way to constantly spy on people.</comment>


Ask HN: Review my web-app for creating Android apps - <AUTHOR>

I'm looking for some help in reviewing my startup and would appreciate any feedback from those of you with Android phones. http://redbirdapps.com
Our first application grew from the idea that if you have a smart phone, you shouldn't have a dumb business card.
Try it for free.  You can have a customized Android application on your phone in a few minutes, no programming required.
Thanks, hackers. <comment><AUTHOR> Clickable link: <http://redbirdapps.com></comment>


Mozilla Firefox: Rolling Out HTTPS Google search - <AUTHOR> https://blog.mozilla.org/privacy/2012/05/07/rolling-out-https-google-search/

<comment><USER2> No REFERER header anymore, bye bye search keywords data, now only in Google Webmaster Tools...

Edit -- From the article: "Additionally, using HTTPS helps providers like Google remove information from the referrer string."

<comment><USER3> Maybe I'm missing something, but doesn't Google redirect through an interstitial page that's always over HTTP, so you do get a referrer that says the traffic came from them?

<comment><USER4> Yes, they do. But they are cutting out the keyword intentionally. There is no technical need to hide the keyword from website owners.

Remarkable coincidence 1: There still is a keyword in the referrer for AdWords customers.

Remarkable coincidence 2: There is a new _charged_ product called "Google Analytics Premium" that promises "more data, features and dedicated support". Shame upon him who thinks evil upon it.

<comment><USER3> Ah, I thought they made that change for everyone a few months ago. Didn't realize it was only for https clicks.

</comment></comment></comment></comment><comment><AUTHOR> A nice side effect of using HTTPS is that Firefox 13+ will be able to use SPDY for Google searches.

</comment><comment><USER5> So the question I have is how big of an impact is this going to have to online marketers? (Especially if Chrome/IE follow suit)

<comment><AUTHOR> They are unhappy because websites won't see HTTP Referer headers and will have to rely on Google's webmaster services to get referral stats:

<https://bugzilla.mozilla.org/show_bug.cgi?id=633773#c43>

<comment><USER6> Will they not see the referer header if their own sites (as indexed by Google) are https?

If they don't switch, puts some sort of upper bound on how much they value the referer data.

(User agents aren't supposed to send referer http->https but scratch above if practice is for them not to send https->https across domains as well.)

<comment><USER7> I don't know about the spec, but in practice, you never get the referer data even when traffic flows from HTTPS to HTTPS.

<comment><USER8> This is actually up to the client. In Firefox about:config, network.http.sendSecureXSiteReferrer is set to true by default, so it does send it, but you can turn it off.

<comment><USER9> Google decides what the referrer is since it redirects first to a url it controls (downgrading to http if necessary).

</comment></comment></comment></comment></comment></comment><comment><USER10> If you want this today, you can go to <http://opensearch.webos- internals.org/gtp.html> using either Chrome or Firefox and make that the default search engine.

As a side benefit, the config also turns SafeSearch and personalization off.

</comment><comment><USER4> This has nothing to do with security. Removing the referrer information puts Google in the position to use (sell?) this information exclusively.

While Google is collecting more and more data about users Mozilla calls it an "improvement" to keep away referrer information from website owners. This is ridiculous.

So Mozilla get's its money from Google and in return they do what Google tells them.

<comment><USER11> I see your point, but Mozilla is in an impossible position.

Encrypting people's search is a good idea. Mozilla should do this, and as others in this thread have pointed out, they could be chastised for not having done it.

The fault is Google's who tamper with the referrer data intentionally to obscure the keyword data. They do this for the users 'privacy', but then you can still get the data if you're a paying Adwords customer. I have trouble reconciling 'privacy' and 'you can buy it'.

<comment><USER12> > I have trouble reconciling 'privacy' and 'you can buy it'.

I'm not sure, but I would assume that the idea is that data from Adwords, etc. is more anonymised. You can't build a system that tracks peoples searches across visits (using cookies).

</comment></comment></comment><comment><USER13> You can do this with chrome by adding a search engine (and making it default):

[https://www.google.com/search?%7Bgoogle:RLZ%7D%7Bgoogle:acce...](https://www.google.com/search?%7Bgoogle:RLZ%7D%7Bgoogle:acceptedSuggestion%7D%7Bgoogle:originalQueryForSuggestion%7Dsourceid=chrome&ie=%7BinputEncoding%7D&q=%s&num=100)

</comment><comment><USER14> Why was this not done when Google started supporting https?

It seems almost negligent to have not done it sooner.

<comment><USER15> Because you have to make sure that Google's https infrastructure can handle the load of 400 million Firefox users?

<comment><USER14> It could already provably handle the load of millions of Chrome users (cf. SPDY: https).

And it's not like it's totally separate from their normal http infrastructure, which could also already provably handle the load of billions of searches.</comment></comment></comment>


Visual Mental Imagery Training - <AUTHOR> https://www.lesswrong.com/posts/8ciFqEjkekqzaTqT6/visual-mental-imagery-training <comment><USER2> I wonder if the people who cannot create any mental imagery can remember their dreams, or if they have any?</comment>


The Islands of Refreshment and the history of micronations - <AUTHOR> http://theappendix.net/issues/2014/7/the-king-of-the-islands-of-refreshment

<comment><USER2> Went down this rabbit hole (specifically Tristan da Cunha) and found a great little documentary about how the island led to genetic research on asthma.

[https://www.youtube.com/watch?v=kKRvtk- GI0g](https://www.youtube.com/watch?v=kKRvtk-GI0g) (part 1 of 2)

Very interesting.

<comment><USER3> Great find, tjbarbour.</comment></comment>


Ford Planning to Drop Microsoft for BlackBerry in Its Car Technology System - <AUTHOR> http://www.bloomberg.com/news/2014-02-22/ford-said-to-swap-blackberry-s-qnx-for-microsoft-in-sync-system.html

<comment><USER2> I had the pleasure of working in the same building as a company that builds specialized navigation systems for high end cars. They almost always used the microkernel that powers BlackBerry's latest operating systems, QNX, and one of their engineers described it to me as probably the only good implementation of a microkernel to date. He felt the microkernel architecture was very important to the needs of an automobile.

From a hardware standpoint BlackBerry is severely behind the times but their underlying operating system appears to be quite impressive. It's just yet another reminder that the best technology isn't enough to win in this mobile game.

</comment><comment><USER3> Someone at Microsoft is getting fired, I guarantee it. Also, wasn't Ford having severe issues with its Microsoft implementation in cars?</comment>


Google DeepMind StarCraft Demonstration (Scheduled 1pm EST) - <AUTHOR> https://www.youtube.com/watch?v=cUTMhmVh1qs <comment><USER2> 32 points and not a single comment?

<comment><USER3> Its a 2+ hour video most people forgot how they got to the page by the time the video ended</comment></comment>


Ask HN: Why doesn't Microsoft build its own Android "fork"? - <AUTHOR> Why doesn't Microsoft just build a custom version of Android?  They get instant access to the vast android apps and they can integrate it deeper with the Windows ecosystem and also innovate in their own "fork" of android.
They could eventually surpass other versions of Android and become the goto android implementation.
I never understood why they didn't go that route - after all Android is open source.  What am I missing? <comment><USER2> Because the OS doesn't matter nearly as much as the apps running on that OS, and Google only allows licensed OEMs to preload Google apps onto their devices (for a fee). Also, how strange would it be for Microsoft to pay Google billions of dollars for them to use their App Store? That would not only be them conceding that they couldn't win on mobile, but that would also reduce them to yet another Android Phone manufacturer that has to spend tens of millions of dollars on low margin devices that won't sell.

Also, Windows Phone was an amazing OS in its own right but failed largely because they couldn't solve apps. It is impossibly hard to compete against Apple or Samsung in this space, especially when the OEMs didn't want to play ball.

(Samsung, HTC and LG made a few Windows phones that they didn't invest a lot of time on. They were most likely an expensive relationship maintenance activity to retain a partnership with MS. I believe this happened because MS imposed really strict guidelines for their OS experience right out the gate that didn't really benefit anyone but Microsoft (OEMs make money on preinstalled apps, for example; that wasn't allowed.) It was much easier for them to just pump more money on making great Android devices than trying to make Windows Phone great again)

<comment><USER3> Is Android inextricably coupled with the Google App store? For thought experiment's sake: could Microsoft fork Android and then host their own app store that supports the same apps that a dev would submit to Google's App store, thus making it possible for devs to get their android apps listed in both stores merely by submitting the app twice?

(Serious question. I know nothing about app stores and/or mobile operating systems)

<comment><USER4> Yes it's possible. It's exactly what Amazon is (was?) trying to pull off.

</comment></comment></comment><comment><USER5> I've read (rumor alert) that this is kind of what they originally had in mind with the WSL (Windows Subsystem for Linux).

This would allow windows phones to run android apps from within Windows (though several obstacles remains, since the play store and other google services are proprietary)

Rumored this worked so well that Microsoft canned it in favor of UWP, because if windows could run android apps there would basically not be any need for UWP. As a result Microsoft changed focus and gave us WSL on the desktop instead.

The above could very well / likely be made up but I like that explanation.

<comment><USER6> There are arguments that Android compatibility killed Blackberry, and those were brought up at the time.

There is also the thing that Android apps without Google Play Services often feel like zombies of themselves. The easiest way to see that is comparing Amazon's FireOS to Android: which apps aren't even available, which ones are, but fail in strange ways, etc.

</comment></comment><comment><USER7> Google Play Store isn't open, control of that is key.

It doesn't matter if the platform it's running on is "Microsoft Android" or something else - Google still controls the store

<comment><USER8> Yes, but apps don't HAVE to use the Play store.

Amazon, for example, has all its apps in a separate store because it doesn't like to play Google's games (the Amazon shopping app is not in the Play store because they don't want Google getting a slice of their profits).

A similar strategy could theoretically work for MS.

<comment> yoz-y Amazon does have its own store because it wants the app profits for themselves, however they do have shopping, kindle, prime and so on apps in the Play Store[1].

I suppose that Play Store has the same rules as the Apple App store. That is, you can have your own checkout and payment system as long as you sell things that are not for content that you consume in the application. This is why Uber and Amazon and other shopping apps do not have to share the profits, but for example Audible does not sell books through their iOS app and microsoft actually pays 30% to apple (now probably 15) for any Office subscription people get when getting it from the app.

[1]: [https://play.google.com/store/apps/details?id=com.amazon.mSh...](https://play.google.com/store/apps/details?id=com.amazon.mShop.android.shopping)

<comment><USER9> You can't buy books via Kindle app on iPhone. You have to manually open Safari, go to amazon site and buy a book there. I believe it's exactly because Amazon does not want to give Apple the cut of the book profit.

<comment> yoz-y Yes, because kindle books are considered content for the application. Its the same for Audible. But the original point was that there is no Amazon Store app on the Play store (which there is) and the other applications are there but crippled.

It is perfectly reasonable for Amazon to not want to give a 30% cut on books to Apple. I wish there would be some exception for this kind of content in the stores.

</comment></comment></comment></comment></comment><comment><USER10> They did and canned it. It was called Astoria and from what I heard it worked really well. The media said the opposite but "Too well" was what one MS guy said to me. They had a ton of people working on the project.

<comment> slipstream- I also heard "too well", as in, developers just wanted to use their android apps instead of making windows apps.

It wasn't totally canned though, it became the Windows Subsystem for Linux.

<comment><USER11> If that is, indeed, the case, then Microsoft makes a bad image of itself by not giving this to their existing users, who desire that a lot.

</comment></comment></comment><comment><USER12> The issue is that while Android itself is "technically" open source, it's a walled garden platform. Microsoft's own apps won't work without proprietary Google software. (Both Skype and Outlook won't launch without Google Location Services enabled, I haven't tried the launcher or the new Edge.)

They can't include Google's proprietary software in their fork, since part of the agreement for distributing that software includes a clause where you agree not to fork Android.

Even if Microsoft created their own location service for Android, changed their own apps, and made their own app store, they'd have to convince other app developers to support it, since Google has spent the last few years convincing everyone to build their apps on proprietary libraries and services. Essentially, they end up in the same boat as Windows Phone: Having a platform without apps, and not a strong incentive for developers to modify their apps to support it.

</comment><comment><USER13> Microsoft would have to provide a compelling reason to use their own app store and other services that would replace the Google ones.

That's not to say it's impossible. Microsoft has been doing a lot of mobile development lately, and if you were trying to build a case for losing the Google Play store Office, Skype, and other business apps are a good place to start.

The pessimistic part of me thinks that BYOD is so prevalent these day that a pure "business phone" locked into the Microsoft ecosystem just wouldn't work. Maybe five years ago it would have appealed to enterprises wanting to roll out a fleet of employee devices but now a phone needs to be as appealing to the consumer as the business.

</comment><comment><USER14> They tried by investing heavily in Cyanogen mod but that didn't go anywhere.

<comment><USER15> Well their investment basically killed Cyanogen inc and Cyanogen mod.

Good that it's open source, so they forked it, and is now known as LineageOS.

</comment></comment><comment><USER16> Alternatively why not build a transpiler for android apps? Java -> C# would be quite doable, and then just style the APIs and UI layouts similiar but too similar to end up in court. Then what dev wouldn't have a go if the process could be 90% automated? Instead they stayed in their own bubble of custom tech that didn't bring anything particularly revolutionary to the table, and failed to attract attention as a result.

<comment><USER17> Android apps aren't strictly java, and they don't depend on open frameworks. Rather, they are built in a combination of languages and depend largely on googles proprietary frameworks.

<comment><USER16> Sure but the other languages such as C++ could be directly supported. The APIS could be copied and renamed. Not hard to duplicate the functionality without seeing the internals, just time consuming.

</comment></comment></comment><comment><USER18> And why didn't RIM!? I'd love to have a phone with a hard keyboard like the old crackberry. That device was awesome.

<comment><USER2> They run full Android with RIM security extensions. See the DTEK line. They've outsourced device manufacturing to TCL but are still in the mobile business.

</comment></comment><comment><USER2> If I recall correctly, Microsoft did intend on having Windows Phone 10 run Android applications (they leaked it early even) but then pulled it back last minute.

</comment><comment><USER19> They don't get instant access, though.

As a developer I already have to support both Google's flavor of Android and Amazon's flavor of Android, with their own separate implementations of IAP, push notifications, etc.

I'd have to support MSFT's implementations too and you'd have to convince me (and the people who pay me) that it's worth bothering to do.

</comment><comment><USER20> Are the days of Microsoft attacking all comers over? Or even trying to carve out their own niche in mobile? They seem much more interested in _playing nice_ with the new platforms -- free Office on Android & iOS, for example, and good integration between phones and Windows on the desktop.

Of course, they've also sold Office on Mac _forever_.

</comment><comment><USER21> Related / triggered by: Microsoft gives up on Windows 10 Mobile | [https://news.ycombinator.com/item?id=15432720](https://news.ycombinator.com/item?id=15432720) (4 hours earlier, 361+ comments)

In case either one gets separated from the other as the front page drifts by.

</comment><comment><USER22> MS is experimenting with deals to have apps pre-installed on Android phones. e.g. An HTC/Lenovo phone with MS Launcher, Edge, OneDrive, Office, Cortana, Bing will be enough of a Microsoft phone. They don't need to go and fork the OS.

</comment><comment><USER23> MS already makes royalty on every significant Android device made

</comment><comment><USER24> They'd fuck that up too.

</comment><comment><USER25> > What am I missing?

You answered your own question here:

> after all Android is open source.

<comment><USER26> That didn't stop Amazon though it's arguable that they have more to offer a consumer from a walled garden approach (Prime Video, Music, Kindle books...).</comment></comment>


Ask HN: Is SQL the new COBOL? - <AUTHOR>

<comment><USER2> You mean, with NoSQL being the new hotness?

No, SQL is not the new COBOL. COBOL was superceded by languages that do the same things better, and then some. COBOL has a huge amount of inertia and is never going away, but not because it's anyone's ideal language for any task.

SQL database systems are in a very different situation. They do many things better than any alternative. NoSQL systems may scale better, for some definition of "scale", and for some parts of some applications, but SQL databases do lots of useful things that are hard to obtain using any other technology.

NoSQL is like a tax shelter. It's a solution to a problem you wish you had, but do not actually have. You aren't rich enough to need a tax shelter. And unless the minimum viable product you created over the weekend is being used by 100 million users, all day every day, you don't need NoSQL.

<comment><USER3> I respectfully disagree: there's plenty of room for NoSQL, even if your users are next to none.

For example, at times, it is just more convenient to store data in a schema- less database (while still being able to query said data relatively easily). One such thing are my logs: I parse and tag them before storing them, and the relational model does not lend itself to that kind of storage.

Document stores, however, work beautifully. I can store entries like this:

{ program: { name: 'sshd', pid: 1234 }, date: { year: 2010, month: 1, day: 5, ... }, useracct: { mehtod: 'pubkey', source: '::1', port: 45678, service: 'ssh2', username: 'user', type: 'login' }, message: 'Accepted publickey for user from ::1 port 45678 ssh2' }

Obviously, some of my log messages won't have half of the properties from the example above, or have other kinds of fields. Storing that in SQL, while maintaining easy queryabilty would be a huge pain in the backside, to say the least.

With a document store, it's a piece of cake.

<comment><USER2> By "NoSQL" I'm referring distributed key/value stores. I know that's not quite right, there are other kinds.

Also, I'm not saying that NoSQL doesn't have any purpose. It definitely does, and long before NoSQL was hot, there were standalone b-trees like BerkeleyDB, Faircom and many others. It may have been unfair for me to drag NoSQL into this, but the question sure sounded like it was suggesting that the sun was setting on SQL due to something, and NoSQL was a reasonable guess for what that something was.

<comment><USER3> Yep, that was my interpretation of the question too. But I wouldn't have put down the document or key/value stores so harshly, since they do have their uses, even outside of high-scalability scenarios.

As for BerkeleyDB and the rest: I'm not too familiar with them, but my impression before was that querying them wasn't as elegant as it is with some of the recent key/value or document stores.

Probably that's the reason these new things receive such a hype: they do have something for them, even if the hype makes those bigger than they really are.

</comment></comment></comment></comment><comment><USER4> Only if Tutorial D became _much_ more viable while I wasn't looking.

</comment><comment><USER5> No, Java is the new COBOL.

<comment><USER6> Because clearly, there are no exciting new projects being built in java. Except Hadoop. And Cassandra. And Neo4j.</comment></comment>


Cylinders in Spheres - <AUTHOR> http://www.datagenetics.com/blog/july22014/index.html <comment><USER3> One of those cases where using integrals rather than geometry is much simpler.

        \pi \int_-3^3 (R^2 - x^2) dx = \pi (6 R^2 - 18)

is the volume of the rotational solid without removing the cylinder. While the volume of the cylinder is given by:

        \pi \int_-3^3 (R^2 - 3^2) dx = \pi 6 (R^2 - 9)

As you can see the difference between the two volumes is 36 \pi.

You can actually show the solution does not depend on \pi without evaluating the integral, and then compute the 'cheat' case, which would not be a cheat once you've made this observation.

<comment><USER4> Nit: the solution doesn't depend on "R" (as you note, this is clear from looking at the structure of the two integrals); it _does_ depend on \pi.

<comment><USER3> typo! And I'm out of the edit window.

</comment></comment></comment><comment><USER5> Nice, I especially appreciate the "cheat answer" to the Gardner Puzzle at the bottom of the page. I have found that kind of meta-reasoning about questions quite useful, on exams and in games like Trivial Pursuit, for example.

<comment><USER6> Looking at the other comments, it seems like most people really like the cheat. I admit it's very cute, and you're absolutely right that this sort of thinking can be helpful in artificial situations like exams and games --- I've used it myself.

That artificiality is why I don't really like that approach, though. It's a brand of thinking that generally works only on artificial problems, because the key component ("you wouldn't be asking me this if it didn't have a well- defined answer") doesn't exist on most problems. Proving that it's constant and then using the r -> 0 trick to calculate the constant is much more satisfactory to me.

<comment><USER7> > that approach is a brand of thinking that generally works only on artificial > problems, because the key component ("you wouldn't be asking me this if it > didn't have a well-defined answer") doesn't exist on most problems

Perhaps changing the footnote hint "no more information is _given_ " to an integral part of the problem spec, and saying "no more information is _required_ ", would make the problem less artificial. There would then be a self-referencial component in the problem definition, self-referentiality being fairly common in nature and engineering.

</comment></comment></comment><comment><USER8> I once encountered question that had cheat answer. It was about trapezoid and it seemed that it had some data missing.

I was amazed that just assuming that the question had one answer allowed to reduce the problem to trivially calculable one by consistently manipulating the variables that were not given.

I was also pretty proud of myself for finding this solution.

</comment><comment><USER9> In the early eighties there was a TV show on German network television and I remember them presenting this puzzle, and I figured out the "cheat solution" as a kid.

It was this awesome TV game show that consisted entirely of Martin Gardner- style puzzles and other fiendish physics/biology puzzles, and contestants who were all scientists. Does anyone by any chance remember the name of this German TV show? I would really like to look up more information about it! It deserves to be remembered.

<comment><USER10> Likely Kopfball ([http://de.wikipedia.org/wiki/Kopfball_(Show)](http://de.wikipedia.org/wiki/Kopfball_\(Show\))), although that is late eighties (and the precursor late seventies)

<comment><USER9> I found it through some extra searching starting from your link- "Kopf um Kopf" ... Is that the precursor you were thinking of?

Here is a video on youtube that shows the awesomeness of this show (don't need to know German to appreciate it- On the start of the show, putting your finger under the device makes it spin in the opposite direction... why? The audience member with the right answer would get a reward.) [https://www.youtube.com/watch?v=1ObdE9n3UF4](https://www.youtube.com/watch?v=1ObdE9n3UF4)

Take a quick peek at a few other random spots of the show in the video and marvel at the awesome scientific experiments on live TV: I think that was one of the most bad ass shows ever, especially since it didn't have that pejorative "science shows are only for kids" thing happening.

<comment><USER10> I'm not sure. Wikipedia claims that he precursor also was called Kopfball, and has some overlap in the times when the series ran.

I just googled the "kopf" that I remembered in combination with "wissenschaft" (science) and "fernsehen" (television), and that popped up, and I thought "that must be it".

I should have been more cautious, though. German TV at the time had quite a few interesting programs about science (in a broad sense) I remember programs where they taught differentiation and integration, live chess (or at least, it seemed like that; if a player takes ten minutes to think about a move, the commenters would explain what the players might be thinking about) by such players as Karpov, Timman, and Anand ([http://en.m.wikipedia.org/wiki/Chess_of_the_Grandmasters](http://en.m.wikipedia.org/wiki/Chess_of_the_Grandmasters)), Hobbythek ([http://de.m.wikipedia.org/wiki/Hobbythek](http://de.m.wikipedia.org/wiki/Hobbythek)) about DIY, (with subjects such as "we build a hot air balloon", "book binding", and "stereo photography"), and that program about personal computing whose name I don't remember.

<comment><USER9> Yeah, Hobbythek was awesome, too... I remember at maybe 8 years old trying to rebind one of my own broken books after probably seeing that exact episode you mention...

It's really easy to get into this mode where you think "If it's not new and American it's crap" because the US in recent years has been so prodigious in creating such a large range of media of many different types, and because the US is not shy in "Americanizing" things from other countries and improving on them. However, there are definitely corners of brilliance lying in the past and in other countries that have been forgotten, and will be rediscovered in future years.

</comment></comment></comment></comment></comment><comment><USER11> The cheat is absolutely brilliant reasoning.

<comment><USER12> Ehh, sort of. It's logically flawed as phrased, in that the first bit is false: "If the problem is being posed, it must have a constant solution" is false; it could just as well be a niftily-simple symbolic solution too.

(I say "as phrased" because as other commenters observe, there are mathematically valid ways to arrive at the result.)

This reminds me of one of my favorite joke proofs from when I was in school. (It's so simple I'm sure it has other originations but AFAIK I independently recreated it.)

"The problem begins with the phrase 'Show that...' or similar. All previous problems that began with that phrase have been provable. Therefore, by induction, the claim I am being asked to prove must be correct. QED."

Unfortunately, this proof line hit a snag about halfway through my graph theory class in which we were assigned a problem out of the book that turned out to ask you to prove a false statement. (Well, a snag above and beyond the fact that that is an invalid inductive proof in general, _ahem_.) It was a typo and clearly accidental, but it was enough to break my proof forevermore. May you have better luck with it.

<comment><USER13> My favorite "cheat" was on an exam in an algorithm-analysis class (lots of discrete math, infinite sums, and proofs). We were asked to prove or find a counterexample to some conjecture or other. Looking at the conjecture, and looking at the clock, I decided the only thing I could complete in time was to find a counterexample. Indeed within a few minutes I found one and wrote it down.

I was the last to finish my exam and handed in my booklet; the prof looked it over and noted I was the only one (out of 5) in the class to get that particular question correct. I explained to him my "reasoning". He said, "yes, that is how you're supposed to do it!"

<comment><USER14> In a linear algebra class, we were asked to prove that a theorem is true if and only if X. (I think X may have been that a matrix had a nonzero determinant.) I had no idea how to do it, but I noticed the theorem was trivially true in the case of a 1x1 zero matrix. Since this matrix has a zero determinant, it spoiled the "only if" part. I wrote it as a counter-example, showing that "if AND only-if" was false, and didn't prove the "if." Full credit!

</comment></comment></comment></comment><comment><USER15> There is a simpler puzzle that's similar to Gardner's, but as unintuitive.

Take an orange and wrap a string around its diameter. Now extend the string by 1 inch and redistribute it around the orange so that it floats an even distance from it. Do the same with the Earth, i.e. wrap, extend by 1 inch and even out into a circle. The gap betwen the string and the orange/Earth - <USER2> one is bigger?

<comment><USER16> Spoiler below!

The relationship between radius and circumference is linear (C = 2pi * r). When the circumference is increased by 1, the radius increases by 1/2pi. Therefore, the gap has the same size.

Let g be the gap. Then

              C + 1 = 2pi * (r + g)
        2pi * r + 1 = 2pi * (r + g)
        r + 1 / 2pi = r + g
            1 / 2pi = g

</comment></comment><comment><USER17> It wasn't (initially) clear to me that the cylindrical hole must enter and exit the sphere.

With that knowledge the solution seems pretty intuitive.

<comment><USER18> How can one drill a 6" long hole through a sphere of more than 6 inches in diameter?

What I mean is if you drill 6 inches into the earth, you haven't passed through the other side...

edit: I think they mean drill a 6 inch hole of maximum width, which of course would just leave a very thin ring of the earth 6 inches tall.

<comment><USER19> In the puzzle statement, "A six inch high cylindrical hole is drilled through the center of a sphere," __through __is the keyword, rather than __into __.

I made the same initial mistake of misreading through as into.

<comment><USER20> In that case, the statement from the article "We could have a sphere as large as a planet, bore a hole 6" in length through it..." seems inconsistent. Either the cylinder is not 6" long, or it does not go through the sphere.

<comment><USER21> I thought that at first, but the length of the hole is dependent on the width of the hole. The wider the hole, the shorter, because wider holes remove bigger caps.

With a sufficiently wide hole, you could indeed drill a 6" hole through a spherical Earth, it'd just look more like a thin ring the diameter of the Earth than a sphere.

<comment><USER22> i still get a confused language impression from that. for me it's the combination of "drill" and "through" (probably was forced to take too much 'wood shop')

i think it would be more clear to phrase it starting along the lines of: position a cylinder concentric and inscribed within a sphere ...

</comment></comment></comment></comment></comment></comment><comment><USER23> I'm just pleased that I finally found a math puzzle on HN that I was able to solve without looking ahead! (Although I called the radius of the cylinder r/a, where r is the radius of the sphere, which ended up making my math a bit messy...)

Regardless, deserves an upvote just for the cheat answer at the end. I like that kind of reasoning!

</comment><comment><USER24> I remember reading and solving this the problem as the end of the article ("A six inch high cylindrical hole is drilled through the center of a sphere. How much volume is left in the sphere?") as a kid. I did it the hard way using the formula's, but the whole point of the puzzle was what this article called the "cheat" answer. It reduces the solution to utter simplicity by application of some elegant logic. It's not something of a cheat -- it's the whole point of the puzzle.

I really was a kid. I was taken in by Gardner's April 1st column that claimed among other things that a proven solution for "Chess" indicated that white would always win and that the opening move was P-KR4 (h4 in modern notation).

<comment><USER25> It only wouldn't be a cheat if the question were framed as "Surprisingly, the volume of the ring is constant, regardless of the radii of the sphere and circle. What is that constant volume?" Otherwise, you don't really know your answer is correct. Maybe, they wanted the answer in terms of R1 and R2. And if you are able to trick them into confirming it, then you're essentially using social engineering to leverage somebody else's work (the person who discovered the surprising result) in formulating your answer. There's no way that the original discoverer could have used that method. That's what makes it a cheat.

Back in middle school geometry, I came up with my own strategy for proving theorems. "You wouldn't ask me to prove it if it weren't true, therefore it must be true. Q.E.D." It didn't go over well.

<comment><USER26> That's why in Uni they usually asked: prove this statement right or wrong (or undecidable).

</comment></comment></comment><comment><USER27> I solved the puzzle in my head before getting to the second paragraph in the article. Here's the reasoning:

1) The area of a circle has a fixed ratio to the area of a square inscribed in that circle.

2) Therefore the volume of a cylinder has a fixed ratio to the volume of a square box of the same height, which sits inside that cylinder.

3) Therefore the biggest cylinder corresponds to the biggest box that can fit inside the sphere.

4) That box is obviously a cube, because what else could it be?

5) If a cube is inscribed in a unit sphere centered at the origin, the corners have coordinates ±1/√3, ±1/√3, ±1/√3.

6) Now you can calculate the volume of the cylinder in your head. Do it!

<comment><USER28> Congrats! But I hope all that is a humblebrag, not a reflection of your standards as a technical interviewer ;-)

<comment><USER27> Yeah, that was definitely a brag, nothing humble about it :-)

When I was a bit younger, I was in fact the typical asshole interviewer who would ask lambda calculus questions. Now I mostly stay away from interviewing, because I can emphasize much more with the pressure that candidates feel.

</comment></comment></comment><comment><USER29> It reminds me of two other neat problems -

1\. Imagine a band stretched taught around the diameter of the earth (which, for the purposes of this question, is a smooth sphere). Now imagine that the band is raised one metre from the ground at every single point along its length. How much longer is it?

2\. Imagine perfectly parallel lines painted on the floor, exactly one foot apart, and a rigid needle of length one foot. If you throw the needle to the floor at random, what is the probability that it crosses one of the lines? (This one has a nice 'cheat' solution just like the OP article).

<comment><USER30> Maybe I am missing something here, but why is 1 a neat problem? You are increasing the radius by 1 meter so the length of the band is now 2 _pi_ (r_e + 1) making the increment 2*pi meters. Is the surprisingly low increment the point of the problem?

<comment><USER31> Most people when faced with 1 and knowing the huge circumference of the Earth estimate (using Sytem 1 type thinking, [http://en.wikipedia.org/wiki/Dual_process_theory#System_1](http://en.wikipedia.org/wiki/Dual_process_theory#System_1)) it would be longer by kilometers.

</comment></comment></comment><comment><USER32> The author never pays off the answer to the initial question- is it a fat cylinder or a skinny one? We know it has height ~1.15R (where R is sphere's radius), but this is not easy to visualize.

<comment><USER33> The cylinder is sqrt(2) (~= 1.41) times as wide as it is high. So... a little fat? (I don't know the healthy baseline for cylinders...)

</comment></comment><comment><USER34> To estimate an answer for the first problem: consider two cylinders with zero volume, the one equatorial (r=R) and the other polar (r=0). Both correspond to zero volume. Hence the maximum volume occurs (hand wave) for a height somewhere between 0 and 2R. Not knowing better, the initial estimate for h is to bisect this interval, to give h_est = R (compare 1 with 2/sqrt(3) ~= 1.15).

It would be nice to construct a first order correction term. Any ideas?

</comment><comment><USER35> Regarding the Napkin Ring part of the problem.

Had the author simplified the formula for "Vanswer" rather than plugging in values for h & c, he would have gotten:

Vanswer = (Pi/6) * h^3

From which is it easy to see that the answer in this particular case is 36 * Pi but it also makes clear that the answer does not depend on R.

</comment><comment><USER36> ...and its Archimedean inverse(his favorite!): a Wilson(tm)* soccer ball enjoys 2/3 the volume and 2/3 the surface content of its minimal cylindrical official Wilson(tm) shipping clear plastic blister pack.

*citation: Tom Hanks in "Castaway"</comment>


MetricsDB: TimeSeries Database for storing metrics at Twitter - <AUTHOR> https://blog.twitter.com/engineering/en_us/topics/infrastructure/2019/metricsdb.html <comment><USER2> A blog post by Dan Luu ('luu) about using this to save twitter a lot of money on RAM for JVMs: [https://danluu.com/metrics- analytics/](https://danluu.com/metrics-analytics/)

</comment><comment><USER3> Still waiting for someone to open source an observability db that uses distributions as the core storage abstraction instead of just single values.

I thought Circonus' IronDB was open source but they've either brought it back closed source or I was mistaken.

In any case it seems weird that everyone is just rehashing the same space with metrics dbs and not moving to where Google/Circonus have been for a while.

<comment><USER4> clickhouse as aggregate functions (Pearson correlation coefficient,quantiles , etc... )[0].

Associated with their Materialized view and Liveview features, you can achieve observability [1].

[0][https://clickhouse-docs.readthedocs.io/en/latest](https://clickhouse- docs.readthedocs.io/en/latest)

[1][https://www.altinity.com/blog/2019/11/13/making-data-come- to...](https://www.altinity.com/blog/2019/11/13/making-data-come-to-life-with- clickhouse-live-view-tables)

Edit: fixed broken link.

<comment><USER5> That second link seems to be broken. The base URL is: [https://www.altinity.com/blog/2019/11/13/making-data-come- to...](https://www.altinity.com/blog/2019/11/13/making-data-come-to-life-with- clickhouse-live-view-tables)

<comment><USER4> fixed. Thanks.

</comment></comment></comment></comment><comment><USER6> Is this open source? How does it differ from TimescaleDB or InfluxDB?

<comment><USER7> MetricsDB appears to be HDFS-based. TimescaleDB is Postgres-based and InfluxDB is memory and log-based. Prometheus is file-based.

<comment><USER8> While HDFS is indeed used for exporting old data and storing some partition mapping metadata, it's clear from the blog post that MetricsDB is much more reliant on BlobStore as well as MetricsDB-specific services.

> The servers checkpoint in-memory data every two hours to durable storage, > Blobstore. We are using Blobstore as durable storage so that our process can > be run on our shared compute platform with lower management overhead.

</comment></comment></comment><comment><USER9> This applies to all time series databases... What is special about time series that makes unique databases attractive? Is it the assumption that only 1 main index is created (time)? Is is that the data is append-only?

In particular, I'd love to know if theres anything major that generic RDBMS's could do better here.

<comment><USER10> Yes, time-dependence and only numbers allows to use a lot of tricks that cannot be done with generic data.

For example, double delta compression, or Gorilla for floating point numbers. For more, take a look at our open source VictoriaMetrics database, which uses all of such tricks.

<comment><USER11> ClickHouse DBMS allows to combine delta, double delta, Gorilla with LZ4 or ZSTD for column compression. But it is not often used as DB for monitoring metrics so something else is probably expected from time series DB.

<comment><USER7> It would work, but Clickhouse is a Russian (Yandex) thing, and SQL isn't really needed for most monitoring and alerting use cases.

If I didn't want to use MySQL or Postgres, I'd rank Prometheus #1 and Clickhouse #2.

The killer thing for Clickhouse is that Percona supports it, so if you want to outsource the installation, mgmt. and support, you can just write a check and get good results.

Also, Clickhouse is a column store with SQL, so you could use an instance for monitoring and another to replace Vertica or Greenplum or whatever so long as it has the client libraries you need.

<comment><USER12> ClickHouse is licensed under Apache License 2.0 and Yandex is incorporated in the Netherlands. What are your concerns with it being developed by russians (other than xenophobia)?

<comment><USER7> No, you're the xenophobe.

I'm concerned with there being a support issue, as well as a smaller user base also affecting support in the long term.

</comment></comment></comment></comment></comment></comment><comment><USER13> I'd be curious to see a couple of organizations that have taken different approaches in this area discuss how they made decisions around investing in observability systems. When metrics gets to be a meaningful fraction of your total data size, and when it gets to be an operational burden on its own, I think it's tempting to ask "Do we really need to emit and store all this?" especially if you acknowledge that only ~2% of writes ever get read.

In some sense, the organization is making an expected value of information estimate, where there's a complex interaction between the data you have and actions you can take, especially to preempt or resolve issues.

<comment><USER5> I've seen two approaches. (We have a lot of customers who deal with this exact problem.)

1.) Down-sample data to aggregates. Keep the aggregated but drop the source data after a certain period of time. You can see historical trends down to the sampling interval but cannot drill down to individual observations.

2.) Use tiered storage. Keep recent data on NVMe SSD and migrate to high density HDD or object storage over time. This allows you to keep more data. Not all DBMS support this out of the box.

</comment></comment><comment><USER14> 1.2 PB saved using gorilla compression! Didn't realize how sparse this data was.

<comment><USER15> Not necessarily sparse. Just lacking in entropy.

You can compress an arithmetic series down to a handful of bytes...

</comment></comment><comment><USER16> And yet another company that builds its own (time-series) database. Why does this keep happening for such a narrowly defined use-case at similar scales?

At 1.5 petabytes, they could just dump this in Google's BigQuery for the cost of an engineer and have full SQL power.

<comment><USER17> Bigquery does not have anything close to real time query support like you want/need for observability data. And plus the cost is more like one engineer per query. That shit is crazy expensive.

<comment><USER16> What do you mean by real-time? BQ can scan a petabyte in less than a minute, but partitioning and clustering will get most simple queries back in a few seconds. If you mean data freshness than BQ supports streaming ingest and will show those rows in the next query.

SQL also lets you do a lot of aggregations in the database instead of pulling back raw metrics. And BQ has flat-rate pricing and discounts for large clients.

<comment><USER17> Less than a minute is not real time. Something like < 100ms for most queries would be real time.

And then do that 10s of 1000s of times per minute, every minute, 24 hours a day.

You can see how the query costs add up...

I know it's easy to come on HN and play armchair architect but this is one area where you are just incorrect in your assumptions. Did Twitter need to build a(nother) in-house database? I don't know for sure. But I do know that BQ is not well suited to observability use cases.

<comment><USER16> As I said, that's for scanning a petabyte. Selective queries would be much smaller and faster, costs can be handled with flat-rate pricing, and there's the BI Engine (in-memory cache) feature too.

The details in the article (1.5PB logical data, 3x replication factor, only 2% of metrics ever read) and the fact that Twitter already runs infra in Google Cloud seem to align well with BQ in my experience (10+ years in adtech building even more complex backends).

Please tell me what assumptions are incorrect so I can reconsider.

<comment><USER17> My experience is ~2.5 years as a senior engineer on the Observability team at Twitter (from 2013-2015). I was part of the migration to Manhattan (mentioned in the post) from Cassandra, new alerting infra, query language design, among many other things. Your adtech experience should make you particularly sensitive to query latencies, so I find it interesting that you're glossing over that.

Yes, only 2% of metrics ever read. That was the same back then too. The kicker is that you can't be sure which will be read, so the systems are built to be able to read any of the metrics with the same SLA. This is especially critical during an outage where engineers will need to _quickly_ read metrics that in many cases were only written only a few seconds ago that they wouldn't otherwise read and are not configured as part of an ongoing alert.

Additionally, the alerting infrastructure that runs on top of the TSDB is configured with 10k+ queries that run every minute. So even at 2% read, you're querying them over and over and over again because you always need the latest data plus whatever trailing data is needed to fulfill the alerting needs (trailing 10m, hour, day, month, etc). This also makes it a particularly hard caching problem.

I can't speak to the state of GCP at Twitter. I know they had started to migrate some things but when I was there it was all colo.

 _Could_ they use BQ? Sure it could probably be tooled with partitions/caching/etc to work I guess, but at the end of the day it's not what BQ is designed for which is data warehousing and BI. _Could_ they have used something that wasn't a custom built in-house database? Yea almost certainly (No secret that there is some NIH going on at Twitter well before this)

This happens alll the time on HN so I'm not here to call you out specifically...but it's really easy to read a company's blog post about their infrastructure decisions and immediately scoff and say "psh why didn't they just use X?"...as if you now have the same context that the engineering team has by reading a single blog post.

</comment></comment></comment></comment></comment></comment><comment><USER18> Could someone tell me the disadvantages of solving the problem in the article using S3? I am actually curious.

<comment><USER7> They're inserting 5 billion events per second (for some reason), and AWS throttles everything (the following link mentions a limit of 3,500/second):

[https://aws.amazon.com/about-aws/whats- new/2018/07/amazon-s3...](https://aws.amazon.com/about-aws/whats- new/2018/07/amazon-s3-announces-increased-request-rate-performance/)

Also, if they're planning to do alerting, S3 would not help with that.

PUTs are 1 cent per 2000:

[https://aws.amazon.com/s3/pricing/](https://aws.amazon.com/s3/pricing/)

<comment><USER19> 5 billion events per second? That seems to be too much for monitoring data even at Twitter's scale. The article refers that they store the raw data and use rollup tables for aggregate queries, I wonder how much the ELT process costs in total.

<comment> dan-robertson Indeed, the article mentions 5b metrics per minute

</comment></comment></comment></comment><comment><USER20> If there is interest, we're working on time-series data labeling tool. Sneak peek at it here:

[https://htx-pub.s3.amazonaws.com/Screen+Shot+2020-06-17+at+1...](https://htx- pub.s3.amazonaws.com/Screen+Shot+2020-06-17+at+17.56.23.png)

If you think it might be useful and you want to try it out drop me an email michael @ heartex.net

</comment><comment><USER21> Can someone offer up how this is different than InfluxDB? Also a bunch of machine log folks are jumping on the observability 'marketing' bandwagon - i.e. Elastic Search, Splunk, Devo, SumoLogic etc. Interestingly don't see them here..</comment>


The Nonprofit Model is Broken - <AUTHOR> http://bhargreaves.com/2010/12/nonprofit-model-broken/

<comment><USER2> It's down.</comment>


Google just released its internal tool to collaborate on AI - <AUTHOR> https://qz.com/1113999/nerds-rejoice-google-just-released-its-internal-tool-to-collaborate-on-ai/ <comment><USER2> I was hoping to see a project which allows users to collaboratively train an AI model, so that open source projects can compete with corporate clusters of 100 GPUs. (e.g. to create useful speech recognition engines etc)

</comment><comment><USER3> We've also been working on ML collaboration at [https://www.tinymind.com](https://www.tinymind.com). In addition to training models collaboratively, we have free, hosted Jupyter notebooks for both Python 2 and 3 with popular ML frameworks that you may want to check out :)

<comment><USER4> Interesting. I like the simple design/interface you've put on top of the various ML engines, including how to tweak them. Overall approach reminds me a bit of Digital Ocean's friendly UI for servers/hosting. Haven't tried the collab features yet.

</comment></comment><comment><USER5> Looks like I can not really use it just yet, need sign up and then be put into a waiting list.

Is it true that Collaborate is on Python 2.7? Since everything is switching to 3.x these days it would be nice they can update Python version as well. This tool is built on top of ipython-notebook(jupyter) which supports 3.x officially already

</comment><comment><USER6> Does anyone have a more in depth source or a link to the source code?

I'm currently using a slightly hacked Jupyterhub to do what I assume is the same thing, I can't work out from that article (maybe I missed it) wheree the kernel actually runs with this?

It's not useful to me unless you can self-host and hack it. If that's possible and they add python3 will probably end up putting it into prod.

<comment><USER7> GitHub.com/jupyter

<comment><USER6> Thanks for replying but I know what Jupyter is and I'm already using Jupyterhub. My question was on more details/source code for this Google service that they've built on top of Jupyter which looks equivalent to a hosted Jupyterhub.

<comment><USER8> I think it's a completely new UI that some people at Google built from scratch, which is much easier for them to do realtime sync with using their API ([https://developers.google.com/google- apps/realtime/realtime-...](https://developers.google.com/google- apps/realtime/realtime-quickstart)). I think they felt there were also some usability issues with the standard Jupyter UI when sync was added in. (Source: I visited the group writing this a few years ago at Google and they showed me a demo which looked just like this.)

</comment></comment></comment></comment><comment><USER9> For googlers, how does this differ from datalore.io?

<comment> d--b Looks the same to me...

</comment></comment><comment><USER7> I already signed up for their free CPU cycles.

</comment><comment><USER10> doesn't work in Safari

<comment><USER11> Use Firefox

<comment><USER12> > ...supports Python 2.7 and has to be used on Google Chrome.

Looks like that's not possible.

</comment></comment></comment><comment><USER13> > With Colaboratory, users create notebooks, or documents, that can be > simultaneously edited like Google Docs.

I wonder how Google came up with this "feature". If there's one thing I don't need, it's other users editing my code behind my back. Probably a case of "just because we can".

<comment><USER14> Would you say the same thing about the collaborative editing features in Docs?

> If there's one thing I don't need, it's other users editing my _prose_ > behind my back.

To me, this sounds like a fantastic teaching tool, a fantastic pairing tool, a fantastic interview tool, and a cool way to collaborate on scripts where Git would have too much friction.

I think this sort of tool can even scale up to projects larger than a single file — I've used Floobits ([https://floobits.com/](https://floobits.com/)) to co-op edit whole Django webapps in Intellij. It's super fluid and productive to use co-editing tools to very quickly collaborate on new feature build-outs, either pairing together on a single file or splitting work between Frontend and backend concerns.

<comment><USER13> > Would you say the same thing about the collaborative editing features in > Docs?

Yes, a simple "sync" button that pulls in the updates and merges the differences (like git) would do for me, and in fact would cause much less confusion than somebody editing behind my back.

<comment><USER15> Then perhaps this isn't for you? Collaboration tools such as these are fantastic for some teams and use cases.</comment></comment></comment></comment>