Randall Munroe Responds to Gelman's Criticism of XKCD - <AUTHOR> http://andrewgelman.com/2012/11/16808/#comment-109366 <comment><USER2> > A sincere thank you for the gentle corrections; I've taken them to heart, > and you can be confident I will avoid such mischaracterizations in the > future!

Why Randall should almost apologize for a comic is a mystery to me.

<comment><USER3> He's not apologizing for a comic -- he's apologizing for mischaracterzing people through his comic. He's not saying the point is wrong -- just that the labels are :)

I think it's a thoughtful, solidly nice and totally stand-up (and mature) thing for him to have done.

<comment><USER4> Yeah, the point is apparently just that unthinkingly accepting p = 0.05 as your cutoff for significance is moronic (this point is utterly uncontroversial among thinking humans), whereas the labels make it seem as if he's trying to say something about the controversy between frequentists and bayesians, _which he didn't even know existed_.

<comment><USER5> > (this point is utterly uncontroversial among thinking humans)

Are you sure about this? Most people don't know any statistics, and the vast majority of people who know _some_ statistics never took it further than Stats 101. So a lot of people _do_ blindly just apply p tests like this.

Randall's point about cancer statistics just serves to give a real world example of this, and why it's so important people better understand it.

<comment><USER4> Well, some people might, indeed, be educated stupid, though I hope to god no actual Stats 101 class teaches people that p < 0.05 = significant, other than as a convention that people might want to be aware of when they encounter the phrase "statistically significant".

I would be legitimately shocked if even a single doctor were willing to defend that p-test as _in principle_ what it is for something to be statistically significant _no matter what thing is at issue_ , rather than as a decent enough heuristic.

Nevertheless, the parenthetical you highlight is, indeed, probably literally false. I was employing a rhetorical device to convey the degree to which I found the point that, evidently, is the one Randall _actually_ intended to be trivial.

<comment><USER6> _I would be legitimately shocked if even a single doctor were willing to defend that p-test as in principle what it is for something to be statistically significant no matter what thing is at issue, rather than as a decent enough heuristic_

Unless things have changed in the last twenty years, when I did a series of interviews with doctors about Bayesian probabilities in relation to an expert system project, you're going to be unpleasantly surprised at both the number who do - and the number who aren't really very sure what a p-test is at all...

</comment></comment></comment></comment></comment></comment><comment><USER7> Oh! I didn't realize this was Gelman's blog. He is, at least, a dude who seriously knows what he's talking about. <http://en.m.wikipedia.org/wiki/Andrew_Gelman>

<comment><USER8> Yes, I originally started <http://news.ycombinator.com/item?id=4769667> with title "Andrew Gelman: I don't like this cartoon", but for some reason mods decided to get rid of "Andrew Gelman" in the title. I mean, the whole point is that it's a _very_ famous statistician, not just another blogger ranting.

</comment></comment><comment><USER9> His name is spelled "Munroe", not "Monroe". Interestingly, the correct spelling of his name is literally the first thing you see when you click that link.

</comment><comment><USER10> LW discussion of the comic: [http://lesswrong.com/r/discussion/lw/fe5/xkcd_frequentist_vs...](http://lesswrong.com/r/discussion/lw/fe5/xkcd_frequentist_vs_bayesians/)

</comment><comment><USER11> Wait, hold on. Isn't the joke that the Bayesian statistician knows that paying back 50 dollars is meaningless if the sun exploded and we're all going to die? If we live, he gets 50 dollars. That's what I got out of it.

</comment><comment><USER12> _> More importantly, now that both you and perhaps Scott Adams have commented on my blog, I am very happy. If only I could think of some way of getting Berke Breathed to comment here, I think I could just retire!_

Could someone point me to the "perhaps Scott Adams" comment? (is it in response to another blog entry?)

<comment><USER13> [http://andrewgelman.com/2011/04/dilbert_update/#comment-5917...](http://andrewgelman.com/2011/04/dilbert_update/#comment-59176)

<comment><USER12> Thanks! :)

(Now I see why it's _perhaps_ Scott Adams)

</comment></comment></comment><comment><USER14> AFAIK, the breakdown is this:

Frequentist is a special case of Bayesian. The Frequentists have much more mature tools, because they've been working on them since Gauss's day. Bayesians (especially less experienced ones) may claim that that Frequentists are old school, outdated, and don't teach undergrads the new Bayesian way of doing things.

Bayesian methods are more flexible and general, but are often slow (computationally), and can be too flexible. A Bayesian can prove anything. Frequentists have trouble eliminating some biases (because their tools aren't as flexible), but also have trouble purposely (or subconsciously) biasing their results.

I'm not going into the specifics of the methods here, just the source of their disagreements.

<comment><USER15> Almost everything written here is incorrect.

First of all, Bayes predates Gauss. It is inaccurate to suggest frequentist statistics predate Bayesian statistics.

Second, neither is a special case of the other.

Third, neither Bayesian methods nor frequentist methods are inherently "more flexible."

Bayesian statistics set up a model and infer model parameters by applying Bayes' rule to the data. Bayes' rule is an indisputable rule of probability.

For any model parameter b, the result of applying a bayesian model is a probability distribution for b given the available data. Criticisms of bayesian models typically center around the fact that you must use a "prior distribution" indicating the modeler's beliefs about b before seeing the data. Bayesian statisticians have a number of responses to this criticism (that some people find compelling, and others do not).

Frequentist methods build models that are justified by their properties in repeated resampling. For instance, a frequentist method is "unbiased" if, given multiple hypothetical samples, it would on average produce the correct parameter b. Frequentist hypothesis testing reports the probability of observing specified data given some assumption about b.

A standard criticism of frequentist methods is that a modeler wants a probability distribution for an unknown parameter given the known data... rather than knowing the probability of observing the realized data given some assumption about the parameter.

</comment></comment><comment><USER16> I love this line.

 _The truth is, I genuinely didn't realize Frequentists and Bayesians were actual camps of people—all of whom are now emailing me._

<comment><USER17> He should be doubly cautious of Rule 34 in this case.

<http://xkcd.com/305/>

<comment><USER18> How viewer must recognize who are Frequentists and Bayesian in porn? Of course it possible to place tattoos with formulas on bodies or put some blackboards in background (conventional plot about student and teacher?), but can it be more elegant? May be some joke about contraception?</comment></comment></comment>


Spymaster - Twitter-based MMORPG - <AUTHOR> http://www.techcrunch.com/2009/05/28/spymaster-the-twitter-game-that-will-assassinate-your-time/

<comment><USER2> Straight out of the pages of a Charles Stross novel. Life in the 21st century does have it's benefits.

</comment><comment><USER3> "Twitter-based" seems a bit generous. It has its own site, and just uses OAuth and lets you tweet A) the things you do in the game and B) invitation links. Seems pretty gimmicky, though I have to admit, that's what I thought about Twitter itself at first, so maybe this game will be huge and I just don't get it.

</comment><comment><USER4> pretty cool game</comment>


Shenzhen park flooded with thousands of shared bicycles - <AUTHOR> http://shanghaiist.com/2017/04/05/shenzhen_bike_apocalypse.php <comment><AUTHOR> What I don't get about bicycles and Sinophone countries is this: in countries that usually price space a premium in cities, bicycles congregate in large clusters, fall into disrepair, and the supply in these clusters always outstrips demand, but the clusters form nonetheless, and are utilized ( they're not refuse ), but always under capacity. The only explanation I can offer is maybe there's something very Chinese about huge clusters of bicycles that is reassuring so long as they are there, even if they are mostly not under active use.

Some strange equilibrium point between inconvenient and useful. I don't know why the bicycle is subject to this "natural law" but is nothing else I've observed.

It might not just be a Chinese thing. I haven't travelled to Russia, South America or Africa...maybe it happens there too.

<comment><USER2> A cycle left on it's own it more likely to be stolen. You park it next to other, nicer, less well-secured bikes in the hope that they will steal those instead.</comment></comment>


Ask HN: Robotics for a 5+ year old? - <AUTHOR> Hey Guys, My 5 year old has been doing lots of talking about robots lately and how he wants one that can do stuff that people do.  I love his amazing imagination when it comes to this stuff and I would love to get him into it this Christmas.  I'm looking for small robot starter sets that we could use as a first step learning tool.  It should be one of those projects that gets a quick first result, even if very rudimentary and then build on top of that as time moves forward.  My initial thoughts were a Raspberry Pi robot as that would probably be the most flexible moving forward, but might be a little over his head and he would be dependent on my free learning time to keep things moving forward.  Any other options out there that might be good for a kid his age?<p>Thanks. <comment><USER2> Lego Mindstorms are probably the best fit . . .

I coached a lego robotics team for the first time this year . . . FLL starts at age 8/9 till age 13/14\. It was a great experience.

There is a Jr. FLL you might want to check in to for him. [http://www.usfirst.org/roboticsprograms/jr.fll](http://www.usfirst.org/roboticsprograms/jr.fll)

Mindstorms will be challenging for a 5 yr old but you can help him build his ideas . . .

The retail version has everything you'd need to get started . . . and would make a good christmas present.

Our local science center has a class you can sign up and take to get some hands on time with the kit before investing in one or maybe you have a friend that has one you can borrow.

Also the next generation of mindstorm came out in the last 18 months or so . . . there are lots of FLL teams upgrading from NXT (old version) to EV3 (new version) so you might be able to get an NXT version from a team that has moved on to EV3 (every team usually have 2 to 3 kits).

Good luck with robotics . . .

</comment><comment><USER3> Piccolo is a neat little plotter/drawing robot that's open hardware: [http://www.piccolo.cc/](http://www.piccolo.cc/)

Unfortunately it seems nobody offers a full kit, so you'll have to gather materials yourself. It looks like it'll take four or five orders... ponoko for the laser cut parts, seeed studio for the pcb, hobby king for motors, and so on. Also not sure it fulfills the "build on top of that" requirement either, though after it is built you could spend lots of time learning programming to make it draw cool things. Think LOGO with an actual pen!

</comment><comment><USER4> Funny, I just posed a related question: [https://news.ycombinator.com/item?id=8652432](https://news.ycombinator.com/item?id=8652432)

You might check out the Lego Wedo set, or Sphero: [http://stemkids.io/robots/](http://stemkids.io/robots/)

Or perhaps Snap Circuits or LittleBits: [http://stemkids.io/electronics/](http://stemkids.io/electronics/)

<comment><AUTHOR> I thought about the LittleBits last year, but they weren't available at the time (sold out I think). So then I thought about them again this year but he is talking about a Robot that can do stuff, even though LittleBits would be a good learning tool. The Mindstorms looks amazing and exactly what he would want to do. But I can't drop that kind of cash right now... so there in lies the dilemma. Even the NXT is a couple hundred on ebay. We will see what I can come up with.

</comment></comment><comment><USER5> Snap Circuits are good for teaching the basics of how electronics work.

MindStorms, in my mind, are really the best choice, despite the high cost. Nothing else is really as kid-friendly as I would like. My wife and I have purchased a few non-Lego sets and none of them have really been comparable.

</comment><comment><USER6> Have you looked at those different Lego construction kits? Double present I would say :)

</comment><comment><USER7> Lego Mindstorms</comment>


Facebook says it sold political ads to Russian company during 2016 election - <AUTHOR> https://www.washingtonpost.com/politics/facebook-says-it-sold-political-ads-to-russian-company-during-2016-election/2017/09/06/32f01fd2-931e-11e7-89fa-bb822a46da5b_story.html <comment><USER2> I implore everyone alarmed by recent Russian assaults on Western Democratic institutions to get their hands on an English translation of 'The Foundations of Geopolitics' by Aleksandr Dugin. Dugin is a Russian academic and the book is essentially a textbook on how Russia should seek to subvert and destabilize Europe, the US and other Russian rivals in order to forge a new empire for the explicit purpose of ethnic Russian domination of the world.

Dugin is not some fringe academic and the book is well known among the elite of the Russian state. According to the Wiki page:

>"The book has had a large influence within the Russian military, police, and foreign policy elites and was allegedly used as a textbook in the General Staff Academy of Russian military."

More on the book's approach towards destabilizing America:

>"Russia should use its special services within the borders of the United States to fuel instability and separatism, for instance, provoke "Afro- American racists". Russia should "introduce geopolitical disorder into internal American activity, encouraging all kinds of separatism and ethnic, social and racial conflicts, actively supporting all dissident movements – extremist, racist, and sectarian groups, thus destabilizing internal political processes in the U.S. It would also make sense simultaneously to support isolationist tendencies in American politics."

We must unite in the face of division, both for the sake of unity and because the Russians are actively trying to divide us.

I haven't been able to find a copy of the English translation online. I was only able to get my hands on one because of a connection that works in DC. If anyone knows where to get one online please post a link.

Wiki page: [https://en.wikipedia.org/wiki/Foundations_of_Geopolitics?wpr...](https://en.wikipedia.org/wiki/Foundations_of_Geopolitics?wprov=sfla1)

<comment><USER3> I'm curious: other than a will to power, is there any philosophical or principle-driven reason for Russia's actions? In other words, why don't Russians want Western liberal democracy for themselves?

<comment><USER4> The Russian _regime_ doesn't want Western liberal democracy because it's incompatible with the self-sustaining kleptocratic autocracy that they are running.

Individual Russian people may or may not want Western liberal democracy for any of a variety of reasons (they "not" side in part driven by regime propaganda), but those preferences are not controlling regime actions.

</comment></comment></comment><comment><USER5> As it is rarely mentioned in the US media, I'd just note that the US pretty openly interferes in Russian elections, and has been doing so for a long time. It's curious this is virtually never mentioned amidst all the accusations of Russia involving itself in US elections.

There are many tentacles to the US octopus, for starters you can look at an organization created by and funded by the US congress, the soi disant National Endowment for Democracy. They have a conference opening next week containing some of the people the US has been using to interfere in Russian elections ( [https://www.ned.org/events/prospects-for-russias- democratic-...](https://www.ned.org/events/prospects-for-russias-democratic- movement) ).

<comment><USER6> > virtually never mentioned

I (anecdotally) hear this tidbit in every comments section relating to Russia's involvement in the U.S. election. Almost as if it's a common talking point.

Reminds me a lot of [https://en.wikipedia.org/wiki/Whataboutism](https://en.wikipedia.org/wiki/Whataboutism)

EDIT: ... And so a conversation was effectively derailed and nothing was gained. Arguing about the validity of Whataboutism or who is doing it isn't the point. It's a deflection technique that works in spite of you calling it out.

<comment><USER7> It's the line that Putin and Lavrov (foreign minister) have been pushing since forever: "you westerners are no better than us, stop shaming us for the shady things that we do". That's an easy way to keep your population happy – tell them that life is no better in other countries. Corruption is everywhere, police state is everywhere, etc. And most Russians believe that.

<comment><USER8> Well it's true in the us, Russia, china, Australia. That's not everywhere but that is quite a lot of ground.

<comment><USER9> Russia does not have actual meaningful elections. That's a fairly fundamental difference.

<comment><USER10> Neither does the US really, in the sense that no matter who wins, you lose.

<comment><USER8> It's true. You can only win an election with piles of money. With gerrymandering and lying. The US government is a train wreck

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> Facebook's blog post on the matter: [https://newsroom.fb.com/news/2017/09/information- operations-...](https://newsroom.fb.com/news/2017/09/information-operations- update/)

<comment><USER12> ... and they do this while the were openly taking money from Russian/KGB campaigners round 2010, dispatching campaign consultants and overall trreating them as first tier clients. Same was true of Google in Russia, while they still had hopes of somehow greasing hands with the establishment. They evacuated the most valuable staff out of Russia around 2015 to Switzerland, when they finally gave up. I still do remember them quitely delisting online resources with corruption exposures circa 2007, that were reappearing with simple reversal of word order.

They may've been just "probing where the water is shallow" in 2010 with "seemingly innocent" think tank drivel and consulting buys, and then went full throttle when it did matter the most.

American top C-Levels, officials, 3 letter agency employees, and other American beau monde all have that "smart, sophisticated, brilliant, but damn naive" note in their personalities. All such types do remind me of people who are "trying to win in a casino," while having "I know what I'm doing" look on their face. They can't win there, it not their league.

To prevail, Americans need to overhaul their political establishment and institutes of power with virtuous and competent people.

If I was the US congress, I would've put it very square, if a dot com like FB actively conspires with Russians, and then pretends that they didn't, then everybody along the chain of command down to founders, c-levels, and major financiers (with their own respective boards) are detained, subject to criminal prosecution and given prison sentences

</comment></comment><comment><USER13> Out of curiosity, are there any laws on the books right now which are meant to prevent this sort of thing?

For example, let's assume a foreign government decided to purchase ads either backing or attacking a US political candidate, and they were completely overt about it. (No hiding behind shell companies or transferring funds anonymously; completely out in the open.) Are there any laws or systems currently in place which would stop them from doing that?

<comment><USER14> TFA:

"Under federal law and Federal Election Commission regulations, both foreign nationals and foreign governments are prohibited from making contributions or spending money to influence a federal, state or local election in the United States. The ban includes independent expenditures made in connection with an election.

Those banned from such spending include foreign citizens, foreign governments, foreign political parties, foreign corporations, foreign associations and foreign partnerships, according to the FEC."

<comment><USER13> > foreign governments, foreign political parties

How is that enforced? A US court can't exactly send the Russian government or the Communist Party of China to prison for breaking the law here, can it?

</comment></comment></comment><comment><USER15> Why would Facebook discriminate where ad $ revenue is coming from? Twitter, Instagram, doesn't matter, money is money.

<comment><USER16> Because foreign agents buying political ads violates US federal law

<comment><USER17> >Because foreign agents buying political ads violates US federal law

WRONG. Nothing prohibits foreign entities from buying their own political ads.

[http://fortune.com/2017/07/12/us-election-meddling-online- ad...](http://fortune.com/2017/07/12/us-election-meddling-online-ad/)

<comment><USER18> You're both right. Sounds like a law that hasn't been updated yet for _online_ ads.

>The laws that prohibit foreign nationals from spending money to influence U.S. elections do not prevent them from lawfully buying some kinds of political ads on Facebook and other online networks

</comment></comment></comment></comment><comment><USER19> If we just keep pointing at the Russian bogeyman, no one will care that individuals like Peter Thiel or companies spent 100 to 1000 times that amount of money on superpacs and eventually political ads for their favorite candidate.

</comment><comment><USER20> Can we see what the ads were? Can we see how they were targeted?

Am I missing this somewhere?

</comment><comment><USER21> I am in Switzerland and I keep seeing suggestions for "Anti trump" groups (None of my friends are part of them). Why?

</comment><comment><USER22> "Facebook officials reported that they traced the ad sales, totaling $100,000, to a Russian "troll farm" with a history of pushing pro-Kremlin propaganda, these people said."

Excuse my ignorance but isn't $100K chicken feed?

<comment><USER23> That's what Facebook self reported that it discovered, not necessarily the whole operation. It's also direct evidence of non-hacking related attempts to influence the election.

Furthermore, considering there is a criminal investigation going on over whether or not the Trump campaigned colluded with Russia on such activities, the dollar amount is pretty meaningless.

<comment><USER24> > criminal investigation going on over whether or not the Trump campaigned > colluded with Russia

It is technically a criminal investigation, though colluding with Russia would not be a crime.

</comment></comment></comment><comment><USER25> Too bad, I was looking forward to the abominable shitshow that was going to be Zuck's candidacy.

</comment><comment><USER26> This is likely just the tip of the digital advertising iceberg with all of this for FB and other advertising giants like Google and Twitter. According to this Wired article[1] Giles-Parscale, the agency behind a lot of the Trump election digital efforts "took in some $90 million, the vast majority of which went toward buying Facebook ads for the campaign."

A Reddit user who "was working at the other end of this pipeline that is selling digital adspace to consulting firms" (according to his post) also posted some very detailed insights as to how funds might have been laundered into clean political donations via small agencies and PACs in the US via digital media buys.

Here's his follow-up post from three months ago which has more info and links to his original post--really worth reading all of it, his original post, the subsequent threads, etc.: [https://np.reddit.com/r/politics/comments/6alzm0/fbi_confirm...](https://np.reddit.com/r/politics/comments/6alzm0/fbi_confirms_activity_in_annapolis/dhfnm85/?sh=8bd62816&st=J2KZA1Y9)

So the issue is less about ads purchased directly from Russia or by illegitimate companies with Russian IPs or anything of the sort as FB addressed in this statement. That is a drop in the bucket compared to the dollar amounts that might have been spent as part of the rest of the Trump media buys. The bigger story may actually be more about Russian (and other funds) funneled through small legit digital agencies and PACs (or through Parscale) in the US who then did ad buys driving for Trump donations which he was then able to legally use as campaign funds (or for enriching himself as the Reddit user hypothesizes).

There are likely FB sales reps and others there who have some insight into the ad campaign objectives, targeting details, and the source of that targeting data. The last bit is interesting there because it is still not fully understood how Cambridge Analytica plugged into all of this, and how they obtained their detailed targeting data. Mercer and Bannon both have direct Cambridge Analytica ties[2].

All of this is to say that I don't think the full story is being unearthed with these data points that FB shared. And I'd be willing to bet that FB sales reps who dealt with these accounts during the election know quite a bit more about what actually went on. However we may never know how deep the rabbit hole really goes, how dirty such funds might actually be, or how much FB, Google and Twitter really profited from the election (and ongoing campaigns).

I want to conclude this by stating that I am just posting my own personal thoughts based on a variety of articles and Reddit posts, and I don't purport to have any inside insight into these companies operations beyond my fairly deep experience in the online advertising space and my understanding of how that operates, so this is largely speculation and you should come to your own conclusions. That said, it is hardly outside the realm of possibility at this point to imagine how this sort of scheme could work, and how all of the players involved might have had strong financial incentives to not rock the boat.

[1] [https://www.wired.com/story/trump-russia-data-parscale- faceb...](https://www.wired.com/story/trump-russia-data-parscale-facebook/)

[2] [http://www.newsweek.com/did-russians-target-dem-voters- kushn...](http://www.newsweek.com/did-russians-target-dem-voters-kushners- help-613612)

</comment><comment><USER27> I guess Citizens United doesn't just apply to citizens.

<comment><USER28> Citizens United was a good ruling because Congress and both political parties have been doing their damn best to limit the number of ways money can get into politics unless it is to them.

How? Simple, donations under $200 per person do not need be revealed until they exceed that number. Also taking foreign donations without proper tracking and the ease of generating new cc numbers / names works as well. However the number one method they use is exploiting campaign fundraising events where they can rake in millions. With the clout and friends in business how can a third party or anyone compete against it?

So while there are issues with money in politics all attempts to limit it have simply been done to protect the two parties who hold near absolute power.

</comment></comment><comment><USER29> The update for this article says that $100,000 ad buy may have reached 70,000,000 Americans.

[http://www.thedailybeast.com/russias-facebook-fake-news- coul...](http://www.thedailybeast.com/russias-facebook-fake-news-could-have- reached-70-million-americans)

</comment><comment><USER30> Regardless of legality, it will be interesting to see how this is spun if/when Zuckerberg runs for public office.

</comment><comment><USER31> In the context of the 2016 election 100.000 usd is a tiny amount.

<comment><USER32> In the context of web ads it's not insignificant.

<comment><USER17> If you can sway a US presidential election from 99% chance of losing, to winning, using only a $100k ad budget. Maybe you deserve to be in power...

<comment><USER32> I'd agree with you if this was a domestic actor and not a foreign actor that presumably wishes to do Americans harm.

</comment></comment></comment></comment><comment><USER33> Does anyone have access to the oft-cited FireEye report on this topic? Can't seem to find it...

</comment><comment><USER34> >Most of the ads focused on pumping politically divisive issues such as gun rights and immigration fears, as well as gay rights and racial discrimination.

When I read this I thought that this is what the US press does every single day wither it's WaPo or Limbaugh. It's all about selling fear and division. There's no longer nuance, no longer presenting both sides of an argument. It's all this is good or this is evil.

<comment><USER4> > When I read this I thought that this is what the US press does every single > day wither it's WaPo or Limbaugh

WaPo, and even Limbaugh, generally does not invent false flag news stories on both sides of divisive issues purely to drive division. WaPo may perhaps distort coverage to favor one side of divisive issues because of editorial biad or focus on existing divisive voices because conflict draws eyeballs, and Limbaugh may generate outright invented propaganda on a particular side of divisive issues to favor a preferred outcome, but neither of those behaviors are at issue, and even if it was the same behavior, neither actor is an outside interloper.

<comment><USER34> They do it for profit though. I don't see the motive as particularly relevant, the result is the same, division and fear.

Limbaugh certainly does. He said Sandra Fluke was a prostitute. I'm sure he's said many other things. WaPo and NYT are a little better, but their writers have gone a little bonkers over Trump hurting their credibility. I'm sure there are some Limbaugh like news sites on the left. There have to be, riling people up is very profitable. What about the DNC pushing Sanders off the ticket story? I mean take as step back and analyze all this stuff the press puts out. It's pretty divisive.

I'd really like to see a side by side comparison between Russian planted ads and comments and domestic ads and comments. Do you think we could tell the difference?

</comment></comment></comment><comment><USER35> When things go sideways, ain't it tempting to find an enemy to blame...</comment>


How To Create a Summer Internship Program at Your Startup - <AUTHOR> http://refer.ly/how_to__create_a_summer_internship_program_at_your_startup/c/7e9b6d0a5b6b11e2b5ab22000a1db8fa

<comment><USER3> This was a great read! Very information and well organized!

</comment><comment><USER4> We are also thinking about setting up an internship program for the summer and this is useful. Does anyone else have any other tips not mentioned?

</comment><comment><USER5> The link should go the article, surely?

<comment><AUTHOR> Just to clarify, the link goes to an original article written by me - <USER2> copy/pasted quotes from the resources listed. I have curated the links, added commentary and opinion and presented them as a blog post. The only difference is that instead of the links being text they are big beautiful images. Referly makes money when people buy products included in posts... anywhere you see the orange "buy" button we stand to make a small commission (usually around 5%).

The best way I can describe it right now is a community curated magazine with some social shopping features. Still a work in progress, but we're already helping more than 20,000 bloggers get paid for producing content on their areas of expertise.

<comment><USER5> Oh, sorry. I saw the paragraph and the "Show" and clicked through to the first one.

<comment><AUTHOR> No worries, I think we have a UX problem where it isn't clear what is original content and what are resources we are linking off to. I'll work on that</comment></comment></comment></comment>


Ask HN: I started a publishing company. What technical books are missing? - <AUTHOR> My name is Dave and I'm starting a publishing business. I've decided to narrow my focus on what I know...software engineering. I don't have a website yet, but I'm compiling a list topics/categories/books that I think are missing in the market but I'd like to hear from you if there's a book that you'd like to see.
Some topics I've come up with so far:
* Project based books. By the end of the book you will have built a complete application. Things like simple databases, operating systems, compilers/interpreters, shells, build your own framework, blockchain.
* Career focused books. Making the jump from junior to senior engineer, soft skills, navigating office politics, etc.
* Remote working - pros/cons of remote working, how to make the transition, etc
My plan is to identify compelling writers and programmers who specialize in these areas and work with them to publish books on these topics. I'd love to get any feedback on other topics you'd like to read about.
Thanks! <comment><USER2> To me, nowadays, the hardest part of learning a new technology or language is not the "thing" itself but the environment around it.

Make content about environments.

Example: I want to start doing Ruby. There are probably 49 ways to setup a decent working environment and hundreds of people putting a lot of tools together to get it started. They all conflict with each other and almost none are clear. Any material that can explain environments in a way that make sense and help you get started as efficient and completely as possible is gold.

This is true for all platforms today.

<comment><USER3> Problem with this is that this type of information gets old very fast.

</comment></comment><comment><USER4> IMO , focus on the project based books. I think this next decade will only see more self educated and meritocratically qualified folks who actively seek out accessible material akin in approach to how they learnt to programme.

IMO, focus on where the industry is going:

1\. Cloud infra

2\. ML.

3\. SPAs

4\. IoT

<comment><USER5> I second this. I'm self-educated, do a lot of freelance work and end up working on a huge variety of projects. While I feel like I've got a pretty good handle on languages and general programming, sometimes it's really difficult to find good quality information on the specifics of how to implement project x on platform y.

</comment></comment><comment><USER6> I always have trouble finding books that describe the "internals" of some technologies I want to learn in a decent and technical way. Some O'reilly books do a good job when they work with the actual people that invented the tools but there are many more out there that don't have good books on that.

<comment><AUTHOR> I would agree with this. I have a much better understand of how a database works after doing a small side project trying to implement one. I can imagine it would be helpful for other technologies as well such a docker, react, redis, etc.

Thanks for the input!

</comment></comment><comment><USER7> There's no satisfactory book on how to read substantial code. The result is that everyone is a writer and no one is a reader. Can you imagine a situation where almost anyone can write literary novels, but very few can read them?

<comment><AUTHOR> I've heard this a few other times in the past actually. Are you talking about strategies for getting familiar with a new code base? Or more general like debugging or code reviews?

Do you have examples of books on this topic that you're not satisfied with?

</comment></comment><comment><USER8> Good Project based books especially for topics like AI, scaling, architecture, hacking and other I can't think of right now are so scarce.</comment>


Should Data Science be taught from primary school? - <AUTHOR> https://mathsgee.com/should-data-science-be-taught-from-primary-school/ <comment><USER2> Def highschool, maybe middle school. Make fundamental statistics appear in middle school so high school transitions into data science putting concepts and methods to work.</comment>


Taking an Ancestry Test Could Make It Harder to Buy Life Insurance - <AUTHOR> http://cbslocal.com/2018/01/18/taking-ancestry-test-makes-buying-life-insurance-harder/ <comment><USER3> This was already happening in 1994. I brought this issue up to my squadron commander regarding DNA samples the military was attempting to gather from everyone. He said, "What, are you afraid we are going to clone you?" To which I said, "No sir, you should be afraid if you cloned me." and we both had a chuckle because we both know I was right.

I explained that insurance companies had already been denying claims for people laid up in the hospital based on "pre existing risk" that they might be at risk for getting cancer or becoming an alcoholic.

He had me submit a FOIA request which bought me some time. It went all the way to Al Gore. I received boxes of paperwork that essentially amounted to "The Chief Medical Officer can decide what to do with DNA information and how it is managed." That bought me enough time to hit my four years and get out without having to challenge them in court. I never gave them any samples and I do not regret my actions.

</comment><comment><USER4> (US perspective) I've wondered if for-profit insurance as we know it can survive changes in advances in actuarial science & related fields.

As I understand it, the entire concept of insurance is based around gathering relatively large groups of people with a solid understanding of their _average_ risks for certain types of loss. Given the averages hold accurate over time, those that need a payout can get one, and the insurance company makes money in aggregate.

However, the better the insurance companies can understand our risk profiles as _individuals_ (going well beyond age/sex/smoking/etc/etc/etc) - <USER2> (in theory) a motivated insurance company is going to be able to determine that I have a 90% chance of developing Serious Syndrome X while you have a 97% chance of developing Minor Issue Y.

Today we might get put into the risk pool. But as the science advances, the risk pools can be cut into finer and finer (and more accurate) risk pools. Why should a profit-seeking insurance company insure me past some point when the data strongly indicates i'm going to need drastic and expensive intervention.

It seems like the three ways to avoid this are: 1\. "Advances" like the one mentioned here don't turn out to have enough predictive power and insurance continues to operate much as before 2\. Some people just can't get for-profit insurance, at least not for health coverage (I'd think accidents will always remain much harder to predict than e.g. heart disease). Pushing them into some kind of not-for-profit 3\. Regulation forces blinders on to underwriters essentially saying "actuarial science advances to here and no further"

<comment><USER5> I don't think that's how risk pools work in the first place.

The idea is not to put people with similar levels of risk in the same pool. The idea is to make the pool as large as possible, charge everyone for their expected cost + a margin, and then pay out everyone for their actual costs.

It's the law of large numbers at work here. Take the mean value of a large number of variables with high variance and you end up with a mean that has low variance.

Not only is it normal for people with condition X to be in the same pool as condition Y, but this is a good thing because the costs for people with condition X are less correlated with the costs for people with condition Y, and this reduces risk in the pool. For example, if you have a pool that only contains people with condition X, and then it turns out that a commonly used medication prescribed for condition X has horrible side effects, then that single problem is disastrous for the entire pool.

> Why should a profit-seeking insurance company insure me past some point when > the data strongly indicates i'm going to need drastic and expensive > intervention.

Because you pass laws that make it illegal for insurance companies to deny coverage on this basis, and then you turn around and pass laws to keep the pool healthy enough for insurance to still be profitable. Or you just nationalize it.

<comment><USER6> > _The idea is not to put people with similar levels of risk in the same pool. > The idea is to make the pool as large as possible, charge everyone for their > expected cost + a margin..._

As a consumer who is fortunate enough to get life insurance at the "Super Preferred" low-risk rate, I would prefer be in a small pool that excludes people with higher actual costs. That's an economic preference, not moral, though my morals are not bothered by using the system as designed.

As long as insurance companies are free to offer plans that admit only really healthy people and exclude people with higher actual costs, my plan will be cheaper. Really healthy people will drop out of the larger pool and into the cheaper pool. The larger pool will, as a result, get more expensive. And the people remaining in what's effectively the shallow end of the pool will be offered a "Preferred" rate that's more expensive than super preferred but less than the large pool.

Eventually you end up with small risk pools, with the least insurable, highest-risk and highest actual cost customers unable to buy affordable life insurance. If you're a 20-year-old obese diabetic who smokes and has a family history of cancer, and the actuarial table says there's a 50% chance that your 50-year $1M policy will have to pay out, your premiums before a margin are 1M * 50% / 50 / 12 = $833/month. That's not really affordable, even if it would be wise for our unhealthy friend if he or she has dependents. My super- preferred policy costs me $21.25/month.

I empathize with the plight of the uninsurable. Given my voting history on health insurance issues, I would probably vote in favor of a law that limited segmentation and would bump my premiums to, say $50/mo and would offer life insurance to the $833/mo individual at the same rate. But I'm not going to jump from my pool into a more expensive pool by myself if that signal will fall on deaf ears.

<comment><USER7> "I sympathize with their problem but I won't disadvantage myself just to help them"

</comment></comment></comment></comment><comment><USER8> This is starting to be crazy. There is almost nothing you can do without having to worry about what some other party may do with your data. Walk in the street and have a video taken that's then processed and analyzed. Buy something and probably hundreds of companies will know about it.

In this particular case the insurance company should not be able to request data from the past. If they want any data they should order tests themselves and only use that data.

<comment><USER9> > This is starting to be crazy. There is almost nothing you can do without > having to worry about what some other party may do with your data. Walk in > the street and have a video taken that's then processed and analyzed. Buy > something and probably hundreds of companies will know about it.

I'm with you on this paragraph …

> In this particular case the insurance company should not be able to request > data from the past. If they want any data they should order tests themselves > and only use that data.

… but you lose me here. By this logic, it seems like car-insurance companies shouldn't be able to look at my driving history, but should have to administer me a driving test themselves. This would clearly make premiums skyrocket, and I don't think is the right solution.

<comment><USER10> Exactly. There is no way to make that work without ending up with a pile of absurd results, or results that are "right" but for legally contorted reasons.

The answer (to this narrow question[1]) is to not do business with shops like 23andme, who are horrible gossips. Find a service that actually respects you and your wishes if you want a test.

Admittedly this will become harder as testing becomes part of medical care. But telling your doctor not to tell the insurance company what's going on would never fly anyway.

[1] To the broader questions of medical privacy, well, burning the private insurance industry to the ground and chumming the PBMs would be a good start.

<comment><USER11> I think the actual correct answer is to legally ban services like 23andMe, or anything like them, from talking to insurance companies. Finding a service that actually respects you is exponentially more difficult than it sounds, as that relies on all the incidents already having been found out. Before this news, most people had no idea that this was happening.

</comment></comment></comment></comment><comment><USER12> From _The End of Asymmetric Information_ by Alex Tabarrok & Tyler Cowen:

The actual problems with health insurance markets have less to do with information asymmetry and adverse selection than with _too much information. That can make some people's insurance very expensive at actuarially sound rates._ For instance if you have a cancer of a given kind, this is verifiable to the outside world, and if the treatment costs are $200,000, the cost of an insurance policy will in turn be about $200,000. Buying the policy won't be cheaper than buying the treatments, and in that sense the market for insurance is not always present. That is a very real public policy problem … The cheap sequencing of the genome may accelerate and intensify these issues. …

[https://www.cato-unbound.org/2015/04/06/alex-tabarrok- tyler-...](https://www.cato-unbound.org/2015/04/06/alex-tabarrok-tyler- cowen/end-asymmetric-information)

</comment><comment><USER13> The whole point of insurance is to distribute risk. At some point, if coverage granularity becomes too high, insurance will just become pointless, right?

<comment><USER14> I think it's more subtle than that. It's also about symmetry of information. If you for example do 23&me you might find it that you have a much increased likelihood of getting a fatal disease soon. Thinking about your family's financial future you might decide to get life insurance because of this new information. Your individual risk in this case is very different from the average, you know this and that's why you are getting the insurance. This scenario if it became common also would break the insurance model because the insured pool suddenly wouldn't include people with average risk but tend towards higher risk. This would lead to higher cost for everyone who is insured. In an extreme scenario the cost might get so high that it only makes sense to get life insurance for people who know they are likely to die soon. Of course the likely scenario is less extreme, but this shows the principle nicely.

<comment><USER15> Very well put. Systematizing the algorithm, we have:

1) Get tested

2) Use the test results to determine the actuarial value of your hypothetical life insurance

3) If the price is less than the value, buy the insurance. If not, don't buy.

<comment><USER16> you could actually turn this into a business model

</comment></comment></comment></comment><comment><USER17> There's a simple anonymity hack, unless of course your relatives "out" you. I bought my 23andme kit from Amazon, then registered on 23andme website with a fake name. I have my results, but as far as 23andme are concerned I'm John Smith.

<comment><USER18> I was going to do this, but someone said that 23andme also takes a survey of family history, health, and demographic information—which is then used for actual research. I don't want to pollute data used for scientific research!

How recently did you do this, and was there any information collection like that?

<comment><USER17> I did this a few months ago, but I'm pretty sure the survey you mentioned is optional, at least I don't remember doing it, so you don't necessarily have to "pollute data".

<comment><USER18> Sweet!

And does the kit have any sort of identifier with it? If so, I was considering some sort of 23andme kit remixing service, where people pool funds to buy a bunch of kits, and then randomly select a kit from the shipment. :-)

<comment><USER17> Just get it from Amazon, the packaging doesn't have any unique identifiers like serial number, so if you register with fake name/email on 23andme - you're totally anonymous.

</comment></comment></comment></comment></comment><comment><USER19> > "If you apply for life insurance they do have a right to get all your > medical records. And if you've had a genetic test taken, they do have a > right to request it"

This data isn't something an individual has full control over. What if one of your relatives took a genetic test? If that relative was denied for a policy on genetic test grounds, can you be denied for the same reasons?

<comment><USER20> Probably, but why would _anyone_ in your family be disclosing their genetic profile when your family knows it has these sorts of things sitting in their DNA? I mean, if you're in a family where a lot of people get certain cancers, or congenital heart defects are common - well - exercise a little common sense and forego any DNA testing that Ancestry.com offers you.

<comment><USER21> Doesn't it seem unrealistic to you to rely on your extended family members not to disclose that information? Whatever you personally believe is "common sense", your family might not have it - or they might simply disagree with you.

</comment></comment></comment><comment><USER22> I think the underlying issue is real, but the headline seems inflated. Has this ever actually happened to anyone?

I recently bought life insurance and I'm pretty sure they didn't even bother to ask my primary care doctor for her records (though they had the right to), and they certainly didn't ask me about any tests I may have done on my own. (I think they did obtain at least some of my Rx history though. I assume from CVS Caremark.)

Right now I think there's kinda the opposite problem with Long Term Care insurance where people who take a genetic test that shows high risk for e.g. Alzheimer's run out and buy LTC insurance which messes up the risk pool.

<comment><USER8> They may nail you later because you didn't disclose something. I know people who have been dropped from their health insurance after years of paying because they forgot to disclose a previous injury. I should add they got dropped when they had a claim. That means the insurance happily took their money until it was time to pay up.

<comment><USER22> I'm pretty confident that I answered every question accurately and that no answers would have changed based on anything I could have learned from a genetic test.

Health insurers were really bad about "prior conditions" before Obamacare. I think life insurance has always been more tightly regulated.

FWIW, my policy says the insurer can only challenge the application within the first two years.

<comment><USER8> "Health insurers were really bad about "prior conditions" before Obamacare"

It's good to know that the current administration is determined to take us back to those glory days :)

</comment></comment></comment></comment><comment><USER23> Also, before you send your DNA off... Keep in mind that these companies are not legally allowed to delete your data.

[https://www.bloomberg.com/news/articles/2018-06-15/deleting-...](https://www.bloomberg.com/news/articles/2018-06-15/deleting- your-online-dna-data-is-brutally-difficult)

<comment><USER24> FTA: "It's a lesson we're destined to keep learning: Once you share something online, you can't really ever unshare it."

Your genetic info is sort of like a trade secret in the USA, but without criminal sanctions when it's wrongfully appropriated.

</comment></comment><comment><USER25> Wasn't there a novel, like 20 years ago, about insurance company having to hide massive profits from using DNA profiling? Does anybody remember its title? It looks like real life has catched up with fiction.

</comment><comment><USER26> Could one poison the records by sending multiple samples of different blood sources attached to the same name? So the database query would'nt be conclusive, but _you_ would know which test is legit.

How would the ancestry testers react if I sent several samples in my name?

Will there be a market for clean blood that you can use to confuse the records and get a sort of plausible deniability of identity?

</comment><comment><USER27> The article does obliquely reference the Genetic Information Nondiscrimination Act (GINA), but then says it can be changed. This law was passed in 2008, for the most part before the rise of consumer ancestry tests. It seems unlikely to me that a major rollback of protections on these data is going to be easy to pass.

<comment><USER28> [https://en.wikipedia.org/wiki/Genetic_Information_Nondiscrim...](https://en.wikipedia.org/wiki/Genetic_Information_Nondiscrimination_Act)

Interesting, I'd like to read more commentary on this act. Anyone?

</comment></comment><comment><USER29> As a person who isn't old, lives below his means, and doesn't have children, I don't see the problem here. From my perspective, life insurance is nothing but a big scam. I don't anticipate dying any time soon, and if I do, I don't have any younger heirs. If I want to build wealth, I can do a lot better with index funds and mutual funds.

Honestly, I just don't see the point of life insurance at all, unless someone has a family with children and doesn't have any significant wealth accumulated.

<comment><USER16> I think most people have a very minimal amount of capital accumulated. Life insurance exists to help their family not starve if they unexpectedly die. Its opt in which totally makes sense to me. I pay for it because it costs basically nothing for me as a healthy young person and it would provide a nice windfall for my girlfriend if for example my plane fell out of the sky.

<comment><USER30> We have survivors' benefits from Social Security in the US. The logical thing to do, from my perspective, would be to make the existing government program designed to insure everyone cover people sufficiently. Given that baseline, the market for life insurance doesn't matter so much.

</comment></comment></comment><comment><USER31> The upsetting thing with this whole "paying for the privilege of giving someone else your DNA" trend, is that even if you are opposed to it, relatives of yours essentially give your data too. For example, apparently, both my mother and father have already sent off their DNA to these various ancestry companies, which by extension means that I'm also in there, albeit slightly detached.

<comment><USER32> Sharing genes with other people has indeed been a source of headaches since the beginning of time (take, for example, the whole Garden of Eden fiasco). Luckily, the advantages of being inside the genetic group, like say inheritance, usually outweigh the nuisance.

</comment></comment><comment><USER33> I literally got my DNA run now because I have real concern that in the future if you ever get it sequenced you will be legally required to share it with insurance companies, law enforcement, and many others. I'm hoping that I'll be able to get grandfathered in and not share mine.

</comment><comment><USER34> Important point from teh article that about half of the posts here seem to have missed - this is _not_ about health insurance, but life insurance.

"while health insurers may not use or consider genetic test results when selling a policy, companies that sell life insurance, can."

</comment><comment><USER35> Out of curiosity, how hard would it be to make an open-source, home ancestry test? Assuming that all the necessary data is there, which equipment would you need to sequence your DNA?

<comment><USER36> Start in the high 4 digits. If you're lucky.

</comment></comment><comment><USER37> This just makes me want to watch Gattaca again</comment>


Ask HN: Big Data case studies - <AUTHOR>

Anyone have good, specific examples or anecdotes of genuinely big data and the architectures you used to analyze it?  (And/or counterexamples of 'big data'...) <comment><USER2> Define "genuinely big data"

<comment><AUTHOR> That's what I'd like to get at with some examples...</comment></comment>


In changing economy, two posts illustrate why some succeed and others fail - <AUTHOR> http://www.ojr.org/ojr/people/robert/201102/1937/ <comment><AUTHOR> This post is specifically about the media industry and being a freelancer, but I think the mindset illustrated (creative ideas about what works is and the path to success) is incredibly important to finding fulfilling, well- compensated work anywhere these days, whether you're an entrepreneurial type or not.</comment>


BloomJoin: Fast asymmetric Hadoop joins using Bloom Filters and Cascading - <AUTHOR> http://blog.liveramp.com/2013/04/03/bloomjoin-bloomfilter-cogroup/

<comment><USER2> For a nice exposition on how to do some really neat distributed joins that make even more interesting use of hashing / bloom filters, this exposition by Edward Kmett is a nice read: <http://comonad.com/reader/2008/linear-bloom- filters/>

</comment><comment><USER3> The title scares me...

<comment><AUTHOR> Heh, too many buzzwords? I was trying to think of a simpler title, but didn't make too much progress...</comment></comment>


What is your take on checking return values? - john-shaffer https://rachelbythebay.com/w/2020/09/02/check/ <comment><AUTHOR> > They actually said that to me: it makes the code messy so they don't want to > do it.

I'm going to give one possible interpretation (my own) of the sentiment I believe the engineers were trying to get across with the statement "it makes the code messy".

Functionality is an asset, and code is a liability. Minimizing the amount of code and making the code "less messy" is a way of reducing that liability. That doesn't excuse not checking for certain error conditions, but I think it's important to point out that non "messy code" has merit on its own.

On the topic of effective error handling, the number of possible invalid states in a program (especially dealing with the messy real world) tends to be combinatoric. Error checking is vital to building a successful application, but my experience is that in practice it quickly becomes impractical to have _total_ potential error coverage. At some point tradeoffs need to be made, and effective engineering is the art of evaluating those tradeoffs.

> "And yet, here we are, because it DID happen."

I'm not trying to imply the author may be suffering from hindsight bias, I don't have any information that could tell me that. I do know that in similar situations I've been in in the past I feel like I've suffered from hindsight bias significantly. Here's a great article on hindsight bias that I think about often. [https://www.lesswrong.com/posts/fkM9XsNvXdYH6PPAx/hindsight-...](https://www.lesswrong.com/posts/fkM9XsNvXdYH6PPAx/hindsight- bias)

The pivotal section from the lesswrong 2.0 article for me

> Viewing history through the lens of hindsight, we vastly underestimate the > cost of effective safety precautions. In 1986, the Challenger exploded for > reasons traced to an O-ring losing flexibility at low temperature. There > were warning signs of a problem with the O-rings. But preventing the > Challenger disaster would have required, not attending to the problem with > the O-rings, but attending to every warning sign which seemed as severe as > the O-ring problem, without benefit of hindsight. It could have been done, > but it would have required a general policy much more expensive than just > fixing the O-Rings.

</comment><comment><USER2> Tony Hoare, famously, called null references a "billion dollar mistake"[0]. Much preferred to checking return values is to use a language which models null references in the type system doesn't compile code that would try to use a null value as if it weren't null.

Ok, fine, but for whatever reason, some of us don't have the option of working in such a language. What to do then? The author seems to think it's obvious that the answer is you check for null in any function that could possibly receive a null value. This means you have to check for null for every reference type in your system, which is madness.

I agree with the programmers who the author thinks "Do Not Get It". When you have a null reference bug, you fix the bug where the reference is generated, not where it's used. If you fix it at the place where it's used, you've barely fixed anything, because the place where it's generated will continue to send nulls to unsuspecting callers who will blow up when they assume it's non-null.

It sounds to me like the fix here is to write a validation routine for the configuration to ensure that it doesn't vend null values to the rest of the program when it shouldn't. The service can validate its config at load time and fail to start up if it's invalid. This will block your deployments, but your customers won't be impacted and your code won't be littered with null checks that fail to address the root cause.

[0] [https://www.infoq.com/presentations/Null-References-The- Bill...](https://www.infoq.com/presentations/Null-References-The-Billion- Dollar-Mistake-Tony-Hoare/)

</comment><comment><USER3> Definitely agree that you should have validation if you can't guarantee the validity of your data, but I think there's a difference here.

Coincidentally, I had a discussion about a similar topic today. Basically how much validation is necessary and at what level should it be at?

      function add(a, b)
        if a is not a number
          throw error
        if b is not a number
          throw error
        result = a + b
        if result > max possible precise value
          throw error
        etc...

I would never have these checks in a function that just added numbers if I could guarantee that any values passed to it are correct.

The team in the article said that the config shouldn't return null so they didn't put checks before using that value. To me it seems like they missed __necessary __validation logic because they didn 't the realize the full extend of the config.

</comment><comment><USER4> Why are we talking about checking return values (which are generated by a function according to its interface and should not be checked) when we should be talking about checking configuration values (which are generated by humans according to whatever they type on their keyboard and should be checked). Also why are we talking about checking stuff when we should be talking about one bug bringing the entire system down?

</comment><comment><USER5> Be sure that your error-handling code checks for errors.

      while(printf("Error!\n")+1 != sizeof "Error!\n")
        ;

Also, it should be bug-free.

<comment><USER5> No, wait, like this:

      void got_error(void){
        char err[] = "Error!\n";
        int len = strlen(err);
        int ret = printf(err);
        if(len != ret)
          got_error();
      }

Obviously, if the error routine faces an error, it should call the error routine. That is how you handle errors.

Er, but maybe the error routine should also return an error flag in that case? The caller could then check it and call the error routine if needed.

</comment></comment><comment><USER6> I'm an amateur developer and I write checking code in functions all the time because you can't be certain of the source of the data!!!

</comment><comment><USER7> RobustProgramming.ps[0] is a great presentation by Stratus Computers about error handling for writing reliable systems.

I posted it to HN[1] a few years ago.

[0] ftp://ftp.stratus.com/vos/doc/papers/RobustProgramming.ps

[1] [https://news.ycombinator.com/item?id=13678251](https://news.ycombinator.com/item?id=13678251)

</comment><comment><USER8> Could be the glibc authors. Why should one ever check for NULL or out of boundary conditions or overlaps or overflows when the spec forbids it?</comment>


In Just 5 Moves, Grandmaster Loses and Leaves Chess World Aghast - <AUTHOR> http://www.npr.org/sections/thetwo-way/2017/02/03/513259703/in-just-5-moves-grandmaster-loses-and-leaves-chess-world-aghast <comment><USER2> There are pretty strong rules about how the draw on FIDE tournaments can be done and the programs that make them have to be certified. In one forum someone said they actually checked:

\----

I see a lot of remarks saying "well, let them show the machine, so someone can check it", "It must be manipulated, this is so unlikely", etc.

The FIDE Swiss Dutch rules are on the FIDE website, in the handbook. There is pairings.fide.com which has a list of endorsed pairing software, meaning it was tested by FIDE to follow those rules. Why is nobody doing the checks?

Guess what? I did :)

Took the SwissManager tournament file from chess-results.com, created a TRF / FIDE rating report file, imported it, verified the pairings.

      Round 1: differences, which is to be expected:
        people show up late, ratings get corrected, mistakes fixed, etc.
      Round 2, 3, 4: equal to the pairing in Gibraltar
      Round 5: a few differences in the group of people with 1.5 and 1 out of 4, nowhere near Hou.
        My educated guess: results of previous rounds were corrected after round 5 was paired
      Round 6, 7, 8: equal to pairing in Gibraltar
      Round 9: in the lower echelons 2 pairings were adjusted (the black players exchanged),
        due to (probably) Israeli not playing Iranian
      Round 10: equal to pairing in Gibraltar.

Does that count as sticking to the facts?

<comment><USER3> If people agree that the software was correct but that the pairings were poor, there is always the possibility of reevaluating the algorithm itself. For example, an algorithm could be chosen that results in more "mixing."

<comment><USER4> You can only optimize an algorithm in so many dimensions. Without making this a debate about affirmative action, I'd like to point out that if you're designing an algorithm to optimize for raw skill comparison in tournament match-ups, optimizing it to rebalance match-up results to be more mixed essentially voids that first optimization. In the aggregate, you're producing very different results by doing that. To put it simply, would you rather have skill-based fairness in a tournament or gender-based fairness?

You can design the algorithm in such a way that your priors are mistaken, or bias creeps in (though _theoretically_ that can be self-corrected). But assuming that's not the case, and the algorithm correctly matches up mostly women v. women and men v. men in a skill parity optimization, you have a fair result for the purpose of a tournemant; i.e. _skill-based match-ups._

If at that point you have an issue with the match ups for reasons of gender or ethnic parity, I would argue that you should seek to correct the upstream issues, not the algorithm. In other words, try to get more women playing chess - make the sport appeal to them more, make it more inclusive, etc. Rebalancing an algorithm is, in my view, a handicap, whether it's applied to gender disparity or any other disparity. I feel it does a disservice to both parties and doesn't really solve the root issue.

<comment><USER3> There can be a balance. One could introduce more mixing while still relying primarily on the current metrics. It's not a binary choice between one or the other.

<comment><USER5> It's actually a binary choice between a gender-biased algorithm and a gender- neutral algorithm. A small bias is still a bias.

<comment><USER6> I don't fundamentally disagree with you, but I note that when you say "small bias," you're acknowledging that there is a continuum between total neutrality and total bias.

<comment><USER7> I don't see how you arrived at the conclusion that the cardinality of biases is greater than the cardinality of natural numbers from the proposition(s) of the post you replied to. If that was the intent of your post, please elaborate.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> There's some good discussion in this reddit thread: [https://www.reddit.com/r/chess/comments/5rm88g/hou_yifan_res...](https://www.reddit.com/r/chess/comments/5rm88g/hou_yifan_resigns_after_5_moves_in_the_2017).

As someone who does statistics for a living and used to play tournament chess, a few of my disorganized opinions/thoughts:

* There's no conclusive evidence that her pairings were tampered with, and the pairings are in line with what seeds someone would be given by a computer. The probability of her playing that many women is very low, but the whole point of randomness is rare events happen (as humans we're very good at detecting patterns in randomness). Her accusations are plausible but not a given.

* I think her goal to break down the barriers of gendered chess is great. The chess world needs more women, and if there were more they would be completely competitive with men. She's fighting a good fight.

* One of the reasons I stopped playing chess is the egos. This one was relatively mild, but purposefully losing games is wrong, even if done in protest. Once you sit down at the chessboard and shake the other person's hand, you're agreeing to a good game. Throwing a game against a woman because you're grumpy she's a woman is not a good thing.

* But, her activist tactic worked, here we are talking about it.

<comment><USER9> > Throwing a game against a woman because you're grumpy she's a woman is not a > good thing.

Not to be pedantic, but Lalith Babu, the Indian Grand Master, is male. I believe the article says Hou Yifan threw the final match against Lalith Babu. She played all the women pairings till she got there.

Not to mention how the winner of that match would feel with such a terrible victory.

<comment><USER8> Not pedantic at all, thanks for the correction!

</comment></comment></comment><comment><USER9> > Hou, who currently outpaces the second-ranked female player by 68 points, > recently left the women's chess circuit for mixed events where she can > compete against men, who fill every spot in the world's top 100 rankings.

I was curious if the women player's ratings were separated from the Top 100 players rankings and thought that the overall FIDE Top 100 ratings was gender differentiated.

On a perusal of the FIDE rankings over the years I noticed that that is not the case.

Here are the stats for this year (Feb 2017):

\</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>--

Top 100 Players:

Rank; Name; FIDE Rating

1; Carlsen, Magnus; 2838

...

100; Artemlev, Vladislav; 2655

101; Cordova, Emilio; 2655

Top 100 Women Players:

Rank; Name; FIDE Rating

1; Hou, Yifan; 2651

Here are the stats for the earliest reported year (2000):

\</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>---

Top 100 Players:

Rank; Name; FIDE Rating

1; Kasparov, Garry; 2849

...

32; Polgar, Judith; 2656

...

100; Fominyih, Alexander; 2594

Top 100 Women Players:

Rank; Name; FIDE Rating

1; Polgar, Judith; 2656

So even though there is a separate Women Players ranking provided, the Top 100 Players is not gender differentiated.

It just happens that this year, Hou, Yifan, the current top women player is ranked lower than the bottom of the Top 100 players ranking hence the top 100 payers (this year) is all men.

What does that mean? Does that mean that women players do not get a chance to play among men? Are there less opportunities for mixed events?

If so, yes that should definitely be changed.

</comment><comment><USER10> Why are tournaments (or rankings; it's hard to tell from the article) sexed in the first place?

<comment><USER11> The major titles, tournaments, and ratings aren't gendered and haven't been for a long time. That said, it is still the case that today's active male players tend to be overwhelmingly better than active female players at chess, so separate tournaments and titles were specifically created just for women. No women are restricted from the "main" tournaments or the "main" titles, though. It is entirely possible for a woman to dethrone Magnus Carlsen as World Chess Champion -- it's just never happened before.

As for _why_ this discrepancy exists, that's really a very different question, and it's one that I don't know the answer to. There are potentially _many_ factors at play. It could be as innocent as there are many more male players than female, so the odds are much higher for super-high-skill players to emerge in the male population than the female population.

<comment><USER12> >As for why this discrepancy exists, that's really a very different question, and it's one that I don't know the answer to.

That's caused by IQ distribution differences between males and females. On average men and women have an IQ of 100, but there are more men that are both smarter and dumber than women.

[https://en.wikipedia.org/wiki/Sex_differences_in_intelligenc...](https://en.wikipedia.org/wiki/Sex_differences_in_intelligence)

<comment><USER13> According to the article you linked, research findings, including those on variability between sexes, are all over the map. This does not at all seem like a convincing explanation to me.

</comment></comment></comment></comment><comment><USER14> It would be good to know the chances for that. I'm guessing most of the other participants at each round were men, so they must have been pretty low...

</comment><comment><USER15> Misleading headline: she threw the match to protest against the tournament setup, having to play against too many women.

</comment><comment><USER16> I don't follow chess, and I didn't know it's gendered. That doesn't make sense. Why would chess contests be gendered? That's like a gendered photography contest.

<comment><USER17> It's not strictly gendered - women are free to enter any tournament and hold any title. There are a number of women-only tournaments and titles, purely to improve the visibility of women in the game. Without women-only tournaments, you'd see very few women at major events.

Only one woman has ever qualified for the world championships (Judit Polgar in 2005). Hou Yifan is the best female player in the world, but she's ranked 105th overall. The second highest ranked female player is 303rd.

Gendered tournaments aren't ideal, but the alternative is worse. Most countries do a dismal job of promoting female participation in chess, with China being a notable exception. Western women just aren't being encouraged to take up the game and aren't made to feel welcome when they try.

</comment></comment><comment><USER18> I was just reading a book about this, "The undoing project". The point of the book was that statically even distribution only works in large numbers, in small numbers its normal to get uneven distributions like this. That's why sample size is so important in scientific research.

So, for example, when have a group of 10 men and 10 women, and you randomly pair them up in a tournament, it is normal and possible for a woman to be playing against 5 women in a row. It's not an aberration because small numbers won't necessarily even out. However, when you get a group of 1000 men and 1000 women, and start paring them up for tournament matches, it would still be possible for a woman to end up playing against 5 woman in a row, but on the whole when taking all 500 matches she plays, it should approach 50, the aberration would be if she had played against 500 women in a row. That would be an indication of something suspicious.

This is an example of the gambler's fallacy.

[https://onlinecourses.science.psu.edu/stat100/node/46](https://onlinecourses.science.psu.edu/stat100/node/46)

</comment><comment><USER19> I don't understand how it was a loss?

[http://www.chessgames.com/perl/chessgame?gid=1860931](http://www.chessgames.com/perl/chessgame?gid=1860931)

<comment><USER20> She resigned

</comment></comment><comment><USER21> What I find interesting and am curious about people's thoughts on here is that while Yifan could have waited until she was able to run the pairing algorithm herself (as someone did see the comment [https://news.ycombinator.com/item?id=13568108](https://news.ycombinator.com/item?id=13568108)), her protest could be no less valid even though it's based on apparently inaccurate information.

</comment><comment><USER22> Question: isn't the drawing of opponents dependent on relative ratings?

If so then it probably makes sense that more of one population would match with each other, assuming there are significant statistical differences in the populations. For Chess, this is most certainly the case (top 100 is almost all male iirc).

<comment><USER23> She's the top ranked woman. If that was the case, she'd be playing mostly against men.

<comment><USER22> Regardless, the only way to know if it was a statistically fair setup is to have: 1) the rankings of all playings in the tournament, and 2) the algorithm used to match players.

If 2) matches players of similar ratings against eachother, and she is more similar in rating to women than men, then I don't see what the fuss is all about.

</comment></comment></comment><comment><USER24> Ignoring the match itself, the commenters seem incredibly sexist. They repeatedly ignored/interrupted a woman saying she'd actually talked to the player to comment on whether or not the player was 'mental' or 'still drunk' or otherwise.

<comment><USER25> What you describe is being rude and bad-mannered. Being rude to a woman is not automatically being sexist.

<comment><USER26> Talking over a woman who is speaking for herself, and discussing whether she is competent without her input, is a classically sexist way to be rude.

<comment><USER25> No, it's not, someone is simply rude and talks over everyone. A rude person will talk over men, women (without being sexist), black people (without being racist), etc.

Either you demonstrate that the arrogance is specifically targeted towards women and because they are women, or you cannot speak of sexism.

</comment></comment></comment></comment><comment><USER27> So she was nowhere near as good as the men who participated in the tournament and rage quit because she had to play against women? Isn't that pretty sexist on her part?

</comment><comment><USER28> What additional value does the ranked pool bring compared to random pool for pairing.

Game pairing in Go for example is usually a random pool (bingo; pick your number from a bowl), after that it is straightforward knockout.

<comment><USER29> From loosely following chess over the past year it seems they normally play a random pool and whoever has the best score at the end wins the tournament.

<comment><USER2> No that isn't the case. There are really only two common types of tournaments. A round-robin will see each player play every other player once but this is limited to small fields (10 is the usual limit).

For big fields like the one the article is about the Swiss System is used. Each player gets 1 for a win, half a point for a draw and zero for a loss. Players are mostly paired against those who are on the same score each round. I say "mostly" since there are all sorts of rules to decide the exact pairings. Chess tournaments will use Dutch-Swiss or sometimes Dubov-Swiss

Lots more detail here:

[https://www.fide.com/fide/handbook.html?id=18&view=category](https://www.fide.com/fide/handbook.html?id=18&view=category)

</comment></comment></comment><comment><USER30> Maybe they need to consider the idea of better-than-random pairings. What this means would be a subject for discussion, but for example there could be heuristics that say something like pairings need to at least look random. What does that mean? Again, it's a matter for discussion, but just to take the Yifan situation as the example, seven out of nine paired with the same gender does not look random. Just saying "it's decided by machines" is not sufficient. They have to say what the rules are that the machines are going by, and what is the source of randomness exactly. And then they have to address the point that maybe pure randomness is not good enough.

</comment><comment><USER31> She resigned? Them why make 5 moves?

I would think that if opponent was throwing the game I'd double down and try to throw it better than her. So the chess match became a chess match. Who can lose fastest. Then resigning is cowardly.

</comment><comment><USER32> Why the hell does chess need to be divided by sexes? Ridiculous!

Kudos to her for competing with the best.

</comment><comment><USER33> [whatever - i'll go somewhere else]

<comment><USER34> I am a man, nothing human is alien to me.

<comment><USER35> ^^ Poe's law.

</comment></comment></comment><comment><USER36> I really didn't like the comments of the tournament GM as quoted in this article.

> I understand: If I was in her shoes, and I suddenly pulled a draw of six > girls one after the other, I would say also, 'What is going on here?'

Maybe in the speaker's culture, using the diminutive term "girls" to refer to professional women in a professional context is considered fine, but it's definitely grating to me. I'd be willing to give that a pass if that were all. But he also said:

> I'm sorry for Yifan, because I think she let herself down a little bit > today.

I really think Yifan is in a better position to judge whether she has let herself down than Callaghan is. Together these two comments read, to me, as quite paternalistic and tone deaf. If this is how he's speaking on the record, one has to wonder if his private attitude contributed to Yifan's decision to throw the game.

<comment><USER37> If he's not allowed to say that he thinks she let herself down, then you're not allowed to say you think he's coming off as tone deaf, right? I think that because there's an issue of gender here, we are scrutinizing every word far too much.

I agree with him. One thing I learned when growing up playing sports is that you always do your best when playing. Throwing a fit or protesting by giving up only lets yourself down. Finish the game, and then go protest.

<comment><USER36> He's allowed to say whatever he wants, and I'm allowed to feel how I do about it, and discuss those feelings on this site.

<comment><USER35> But apparently not without being cowardly downvoted for explaining how freedom of speech works.</comment></comment></comment></comment>