Ask HN: largest number you've had to store in an application? - <AUTHOR>

For a "real" purpose (as opposed to a user banging on the keyboard).  Just curious when / if / why we'll need more than 64 bits. <comment><USER2> I used to work for an egg wholesaler. We paid the farmers by the egg according to grade. We accounted in eggs.

Do you know how many eggs are sold in the UK?

We had to write a special class to handle eggs because we blew the unsigned integer on a VAX.

<comment><USER3> Bignums, my friend. BCD is also good for real-world numbers, like money or eggs.

</comment></comment><comment><USER4> You don't have much space left in 64 bits for some financial numbers. FX swap trades with a billion dollars on one side and some very devalued currency on the other (e.g. zimbabwe dollars 1:60,000,000 at one point).

It gets worse when you start adding those numbers up.

</comment><comment><USER5> Heh, I actually overran a signed integer 2 years ago in an app for a major financial company.

Yes, that's > $2,147,483,647

Thanks, subprime mortgage crisis!

<comment><USER3> <http://www.ddj.com/java/184405653>

<comment><USER5> Thanks, but we weren't tracking cents. Nor using Java.

</comment></comment></comment><comment><USER6> Huge. Longer than 2^30 digits. I was working on statistics. Doing probabilities requires doing some crazy factorials. These often require handling large freaking numbers. I'm a PHP guy, so BC Math and GMP Math are my saviors in this area.

Thankfully, all the huge numbers are just in the computation of the statistics and don't need to be stored long term in a database. The thought of having to store an integer in a BLOB is frightening.

<comment><USER7> I stumbled across a database where I work where videos were being stored not in the filesystem, but as BLOBs in a DB. Thrilling.

</comment></comment><comment><USER8> If you want universal uniqueness you should be looking at 128-bits (<http://en.wikipedia.org/wiki/Universally_Unique_Identifier>)

<comment><USER9> If you want universal uniqueness, 128 bit won't last you too long. The number of atoms in the universe is estimated to be around 10^80, so you'd need about 266 bits to address every single atom. Give it some headroom, go to 512 bits...

</comment></comment><comment><USER10> I built an aviation mapping engine that used fixed point math for lat/lon representation. It basically used an entire 32bit signed int and gave a guaranteed minimal resolution of 3 feet. It would quite often go to the ragged edge of what you can fit in a 32 bit quantity.

The fun bit was that you had to use 2 combined registers (ARM) to represent a 64bit quantity so you could do multiplication and keep all your precision.

Oh, for the luxury of a FPU...

</comment><comment><USER11> public/private keys for RSA - 1024 bit numbers.

</comment><comment><USER12> factorial(30..50) -- for factoradics (<http://en.wikipedia.org/wiki/Factoradic>). Representing the reproducable state of a list of randomly ordered items.

</comment><comment><USER13> I implemented Tupper's Self-referential formula which depends on an extremely large number (although I suppose that's cheating a bit, the number is actually an encoding of a bitmap).

</comment><comment><USER14> I've had to build a system to theoretically handle indexes up to 1.2553643905927429e+30 but we haven't got that far yet (biggest # is a billion billion billion or so)

</comment><comment><USER15> 1000000! took me about an hour to calculate on a supercomputer in 1988. It's roughly 5 million decimal digits.

</comment><comment><USER16> I have worked on Financial Apps where we have ended up blowing away the 64-bit threshold.

</comment><comment><USER17> I guess astronomical modeling uses ... astronomically big numbers?

</comment><comment><USER18> EPC Gen2 RFID tags have a 96 bit unique id that we store

</comment><comment><USER19> Do parts of RSA keys count?

</comment><comment><USER20> twitter id's is the first thing that comes to mind.

<comment><USER21> These are also the largest numbers I've had to store, however 64 bits should suffice for a very very long time for twitter id's. If everyone on earth (assuming 6 billion people) tweeted 100 times per day, it would take over 84 million _years_ to overflow 64 bits.

<comment><USER22> But if you had 84 million inhabited planets, with 10 tweets a day, it would only take 10 years (correct me if my math is wrong).

</comment></comment></comment><comment><USER23> 8374874823748327491</comment>


Facebook bug hunter paid $10K by community, not company - <AUTHOR> http://www.zdnet.com/facebook-bug-hunter-paid-10k-by-community-not-company-7000019652/

<comment><USER2> I'm happy for the guy and all, but Facebook is doing the right thing by not paying the bounty. They specifically bar messing with real user data, while they could have handled the original report better that's still not justification. If you bend the rule once, everyone will want you to bend the rule "just one more time" for them.

<comment><USER3> I haven't been paying close attention to this, so forgive the ignorance if I'm wrong: didn't he try to use the correct channels, but was turned away?

<comment><USER4> The big idea as I understand it is:

He proved the bug on a live account instead of one of the prescribed test accounts

He reported it as "this is a bug" not as "this is the procedure to use in order to reproduce the bug"

He demonstrated critical inability to report the steps required to reproduce the bug.

It must get to the point where this money faucet receives hundreds of people submitting "bugs" where their friend has really left their account logged in and it's been exploited via social engineering, or some other "not a bug", so bugs are closed when they're submitted with not enough information.

I understand and subscribe to this strategy myself.

<comment><USER5> You missed the part where FB -- admit to and are fixing -- lack any process to deal with those that are either not aware or don't understand the process. At no point do they instruct the reporter that it'd be helpful to provide more steps and that he should not be using live accounts to demonstrate the PoC. Nay, they merely shrugged him off and stated his actions were "not a bug."

<comment><USER4> You know, I went looking for help on something this morning (why does ntop crash so much?) and at first all I could find was a post asking a similar question, and a reply about how to ask the question better.

The GP replied (paraphrased) "there are a lot of posts on this board asking for help and 95% of the replies are about how to ask the question better. You would do well to try and read between the lines a little bit, just try interpreting and answering some questions instead of just posting all about how poorly the questions are asked."

The point is, that guy wasn't offering money. He was supporting free software. I agree it's easy to make a blanket response for posts not providing enough information, but if they can't fill out the form correctly enough for the reviewer to duplicate the result, why should they get the money?

Just to close the loop, I did find another post eventually where the author explains he "is aware" that ntop crashes frequently when it's configured to monitor multiple interfaces, and you should use the SVN builds since they are more robust. The bug still exists in FreeBSD and I update my ports tree every day.

Searching for "ntop quits" again to put some dates on these posts, I see the complaint was in 2003 and the post addressing my issue was in 2011. I guess it still hasn't been fixed, but the software is still well-known. Anecdotes are like...

</comment></comment></comment></comment></comment><comment><USER6> Not a bad pay day for a critical bug. Forget the black market, hypothetically speaking I wonder how much an ad agency would have paid for something like this if they can use/abuse it for a week before fb catching on (assuming the agency is unscrupulous).

<comment><USER7> Imagine... being able to post ads on anyone's timeline!

Now THAT is something you could make profit with!

</comment></comment><comment><USER8> Did anyone here donate to this guy? If so, can you explain why you did it?

<comment><USER9> I didn't, but I suppose many have facebook accounts and would prefer people like him keep giving information about exploits to facebook instead of selling the information to the highest bidder as a "zero day" or just leaving it unfixed so someone less scrupulous can find it and use/sell it for nefarious means.

It could also be people who like to see facebook and/or its figure head with egg on face, who want to encourage this fellow (and others) to mess with Zuckerberg's profile again for that entertainment value.

<comment><USER10> I also didn't, but some of the articles on the subject paint "the guy" as a down-on-his-luck hacker in developing country and ancient computer hardware trying to do the right thing. It's presented as almost a charity donation or a scholarship to someone in less than affluent circumstances.</comment></comment></comment>


Show HN: Present – A terminal presentation tool with colors and effects - <AUTHOR> https://github.com/vinayak-mehta/present <comment><USER2> That's a very nice tool! I think it should really make it easy for programmers to make presentations with inline code blocks. `reveal.js` is really a blessing in that regards!</comment>


Devastating startup customer support mistakes - <AUTHOR> http://www.peopledelight.com/recent/startup-guide-customer-support/ <comment><AUTHOR> Curious if folks agree with our conclusions here...</comment>


A few people develop a compulsive urge to crack jokes - <AUTHOR> http://www.bbc.com/future/story/20160308-the-curse-of-the-people-who-cant-stop-making-puns <comment><USER2> I make a lot of jokes and incongruous juxtapositions and things like that, I hope not anywhere close to 'disinhibition' mentioned in the paper. But it makes me wonder how much laughing at a joke is appreciation of a witty idea vs how much of it is a social contract where you say something mildly interesting, laugh, and other conversation participants laugh.

It's still a genuine social exchange, but the point is that humor has empathy as a key component, and it helps us bond. If you were to convey the same concept in a tone of anger or disappointment, most listeners won't laugh. So maybe humor and rants are two sides of the same coin: socialized salience.

But like any other social contract, it can be abused. If you frame a sentence in the cadence of a joke or rant, you're expecting the listener to respond. If you rant for a very long time, people stop listening. If you make jokes 24/7, the social activity of joke-telling is no longer a two-way street, and people stop laughing. If you deliver nonsense like a joke, you may get the knee-jerk social laugh, but not the witty-idea laugh (unless nonsense _is_ the witty idea). If it's inappropriate to bond in a situation, and you tell a joke, you might not get a laugh, let alone an acknowledgment.

For various reasons, patients described in the article only have pieces of this whole system, and struggle with the social aspect or the witty-idea aspect.

<comment><USER3> I would say there's a definite social component. I have some ability to speak Spanish, and work with several Spanish speaking coworkers. Sometimes I completely miss the joke, but laugh anyways, because everyone else is.

</comment></comment><comment><USER4> I once sat next to a businessman whose only conversation was making joke after joke during an entire dinner. They were not his own, just regular old jokes. He'd built a business worth 300 million euro over thirty years and he literally told me jokes for two and a half hours. By the end I struggled to even smile let alone laugh. He didn't appear to notice.

Perhaps this explains it.

<comment><USER5> I suspect he believes the jokes are good because people pay attention to him because of the $300M, not because of the jokes but it's hard for him to tell the difference from where he's sitting.

<comment><USER4> The funny thing is until you said this it never occurred to me for a moment. My impression was much more of someone that couldn't help themselves. More as if he was terrified of even a moment's silence.

</comment></comment></comment><comment><USER6> I passed this on to a colleague whose jokes are both constant and terrible (though far from the condition described).

As I said to him, I'd be worried about myself ... but thankfully my jokes are always funny.

It's an interesting insight into what sparks 'humour', and how much of it is physical rather than intellectual.

<comment><USER7> I'm sure your colleague thinks his jokes are "always funny," too.

<comment><USER8> ....you missed it..

</comment></comment></comment><comment><USER9> Someone I know had this for the earlier part of their life (till about 35). Mostly, it was a defense mechanism, against extreme nervousness, anxiety, and "out-of-placeness."

They're much much better now. But will still let loose when extremely stressed out.

<comment><USER10> Did your acquaintance have a stroke/cancer? I doubt your friend had "Witzelsucht."

Instead, I think you're describing a socially awkward introvert performing cringeworthy feats. Awkward in the moment, but hilarious in hindsight.

<comment><USER9> You're right. It may not have been "Witzelsucht." But it makes me wonder if there's a mild form of "Witzelsucht" that's rampant in the population making people do cringeworthy things.

</comment></comment></comment><comment><USER11> So finally we understand Chandler's condition (Friends).

</comment><comment><USER12> I think that Reddit is making this a lot worse, at least for the people using it. Pretty much everyone there is constantly trying to make jokes in order to get a few upvotes and as a result even the most serious subreddits have people who feel the need to crack jokes. It is really fucking annoying.

<comment><USER13> This is why they introduced the [serious] tag, which literally prohibits jokes in the thread. This was a very welcome feature.

<comment><USER14> Welcome to whom? I'd rather have the wisecracks.

<comment><USER15> Welcome to the people who want to participate in serious discussions. You still have threads, and whole subreddits for that matter, with jokes. But before this tag, people who wanted a joke-free discussion were unable to do so on Reddit, and had to dig through piles of puns and jokes to get some thoughtful comments. Now we have space for both groups.

<comment><USER14> Yeah, but I feel that the combination of jokes and serious discussion is better than either one alone.

</comment></comment></comment></comment></comment><comment><USER16> At least they aren't developing a compulsive urge to crack farts...

<comment><USER14> Is "crack" the verb usually used here? Seems tenuous.

<comment><USER16> I was hesitant, but pulled the finger, so to speak.

</comment></comment></comment><comment><USER17> I have the compulsive urge to crack nuts.

Only when trying to eat them, mind you...</comment>


Ask HN: How legal is this?  - <AUTHOR>

I'm developing a web site and mobile app for road travelling and I wanted to include service areas, which, not being the focus of my project, represent very useful auxiliar information.
It just so happens that I found a website that has these service areas and then has free-to-download files for tomtom and garmin. These files don't contain all the info the website does though. So, I took these files and I wrote a crawler that goes to the website and gets the rest of the information for each entry. So now I have all this information to include in my product.
Now, I've noticed that despite the website being free, they do have a paid mobile app with this data. So... I'm not sure they will be very happy that I include it somewhere else, but is it actually illegal? The files are free to download and the website free to access (without any need for registering).
I haven't made my mind on whether or not my project is going to be just for me or if I will have greater ambitions for it, but I'd like to be prepared just in case. <comment><USER2> I don't know how legal it will turn out to be, but it sure is stupid, because they can almost certainly sue you and you're not established enough to defend yourself.

</comment><comment><USER3> First, for legal advise, always consult a lawyer. Even if there's a lawyer responding, they're not _your_ lawyer.

Second, check if the site has a EULA or so. If crawling is against the EULA, you might be guilty under the Computer Fraud and Abuse Act (penalties for this can be pretty severe).

Third, data itself is not copyright-able, so in theory it could all work out.

IANAL, this is not legal advise.

<comment><AUTHOR> Thanks, that was what I thought "data itself is not copyright-able".

And yes, if I move forward, I understand I will need real legal advice.

</comment></comment><comment><USER4> This didn't work very well for Padmapper.[1]

[1] [https://news.ycombinator.com/item?id=4286325](https://news.ycombinator.com/item?id=4286325)</comment>


Ask HN: Which is the best rails tutorial you've seen? - <AUTHOR>

<comment><USER2> <http://railstutorial.org/> really got the ball rolling for me.

<comment><AUTHOR> Thank you! And if I may ask, why tunaslut?

<comment><USER2> I'm putting the band back together. We're going to play two types of music, and we will be called tunaslut.

</comment></comment></comment><comment><USER3> <http://railsforzombies.org/>

Gamifying rails learning</comment>


The Hong Kong fight to cash in Japanese military yen - <AUTHOR> http://www.bbc.com/news/world-asia-china-33906603

<comment><USER2> I'd like to see the reasoning for declining the currency today. Presumably it's more than just a simple "because we don't want to". Right now the article only focuses on one side of the story.

<comment><USER3> Not establishing a precedent is probably the biggest issue, the reparations that could be legitimately charged to the Japanese for a half-century of very very bloody imperialism (the Greater East Asian Co-Prosperity Sphere was killing an estimated quarter million a month when we dropped the bombs) would make Versailles look small.

And Versailles should have taught us that declaring collective guilt on a people is unwise.

<comment><USER4> > And Versailles should have taught us that declaring collective guilt on a > people is unwise.

I just shudder of the day Germany backlashes badly after 70 years of perma- guilting. I think that the japanese approach to history is much more stable in the long run.</comment></comment></comment>


Apple tells Congress it found no signs of hacking attack - <AUTHOR> https://www.reuters.com/article/us-china-cyber-apple/apple-tells-congress-it-found-no-signs-of-hacking-attack-idUSKCN1MH0YQ?feedType=RSS&feedName=technologyNews <comment><USER2> So far those who refuse to believe Apple and Amazon in this issue have said that Apple's denial wasn't specific enough (hard to understand), or that Apple is forced to lie (hard to imagine how that would go over with the SEC and their investors), and they've complained that Aplle's denials don't come from executives (which doesn't actually matter - execs and board members are responsible for official corporate statements), or that the InfoSec exec and engineer's denials don't count because they were off the record.

Now we have a senior Apple exec, who is in a position to know the facts, providing a written statement to Congress. A statement that could land him in jail if it turns out to be made falsely.

Honestly not sure what else Apple can do to convince someone that BW's story is incorrect, but given the lengths they've gone to so far, I would be surprised to see them stop here. I wouldn't want to be Bloomberg right now - I'd bet they're in full on panic-mode in damage control.</comment>


Impossibly Hungry Judges - <AUTHOR> http://nautil.us/blog/impossibly-hungry-judges <comment><USER2> Here is the refutation being referred to in the article, the article author did a poor job explaining it, and the reason to reject isn't "effect is too large", because the effect is real -- but this study explains it: [http://journal.sjdm.org/16/16823/jdm16823.html](http://journal.sjdm.org/16/16823/jdm16823.html)

1\. Prisoners are ordered by whether they have an attorney (so those going last in a session are self-represented)

2\. Judges order prisoners often by "complexity of the case", so the ones taking most time are first (and therefore most likely to have a favorable decision, a short/easy case is probably a no-parole situation)

3\. Statistically, the cases that gained parole take longer than those that don't, so even if they are random/normally distributed, then if one falls at the end, it will come back to the beginning of next session. A simulation shows that even with random ordering, you still get the same graph (because the long/complex cases that could be paroled tend to be moved to next session when the session is almost out of time).

So after reading this, the graph means: Ratio of cases with a lawyer that can be finished in the remaining time in the session.

Both values are decreasing as the session continues, so it produces a heavy down sloping graph that resets each session to some approximately random value (.65).

<comment><USER3> I don't remember much from my operating systems class, but I do remember that "shortest job first" is a more efficient scheduling algorithm than "longest job first".

I cannot fathom why anyone would want to schedule 12 cases that will likely take 15-30 minutes each after the one that will almost certainly take more than 4 hours. I understand that type of scheduling in hospitals, because the complex, time-consuming case is probably the guy who will die if not treated _immediately_ , while the simple cases can afford to wait, but no one is going to die or go broke if their court appearance gets pushed back a week because too many "easy" cases suddenly popped up and pre-empted it.

<comment><USER4> The big thing missing from that assessment is case _variability_.

The clear-cut case with a plea bargain already assessed is short, but also predictable. If you schedule it at 11:50 AM, you'll still go to lunch at 12:00. The case that's going to take 16 different motions with unclear evidence isn't just long, it's unpredictable. Even if such cases take an average of one hour, you don't want to put them at 11:00 AM.

Shorter tasks are also easier to reschedule. Pushing three short cases from before lunch to after is much lower-impact than interrupting one big case midway through.

So I don't think your assessment is wrong, but there're other factors deciding this.

<comment><USER5> You impact fewer people by delaying a long case vs 5 short ones. But, you are likely to forget details and/or spend lunch thinking about a case that's open which is IMO more motivation.

</comment></comment></comment></comment><comment><USER6> A lot of replies in here seem to think the article is arguing that the _data_ be disregarded. My reading is that the author is saying that _because_ of the data, the conclusion that the cause is purely psychological must be disregarded.

The argument for that being, partially, that if the effect is caused by hunger that would predict widespread issues elsewhere in society that we do not see and, partially, that there are no other psychological effects this strong.

Another hypothesis is needed. For instance, anyone who has spent time in a courtroom knows judges like to get small simple cases done first to get people on about their day - perhaps the effect is because more complex cases scheduled towards the end of a shift at the bench are more likely to get a negative ruling?

<comment><USER7> From the article: "If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45 a.m."

This looks compelling at first but how many people are really making any considerable decisions on an empty stomach before lunch? Everyone at my company sets up meetings earlier (9 or 10am) or after lunch because they know how they are when they are hungry and/or thinking about their lunch break. How many people are actually subject to this effect on a given day and how many are making important decisions at that time?

There are a lot of aspects to our daily life that could be under this effect without us observing them individually or as a whole as "minor chaos".

Automobile drivers could be more aggressive and dangerous when hungry. "the highest percentage of them (crashes) occurring between 6-9pm -- evening rush hour. Commuters rush home daily to eat, spend time with their families, watch television and/or get to work on a second job." [1] and "Nationwide, 49% of fatal crashes happen at night, with a fatality rate per mile of travel about three times as high as daytime hours." [2]

Perhaps this effect is causing more automobile crashes but as individuals we chalk it up to people just wanting to get home under their own free will (because we all want to get home after work for a variety of reasons). Or we've just become accustomed to how evening rush hour is and we don't observe it as a more chaotic event affected by psychology.

Separately, why is "minor chaos" necessary to prove the data? This is just the author's parameters. Even though what the judges do is messed up in the hungry situation, it doesn't cause minor chaos and they're making substantial decisions. And the aggregation of this effect could be minor chaos but still not readily observable as the author suggests.

So to the author's key argument about minor chaos - if this effect has always been around then what we know about our daily life might not have attributable and observable chaos even though it does. Or the chaos might not even exist because the number of people truly affected is very small on a daily basis.

[1] [https://coverhound.com/blog/post/what-time-of-day-do-most- ac...](https://coverhound.com/blog/post/what-time-of-day-do-most-accidents- occur)

[2] [https://www.forbes.com/2009/01/21/car-accident-times- forbesl...](https://www.forbes.com/2009/01/21/car-accident-times-forbeslife- cx_he_0121driving.html)

<comment><USER8> > How many people are actually subject to this effect on a given day and how > many are making important decisions at that time?

That is the point, this effect is so remarkable, that even a small number of people making a small number of decisions over a long period of time is going to add up to a huge effect.

<comment><USER9> Maybe it does. How many 2pm meetings are had to correct for 11am meetings? The thing is, most systems have a lot of checks and balances built even. Even if a decision is made, it is quite easy to revisit it hours later. Even days after it is often pretty easy to revise a decision.

It would be better to look in areas where decisions are irreversible and consequences are immediate instead of society in general.

</comment></comment></comment></comment><comment><USER10> This feels like half good science (be skeptical of outlandish results) and bad science (can't be true! so... dismiss it?). Humans think in narratives, which is why every time the stock market goes down 3 points it's becaise "investors are nervous about Syria" then when it rebounds it's "investors unfazed by Syria" (or whatever).

The scientific approach should be to come up with a plausible narrative and then do everything possible to discredit it. If the narrative survives, it is likely accurate. This hungry judges narrative seems inaccurate for a lot of reasons, the double dip being the easiest reason for dismissal. But it does still beg the question: why this pattern? Are cases staggered in a specific way? How many cases were looked at? Does this pattern still hold and could we sit in on a case and see it happen in realtime?

<comment><USER11> As scientists in a field dismissing an outlandish result out of hand is horribly dangerous. As lay consumers of science who don't have the time to double check every paper we hear about we should absolutely ignore counter intuitive papers until such time as they're replicated and accepted by the relevant scientific community.

<comment><USER12> Dismissing a result doesn't mean it won't come back later with better data and modelling. It's really not that dangerous.

</comment></comment></comment><comment><USER13> A) Cases are not presented in a random order, B) judges attempt to complete all cases from a given prison before a break is taken, C) the study groups together "deferred" and "not granted parole" \- it's possible "deferred" cases are pushed to the back of the prison's docket. Also, D) shared representation is common, and in these cases it's possible and likely for the representative (who chooses what order to present his cases to the judge) to present the most promising case first.*

How this at all boils down to "hungry judges" and then gets reported as such is basically everything wrong with science reporting (which is to say, almost everything is wrong with science reporting).

* Probably oversimplified from [http://www.pnas.org/content/108/42/E833.full](http://www.pnas.org/content/108/42/E833.full)

</comment><comment><USER14> I really appreciate the author's critical analysis of this correlation presented as "fact" by Radiolab and I love how Hacker News and other blogs take these types of scientific findings and dig in for the truth. I think the PNAS paper refutes the original conclusion pretty thoroughly - I wish the Nautilus author would just explain that.

I don't think we should dismiss effects just because they seem really large (as the Nautilus author claims) but I do think that it's incredibly irresponsible of Sapolsky and Radiolab to be uncritically citing a study that looks like it was debunked in 2011.

I also think it's strange that the author cites the SJDM paper which is much, much less convincing, claiming that it refutes the original experiment. It looks to me like that paper just shows that by simulating a non-random order of parole requests they can create data that looks like the original experiment.

I love that Hacker News posts these things and people go through and analyze the papers. No one outside of the specialized field could possibly have time to analyze all of these papers but they clearly have implications that matter for everyone. I wish that popular science shows would do a more thorough analysis of these results on their own.

</comment><comment><USER15> For some reason I assumed the title referred to a problem like the "Byzantine Generals", or "Dining Philosophers." This is much more interesting.

</comment><comment><USER16> This author misuses a gaussian distribution by way of claiming that a standard deviation is outrageous...all on a phenomenon that is overwhelmingly likely NOT to be gaussian distributed.

<comment><USER17> For anyone who's curious, the procedure used is described in [http://www.aliquote.org/pub/odds_meta.pdf](http://www.aliquote.org/pub/odds_meta.pdf). The standardized mean difference is given by:

      log((65/35)/(5/95))/(pi/sqrt(3)) = 1.9646484930140864

The justification for this procedure is that the judges are assumed to make decisions by dichotomizing a continuous variable with logistic distributed error (this one statistical justification for logistic regression; see [https://en.wikipedia.org/wiki/Logistic_regression#Latent_var...](https://en.wikipedia.org/wiki/Logistic_regression#Latent_variable_interpretation)). The mean difference in the continuous variable is given by the log odds ratio times some constant, and the standard deviation of the continuous variable is pi/sqrt(3) times the same constant. Because the logistic distribution resembles a normal distribution (see [https://en.wikipedia.org/wiki/Logistic_distribution](https://en.wikipedia.org/wiki/Logistic_distribution) and figure in paper), the standardized mean difference given by this method will approximately equal the standardized mean difference of the latent continuous variable.

</comment></comment><comment><USER18> There are a lot of problems with the conclusions Radiolab and other science journalism draws from the original study. But this article isn't better: the author dismisses the results of the study as impossible, because it _seems_ impossible, because we would already know, etc.

<comment><USER19> If I ran a study that said whenever a bell rings at 11:45AM students in public schools exit their classrooms (99% within 2 minutes) and quickly file into the cafeteria, and that this was evidence for mass systemic Pavlovian conditioning, what would you say?

Might you say, "that's an impossibly unnatural conclusion", given everything we accept to be true, including our lived experiences--as students, as human beings.

Of course, it's a _logical_ fallacy to say so. But pure, unadulterated Aristotelian logic has surprisingly little application even in science. While as a logical matter such a conclusion on its face might not be impossible, it's not necessarily hyperbolic to label it so even in scientific discourse. Not the least because everything we know about Pavlovian responses suggests that there should be far more outliers, such that there must be at least some structural component involved, not a purely psychological response. In other words, the result is too _clean_. The real world is much messier, especially in the world of human psychology in complex settings, and the odds of seeing such a clean and consistent response relationship is extremely slim.

Quantum physics suggests that you could spontaneously teleport a kilometer away. Would I be wrong in believing impossible a paper that concluded that you spontaneously quantum teleported from home to work? What if the paper fails to establish--even nominally--the absence of other, less shocking, explanations?

Logically speaking, the teleportation might not be impossible, but in most others contexts (including statistics and other forms and methods of reasoning) one might fairly call the conclusion impossible, preserving the label implausible for scenarios more deserving of critical assessment.

<comment><USER18> > If I ran a study that said whenever a bell rings at 11:45AM students in > public schools exit their classrooms (99% within 2 minutes) and quickly file > into the cafeteria, and that this was evidence for mass systemic Pavlovian > conditioning, what would you say?

> Might you say, "that's an impossibly unnatural conclusion", given everything > we accept to be true, including our lived experiences--as students, as human > beings.

No, I would say that there are alternate theories which better explain the data. And it would not be a logical fallacy to say that.

> Of course, it's a _logical_ fallacy to say so. But pure, unadulterated > Aristotelian logic has surprisingly little application even in science. > While as a logical matter such a conclusion on its face might not be > impossible, it's not necessarily hyperbolic to label it so even in > scientific discourse. Not the least because everything we know about > Pavlovian responses suggests that there should be far more outliers, such > that there must be at least some structural component involved, not a purely > psychological response. In other words, the result is too _clean_. The real > world is much messier, especially in the world of human psychology in > complex settings, and the odds of seeing such a clean and consistent > response relationship is extremely slim.

The problem with the "argument from too-clean" is that clean data occurs all the time. If you drop a ball in my living room it will fall down 100% of the time. Are we to disbelieve these results because they are too clean? Obviously not.

Look at your analogy. Obviously the migration to the lunch rooms isn't caused by Pavlovian response (at least not exclusively) but that's not because the data is wrong: the students _do_ migrate to the lunch room at the described time. If you're claiming this data is too clean to be believable, then you're claiming that the students _don 't_ migrate to the lunch room in consistent numbers. This is just as wrong as the Pavlovian response theory.

</comment></comment></comment><comment><USER20> > _But I want to take a different approach in this blog. I think we should > dismiss this finding, simply because it is impossible. When we interpret how > impossibly large the effect size is, anyone with even a modest understanding > of psychology should be able to conclude that it is impossible that this > data pattern is caused by a psychological mechanism._

That's not how science works.

<comment><USER21> Isn't it? The answer wasn't that judges were making decisions based on their hunger, it was that judges arrange their schedules for some reason other than random. If a biologist working with samples gets a reading that can't be explained by a natural process but can be explained by a miscalibrated piece of equipment or a test tube that wasn't properly washed, it absolutely is how science works for the problem to be identified and the labwork to be run correctly, rather than publishing a result based on bad data. It's the same here; if the magnitude of the effect you find is at least an order of magnitude higher than any other study of similar effects, before publishing the finding you should have an airtight case that what you found is the effect you're measuring, not a data quality problem. They couldn't have an airtight case because their findings weren't driven by what they thought they were measuring.

</comment></comment><comment><USER22> > If hunger had an effect on our mental resources of this magnitude, our > society would fall into minor chaos every day at 11:45 a.m. Or at the very > least, our society would have organized itself around this incredibly strong > effect of mental depletion.

Kind of pedantic, but we indeed organize society around lunch time

</comment><comment><USER23> It would be great if such kind of criticism would start by just writing down what it sees in that diagram. What I see without training looks totally normal and as if it would be validating the claim strongly.

</comment><comment><USER24> > But I want to take a different approach in this blog. I think we should > dismiss this finding, simply because it is impossible.

That is truly begging the question. He goes into details about what the impact to society would be if we're that affected by hunger/fatigue, but the validity of the finding still remains.

> It is up to authors to interpret the effect size in their study, and to show > the mechanism through which an effect that is impossibly large, becomes > plausible. Without such an explanation, the finding should simply be > dismissed.

Again, horrible circular logic here. Simply disregarding things because they're "impossible" is the antithesis of science. The authors of the paper gave a hypothesis that explains the (undisputed) data.

You should not simply dismiss this by just saying "it's not true because it can't possibly be true". For instance, try explaining the germ theory of disease to somebody before Anton van Leeuwenhoek invented the microsope.

<comment><USER25> _Based on this data, the difference between the height of 21-year-old men and women in The Netherlands is approximately 13 centimeters. That is a Cohen's d of 2. That's the effect size in the hungry judges study.

If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45 a.m. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would be illegal. If a psychological effect is this big, we don't need to discover it and publish it in a scientific journal—you would already know it exists. Sort of how the "after lunch dip" is a strong and replicable finding that you can feel yourself (and that, as it happens, is directly in conflict with the finding that judges perform better immediately after lunch—surprisingly, the authors don't discuss the after lunch dip)._

It is impossible because we have very strong evidence against it.

<comment><USER26> The flaw is that the leap from decision in difficult cases to dangerous driving or other behavior are wild guesses. He sets up straw men versions of what the impact of the evidence should be, then rebuts them.

The reality is that the same psychological state that makes a judge less lenient might also make us better drivers, not worse.

<comment><USER25> You're missing the point. The point is that the effect sizes in question are large enough that people should notice them in their day to day lives, because the proposed mechanism of action should not only apply to judges, but should apply to everyone everywhere. But we don't see this big obvious affect in other places.</comment></comment></comment></comment>


To Learn Japanese, You Need a Rocketship - <AUTHOR> https://learnjapanese.best/2019/10/07/to-learn-japanese-you-need-a-rocket-ship/ <comment><USER2> Daje V, this is so interesting</comment>


Nylllon - Useful threads - <AUTHOR> http://nylllon.com/

<comment><AUTHOR> Not sure how you guys feel about fairly blatant spam posting but hopefully this will prove a useful resource to some of you. Cheers.</comment>


Do Startups Create Jobs or Destroy Them - <AUTHOR> http://www.betabeat.com/2011/11/03/do-startups-create-jobs-or-destroy-them/

<comment><USER2> Without having read the article, the answer seems quite apparent to me. They create jobs in one place and destroy them somewhere else.

Startups are really just businesses. So, when people say "I'm going to do a startup", why don't they just say "I'm going to start a business". Well, partly because if you said "I'm going to start a startup" it would sound redundant.

But why don't we say "business" instead of "startup"? The just-so story I am drawn to is that startups must be disruptive. A business can be opening a chain of car washes, or a McDonald's franchise. Nobody is going to use the word 'startup' there. However, what if I started a business, and my goal was to develop a paint that never required washing. That would almost certainly be considered a 'startup'.

Now, when you disrupt, you are disrupting commerce that is happening. It's real money changing hands, people's jobs and so on. For the 'no wash paint' startup, success means putting all the car washes out of business, eventually anyway.

Finally, should we care? No. Nearly every business would fire all of it's employees if it could. The carwash you are putting under? If it could do the same quality of washing with half as many employees, it would fire half of them. If it could do the same job with no employees, it would fire all of them. The point is that there is no moral issue here, merely an issue of who is going to collect the money. And of course the money not going to carwashes will go to something else blah blah blah saddle makers blah blah blah Model T.

Now, there is a Player Piano scenario to worry about. What happens when, inevitably, we automate all the mindless tasks we currently pay people to do? Sure, lawns will look great, cars will be clean, shelves will be stocked, houses will be immaculate, and you'll never see peeling paint again, and ipods will be fully assembled by robots. What are all the mindless people going to do? I don't honestly know. We need to figure this out.

<comment><USER3> I think the Vonnegut reference is an apt one. But there is also a simpler question at hand. When we romanticize startups, do we actually undermine the larger, perhaps less interesting companies who are better at generating employment and economic growth.

In the same way that the American dream of home ownership came back to bite us, perhaps the cult of entrepreneurship could use a hard look.

<comment><USER2> You don't increase wealth by being inefficient. The cult of startups leads to more startups, but it doesn't make them more successful.

There are some exceptions, but the market can always be irrational for a little while. LinkedIn and GroupOn aren't real businesses and eventually the stock price will reflect this. However, in the short run they are able to pump the stock price by having a microscopic float and getting listed on things like Russel 2000.

Oracle, Cisco, Intel, HP are all startups, they are just very old and established. They employ tons of people. They aren't doing us any favors if they barely survive, however. Inefficiency makes us all poorer, even if in the shortrun it makes some people better off because of a paycheck (HP employee for example).

</comment></comment></comment><comment><USER3> Let's take an example - Etsy has grown to 80 engineers in the last year. If 50 of these workers had decided they wanted to go their own way and create a startup, how many jobs would have been created and how many of those people would be out of work?

Easy money at the seed stage means a lot of companies that are small ideas trying their hand at building a business.

</comment><comment><USER3> Startups have different goals. Some want to stay lean and disrupt old industries. Others, like Groupon, are inventing new kinds of comapanies and hiring like mad.

Whatever the net-net is in terms of jobs created or killed by startups, the US is better off for having a class of creative people in the technology space trying to create new business models.

</comment><comment><USER4> I'm always surprised that people claim things steal jobs. If you look at the record of the unemployment rate, it's pretty obvious that it stays the same over time. There are ups and downs, but they are not caused by inventions or efficiency increases.

</comment><comment><USER5> When you automate a process to scale, you reduce the need for people. In that sense, startups destroy jobs.

But when you create new ways in doing things like unlocking value (like how Facebook has done with connecting the world), jobs are created.</comment>


Show HN: DataTau opensourced – Django based HN clone - <AUTHOR> https://github.com/datatau-net/DataTau <comment><AUTHOR> Hi HN,

As promised, we open sourced the app powering [https://datatau.net](https://datatau.net), a Data Science newsboard based on HN. To give it some context, original site (in .com domain) has been down from May 15, so we cloned it and continued the project (got no answer from original maintainers of the site). We published it on HN last weekend and made to the front page for some hours.

We have no previous experience with open source projects more than opening some pull requests, so any help from you guys is very welcomed.

In case you want to start a community based on HN style and feel (and do not like alternatives like lobste.rs for some reason), this repo can be a good starting point. Not all HN functionalities are implemented and there are lots of things to polish and improve, but the core is there.</comment>


Awesome Bookmarks – Kill time by exploring my personal collection - <AUTHOR> https://eryb.space/2020/06/21/awesome-bookmarks-kill-time-by-exploring-my-personal-collection.html <comment><USER2> Thanks very much. I found a blog I'll be adding to my rss feed as well as a few useful tools. Bookmarks can be really personal things, so I respect the courage in sharing.

<comment><AUTHOR> I manually removed personal bookmarks before sharing the list. :p I shared it because I found it interesting revisiting them and thought that others might too enjoy.</comment></comment>