On Complexity: when you can't even get time right - <AUTHOR> http://www.rebol.com/article/0400.html

<comment><USER2> Man, I expected a much more interesting article based on the title.

</comment><comment><USER3> I totally agree people should stop making excuses for Microsoft. At the same time, rants against complexity in general always make me think of this quote:

"For every complex problem, there is a solution that is simple, elegant, and wrong." -- Mencken

</comment><comment><USER4> Changing timezones have caused some of the most complicated things in the coding I've done. It seems so simple, but its never right. Look at how many different libraries are in the standard Java library to handle time.

<comment><USER5> Sun got it wrong the first time and again the second time. I haven't looked at the Java 7 api, but they'll probably get it wrong the third time.

OK, I took a quick look. It's better, but still not what I would consider an easily usable system. I will still have to write a parser that will take unconstrained user input. There are 8 or 10 formats that are unambiguous and easy to recognize. Do it. Joda-Time has too many classes. Will it work seamlessly with SQL Date?

</comment></comment><comment><USER6> Ironically I think it's just as likely that it was simplicity that caused the error rather than complexity:

 _Older Microsoft Windows systems usually store only a single start and end rule for each zone_ ^

If only they were forward thinking enough to know that laws change... and thus DST will eventually change too. A similar form of naive simplicity caused the Y2K mess. I think this falls in line with that incident.

I agree that Microsoft should have properly solved this problem on all OSes and pushed the solution back in 2006. It's ridiculous that this is still a problem on a recently installed recently patched box. Especially on XP which is currently their core market.

^ <http://en.wikipedia.org/wiki/Daylight_saving_time>

<comment><USER7> That's not simplicity.

That's simply being lazy and wrong.

Timezones have been changing for over a hundred years.

<comment><USER8> Yeah. And the changelog for the tzdata package is quite interesting in that regard. Usually changes in timezone information occur months or years before they actually need to occur (including recognizing leap seconds).

</comment></comment></comment><comment><USER9> OT: every time I bump into Rebol, I make a mental note: "would like to find an excuse to use this language someday."

      Some many cool languages...so little time ;)

</comment><comment><USER10> The worst part about time on Windows is that the system clock is _always_ set to localtime.

With that as the default, nearly all Windows software is completely timezone- ignorant, which is especially shitty for web software. The most support I ever see is a user preference for "Server time +/-N:00", which forces foreigners to manually fiddle with it twice a year.

Just one of the many bullshit misfeatures that Microsoft should have fixed with NT but chickened out on.

</comment><comment><USER11> I found this title a bit ironic. Although MS should have gotten it right, keeping time is an inherently complex problem.</comment>


Ask HN: Fabrication startup name? - <AUTHOR>

I remember a company that you could design products and I believe they cut them out of wood on demand as people ordered them.  Can anyone help me out with their name? Google isn't helping. <comment><USER2> this recently appeared here in HN. emphasis on stainless steel tho.

[http://www.crunchgear.com/2009/08/04/the-future-of- fabricati...](http://www.crunchgear.com/2009/08/04/the-future-of-fabrication- is-here-shapeways-announces-stainless-steel-printing/)

shapeways.com

</comment><comment><USER3> Big Blue Saw?

<comment><AUTHOR> Not the one I was thinking of but I bookmarked them.</comment></comment>


Show HN: An opinionated, scrmum focused product management tool - <AUTHOR> https://lesspie.com/ <comment><USER2> This seems super generic. Why is this tool better than any of the others? Also, smooth scroll is a blight that should be eradicated.</comment>


Arachni Security Scanner - <AUTHOR> http://www.arachni-scanner.com/ <comment><USER2> I tested it! Happy to see that it's working flawlessly in Windows 7, including the web interface. It's fast too! I'd say it's in the middle between Nessus scanner and Owasp Zap for the following considerations:

\- OWASP ZAP is an HTTP vulnerability scanner only and takes some time to complete.

\- Nessus is a complete vulnerability scanner (not only HTTP) and of course testing takes even more time to complete.

\- Arachni, also seems to be just an HTTP vulnerability scanner, but it appears to come up with useful results faster that ZAP (though this is surely due to the former doing more in-deep scanning). It is well designed and easy to use.

In any case it's nice to have another advanced tool like this in our pockets, and it's another big success for a Ruby (on Rails) application!

Please let me know your opinions!

</comment><comment><USER2> no windows version :\

but looking at [http://www.arachni-scanner.com/blog/help-us-test-windows- sup...](http://www.arachni-scanner.com/blog/help-us-test-windows-support-xml- json-auditing/) they seem to be working on it... (some months passed already though..)

<comment><USER2> And... there we go! [http://www.arachni-scanner.com/blog/windows-package-and- rest...](http://www.arachni-scanner.com/blog/windows-package-and-rest-service- now-in-the-nightlies/)</comment></comment>


George Washington's "Yelp Reviews" - <AUTHOR> https://daily.jstor.org/george-washingtons-yelp-reviews/ <comment><USER3> This was an interesting endeavour and I have to say a good initiative by someone who could have just stayed in luxury and not seen the realities of most poor travellers.

<comment><USER4> Washington was an exceptional man who walked away from near-total power. It's only been done a few times in human history.

<comment><USER5> What would the best book be on him? I'm an Australian and my grasp of U.S. history is, to be frank, almost non-existent.

<comment><USER6> I haven't read it yet (will soon), but I've heard nothing but good things about the Ron Chernow biography. I'm reading his biography of Hamilton right now, and I'm really impressed. I think he strikes just the right balance between academic rigor and narrative storytelling.

<comment><USER7> Can confirm. I read them both last summer. Amazing people, fantastic writing.

<comment><USER5> Thanks, appreciate the info, I'll get the book and have a read.

</comment></comment></comment></comment></comment></comment><comment><USER8> That was an entertaining read. Thanks for the link.

</comment><comment><USER9> Recently having read Gore Vidal's book 'Burr', Washington come out as a poor military leader, whose terrible military decisions prolonged the American Revolutionary War by years. In fact none of Washington, Jefferson or Hamilton come out as people of virtue and character as per the book.

<comment><USER10> Not having read that book, I would say you have to consider the source and why they might have come to those conclusions. Now, it's possible Gore is correct, but then again, why are others incorrect in their assertion, if Gore is right?

</comment></comment><comment><USER11> (I have a comment that may be relevant to some other observations here, but mostly unrelated to the piece itself, amusing as it is.)

To understand the USA, and modern politics in general, I find it very helpful to know a little bit of Roman history. The US were deliberately moulded by the very knowledgeable founders as a revised and improved version of the classical Roman republic. The attempt was wildly successful, and measured by most any metric it widely beats the performance of the original system.

There is one crucial point to the original Roman republic: it was founded by aristocrats who were fiercely jealous of their liberty and equality _to each other_. Their collegial Senate ruled since they killed their king around 509 BC, until they killed Julius Caesar some 465 years later - republicans have this 'sic semper tyrannis' thing for assassinating monarchs.

A second point is that republicans have a tight lock on the perception of the advertised virtue of their system. Whether it is actually better for the plebs, or the whole of society, to be ruled by a senate or a monarch is never to be left in doubt.

Augustus craftily shoved the monarchical empire down the aristocrats' throats by claiming the whole time that he was merely making the republic function as intended. However the immediate successors had a tougher time, and were quite probably historically smeared by senators who resorted to pens instead of swords.

<comment><USER12> That is quite broad, and not related to the piece. I don't disagree with you, but it sounds like you're wandering far afield to make some other point not even close to where the article is going. Banging on your own drum as it were.

The well-educated of the time were taught classics, so yes, Romans had an influence, as did the Greeks. They were protestants, more of a deist nature, but protestants. So there's that influence. Wars of religion were recent for them. Up until 1776 or so, they considered themselves Englishmen. It was a time of great philosophical minds. You can't talk about the founding of the U.S. without talking about John Locke, for instance.

It's not that the Roman metaphor is completely inaccurate, it's that it's only a tiny slice of a very complex picture. Looking at the U.S. as some modern-day version of Rome can lead to all kinds of silliness. It's fun to do, sure, but best used very sparingly. It can lead to having all sorts of huge gaps in knowledge while assuring yourself that you understand the important parts of what happened.

<comment><USER11> > without talking about John Locke

That was the brilliant new technology part, with the "consent of the governed" and its production.

I was just going for the Cincinnatus, anti-monarchical, angle - why go to such lengths to avoid any appearance of autocratic power, or shred of desire for it. Not even imposing on important families - or favoring them with - <USER2> presidential visit.

(For another amusing story of president-elect travel, "Plums delivered nuts safely" is a good search. Coincidentally about a life that years later ended for the reasons mentioned.)</comment></comment></comment>


Weekend Hack: SpotCast - DJ your Spotify to your friends - <AUTHOR> http://gist.io/3625870

<comment><USER2> I've been playing with SpotCast quite a bit - it's great fun so far :)</comment>


Correlation is usually not causation. But why not? - <AUTHOR> http://www.gwern.net/Causality <comment><USER2> All I know about causation and correlation I learnt hunting bugs in large legacy software systems. In that environment I got the impression that correlation almost never equalled causation, but that's only because the hardest bugs, the ones I remembered, were hard because the obvious correlations did not help identify the root cause. A similar argument might be made for scientific studies: most of the easy causes that can be identified from correlation have already been found, leaving the majority of new studies with correlations that don't easily establish causation.

<comment><USER3> I think this low-hanging-fruit idea is generally true for all of science: after centuries of science as a profession, pretty much all the easy stuff has already been done.

It's actually an argument for more cojones in science-- being willing to do bold stuff and explore "crazy" hypotheses. If all the easy stuff is done already, then picking methodically and timidly among the dregs is unlikely to ever yield anything.

<comment><USER4> An alternative viewpoint: everything's easy after it's been done, but it's hard up until then.

> after centuries of science as a profession, pretty much all the easy stuff > has already been done.

But with the benefit of all that has already been done, shouldn't we be able to do things now that were previously impossible? In other words, "easy in 2014" != "easy in 1200".

> It's actually an argument for more cojones in science-- being willing to do > bold stuff and explore "crazy" hypotheses.

Maybe doing whatever's easy is the most efficient (by time, money) way to make progress?

<comment><USER3> I'm not sure... seems like what you say may hold for a while, but eventually you start hitting a more objective sort of hard: things that are hard for human beings to comprehend due to the limitations of our intelligence itself. Beyond that there's probably an even harder hard-- when you start actually running out of new things to discover. Can there really be an infinite number of physical laws, principles, and useful relations? Or at some point have you actually found most of basic physics?

Once you start hitting those, you've either entered a permanent era of diminishing returns or one where you can only really make progress by radically redefining problems, making leaps, or trying wild and crazy ideas in the hopes of unlocking some isolated seam of high-value research that isn't connected to the others in the fitness/value state space graph.

<comment><USER4> Sure, we can't rule out those possibilities ... but to me it just sounds like wild speculation.

</comment></comment></comment></comment></comment><comment><USER5> This question is bound to be a can of worms. There has been a great deal written about the matter, particularly in regard to observational studies.

Drawing causal inference is overwhelmingly likely to be wrong when there is a good chance that unknown variables are influencing the correlates observed. In health-related sciences that more often than not is the case.

A few years ago there was a study correlating hours of TV watched and ADHD symptoms in children. The news media picked up on these "findings" and of course the causal influence of TV on ADHD was reported.

It was obvious that saying watching TV caused ADHD was absurd, that other variables weren't taken into account, e.g., some other characteristic of ADHD kids prompted watching TV more than other kids.

There was a great article published in PLOS several years ago (ATM I don't have the link) showing mathematically that the odds were about 1 in a million that an observational study like the above would turn out to be a "true" causal relationship, and the author concluded most published studies were junk.

In experimental studies, variables are limited and controlled as well as possible. With fewer and known variables, correlations would have a greater chance of revealing a reproducible causal relationship among events.

The discussion gets tripped up when it comes to defining "cause" or "causal relationship". The theory is controlled as an experiment may be, there's a possibility that unknown variables were present and affected the phenomena occurring in the experiment. Conclusions can't be absolute, but only true to some probability.

I think the history of science over the last 100 years or so has something to say about the nature of "causal relationships".

<comment><USER6> > There was a great article published in PLOS several years ago (ATM I don't > have the link) showing mathematically that the odds were about 1 in a > million that an observational study like the above would turn out to be a > "true" causal relationship, and the author concluded most published studies > were junk.

I very much doubt the second part of that, that most published studies are junk. The idea that not showing a causal link and only a correlational one makes a study junk is not held by anyone in the field who garners a whole lot of respect or notoriety. I'm not saying the quality is equal whatsoever, simply that correlation studies are not inherently junk - some are fantastic and some are quite the opposite.

There is no question that journals in general are pumping out a significant amount of junk (in studies of all types), but I speculate the root cause of that has more to do with the rise of "publish or perish" and significant increases in grad school enrollment. And even worse the notion that for a grad student, the number of publications they have their name attributed to is more important than the content they publish in terms of employment after they graduate. So there is a situation where people have more pressure to publish than ever before [0], there is more competition for scarce funding so studies are vastly underfunded and studies are rushed so another can begin to add to the resume.

That doesn't mean there is less good science being done either! There probably is more good science being published now than ever before, the problem is the signal:noise ratio has gone down making it harder for good studies to get media attention, and easier for the media to latch on to whatever story they think will get viewers.

[0]: Sidenote, this also makes it increasingly challenging not only to have high quality research, but research that only is making a correlation as opposed to going through and providing evidence to claim you might have a causal link.

<comment><USER7> This, I think, is the study the first poster meant, Ioannidis 2005, 'Why Most Published Research Findings Are False':

[http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fj...](http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0020124)

It's quite a famous paper. Many rebuttals, comments, blog-posts etc, have been published, have a look at the comments on the PLOS site, I especially like this blog post summarizing more research:

[http://simplystatistics.org/2013/09/25/is-most-science- false...](http://simplystatistics.org/2013/09/25/is-most-science-false-the- titans-weigh-in/)

From the same author, here's a small overview of the discussion at the end of 2013:

[http://simplystatistics.org/2013/12/16/a-summary-of-the- evid...](http://simplystatistics.org/2013/12/16/a-summary-of-the-evidence- that-most-published-research-is-false/)

Quote on the Ioannidis paper:

>Under assumptions about the way people perform these tests and report them it is possible to construct a universe where most published findings are false positive results. Important drawback: The paper contains no real data, it is purely based on conjecture and simulation.

</comment></comment></comment><comment><USER8> Because you can find things that are correlated, but have no practical relationship - many (frequently humorous) examples here:

[http://tylervigen.com](http://tylervigen.com)

<comment><USER9> Datamining correlations like that confuses two issues: sampling error and 'everything is correlated'. When you find a correlation like that, it's unclear whether it's a spurious correlation which will disappear if you just collect more data (and caused by slicing the data in a thousand ways without any compensation by this) or whether there genuinely is some sort of opaque correlation which will persist no matter how much data you collect (maybe 'US spending on science, space, and technology' _really does_ correlate with 'Suicides by hanging, strangulation and suffocation' through something like government stimuluses and economic growth).

</comment></comment><comment><USER10> Bad winter weather can cause auto accidents, and we expect a positive correlation between bad winter storms and winter auto accidents. Okay, but in the northern hemisphere, living in more northern latitudes also correlates with winter auto accidents but does not cause them.

For heart disease, we know that the main causes have to do with aging. Well, then, since now the audience for TV news is comparatively old, we can expect that watching TV news has positive correlation with heart disease. Still watching TV news does not cause heart disease.

In the US NE, hurricanes are positively correlated with pretty leaves on the trees, but the leaves do not cause the hurricanes, and the hurricanes do not cause the colors in the leaves. Instead, hurricanes are caused by the surface waters of the Atlantic Ocean hot from summer sun with cooler air on top, and that situation is caused by the fall weather with also causes the colored leaves. So, colored leaves have a _spurious_ correlation with hurricanes but do not cause them.

Correlation is much more common and much easier to establish than causality. Usually the convincing evidence of causality if some basic physical connection.

<comment><USER11> Your first example is wrong. If you move to a more northerly latitude then it will increase the risk of accidents for you (i.e. there is causation). It is indirect causation, but then ultimately (almost?) all causation is going to be indirect if you take the reduction far enough. That is bad weather doesn't directly cause accidents, but ice on the roads does ... etc.

<comment><USER10> No: The bad winter weather, say, ice on the roads, is the _cause_ of the accidents. If happen to have a winter with little or no snow or ice, then the number of auto accidents will go down no matter what the latitude: So, latitude can't be the cause; ice and snow on the roads is the cause. So, if live in a lower latitude, say, closer to the equator, and get a freak winter snow storm with a lot of ice and snow on the roads, the we will see auto accidents in spite of the latitude.

For indirect causation, I just said winter weather to try to be brief. But, sure, the cause is the ice and snow on the roads, not just the _weather_ in some vague sense. I was trying to keep the explanation simple, avoid discussing indirect causation, and concentrate on the OP issue of causation versus correlation.

</comment></comment></comment><comment><USER12> Normally, causation has a couple confounders for correlation, such as

\- inverse causation \- common cause \- random correlations

The first two should be relatively stable and reproducible, and we could then proceed to find out about causation with an intervention study (e.g. if "good education" directly causes "good job prospects", will it help if we give people good education that wouldn't normally get one? Or does it improve people's educational achievement when we give them better access to jobs?)

The third isn't reproducible, but should normally be relatively rare. Why we're seeing more and more non-reproducible results is usually

\- People fishing around in data sets for correlations, or tweaking experiments until they find a correlation

Because of this, we end up with a great deal of correlations that have nothing to do with causation.

For a related but different perspective, see this article ("Language is never, ever random", Adam Kilgariff) [http://www.kilgarriff.co.uk/Publications/2005-K-lineer.pdf](http://www.kilgarriff.co.uk/Publications/2005-K-lineer.pdf)

</comment><comment><USER13> The article is a little dense for me, and presumes knowledge about Probabilistic Graphical Models (PGMs) and directed acyclic graphs / causal Bayesian networks (DAGs) without introducing the background knowledge - so, I expect my reading of it missed a lot of the details.

But the one thing I came out wondering, is if you do a proper randomized double-blinded study with a large population sample, say, taking 1000 people with a particular illness, completely randomizing them, and then giving 500 of the sample a particular treatment, and 500 of the sample a placebo that is indistinguishable from the treatment. If an unbiased third-party, then observes the groups (still without knowledge of which group had which treatment), and if one group shows markedly different results (say, 490/500 of the treatment group recover, and only 10/500 placebo group recover), is it fair to say that in this particular scenario, that correlation of recovery with the treatment implied that the treatment caused the recovery?

<comment><USER14> Well it's fair to say that experiment shows causation but no I don't think you can say the correlation shows it works. I think it goes against the definition of the word.....

I found the article too dense to get as well, but I do think 'perhaps' we could use correlation more to assume causation. I think we are too cautious and certainly the haters always bring in correlation to stop science articles they think don't follow their beliefs. Not sure if that's compatible with the OP or not, it hurt my head.

<comment><USER4> > it's fair to say that experiment shows causation

No, that experiment doesn't show causation.

> I do think 'perhaps' we could use correlation more to assume causation. I > think we are too cautious

Why do you think that? Is it because you think that science is moving too slowly (thus wasting time and money)? Is it because you think that good results are mistakenly discarded?

"using correlation more to assume causation" should produce results more quickly, both correct and incorrect. Do you understand the costs of incorrect results? First, incorrect results can have negative consequences for the consumers: for example, drugs that have harmful impacts and no beneficial ones for the people taking them (in addition to the cost of the drug, and the opportunity cost of not doing something else known to be beneficial). Second, time is wasted following up incorrect results and work based on incorrect results would be worthless. Third, the incorrect results would have to be overturned (and this communicated to everybody who had taken them as correct).

I think deciding whether to "use correlation more to assume causation" needs to take into account the costs and benefits of incorrect results, as well as the costs and benefits of correct results. Do you have any data on this? (unfortunately, I do not)

> certainly the haters always bring in correlation to stop science articles > they think don't follow their beliefs

Can you support this accusation with examples of it happening to articles that actually do a good job of showing their result is something more than a correlation? Or are you saying that this happened to articles that just present a correlation?

</comment></comment></comment><comment><USER15> Correlation vs causation is not actually a complex mathematical problem. The issue is more philosophical.

Bayesian networks are a highly sophisticated and flexible framework for thinking about causation. And yet the essence of Bayesian networks can be captured in much simpler methods like Instrumental Variables or simply regressions with controls.

In all cases, the true distribution of observables (which we can estimate from samples) combined with some assumptions about the possible nature of causality (either very sophisticated in the case of Bayesian networks, or very simple in the case of instrumental variables) lead to the actual causal relationships.

Where do these assumptions come from? Consider a randomized trial. Even though physically, it is possible that some hidden cause influenced both the random number generator which selected patients into a trial, and also whether patients will get better. And yet people universally believe that there is no such mechanism, and so they believe that randomized trials prove the causal relationship between taking a drug and getting better.

No physicist ever wrote down an equation proving this. It is simply something we deduced from our human understanding of how nature works.

Put simply, causality is not a physical notion, neither is it a statistical notion. It is a part of our intuitive understanding of the physical world, in which a higher level notion of causality exists, beyond that described by special relativity.

<comment><USER16> To add to your point, we cannot prove or disprove the existence of causation (Try to conceive a falsifiable experiment about causation and I would argue you would end up with a metaphysical crisis). The 'is-ought' issue David Hume showed, where just because something 'is' a way does not indicate how it 'ought' to be or will continue to be, has highlighted to us how difficult it is to think this problem for hundreds of years. Another way to look at it is to ask "Who or what is to guarantee the law of physics will remain the same tomorrow? What is to stop the speed of light changing to 1 mile per hour?"

This isn't to say the article posted is of no use. Having a 'graph-like' mental model of how things work is incredibly useful, as most education simplifies real-world problems into a few key issues. Although most non- computer science issues can be reduced successfully using the 80/20 rule in real life, sometimes some problems require us to look at the 100s of contributing factors to allow us to solve the problem we're facing properly.

The more 'graph-like' problem solving becomes acceptable as way to solve issues the better of everyone will be.

</comment></comment><comment><USER17> The difference between correlation and causation is merely conventional. Does the striking of a match cause it to ignite? There is no way to prove that it does, only the correlation between the striking and the ignition makes us say they're causal. Correlation is the only way to determine what's causal and what's not.

On the other hand, if you want to look at it philosophically, then the only sensible definition of "causation" is "anything necessary for a thing to exist." So what's necessary for a thing to exist? Every other thing in the universe that is not that thing! Pluto causes us to exist right now because it hasn't turned into a giant space goat and swallowed the Earth. Yes, the fact that something DIDN'T prevent the existence of a thing is also ultimately a cause of that thing's existence. Causation in its purest form can help us understand the nature of reality, but it can't help us predict anything. It's only when we draw a line between causes that we can control and causes that we can't that it becomes a practical tool (not that understanding reality isn't practical).

So you can see, this whole correlation != causation discussion is pure nonsense. Get some philosophical skill before you try to discuss philosophical issues and stop embarrassing yourselves. Sheesh, it's ridiculous.

<comment><USER18> What? Your comment is pure nonsense. Of course you can prove that striking a match causes it to ignite. There's a long chain of events, each of which trivially physically verifiable, which starts with you moving the match head while applying a given amount of pressure against the material lining the matchbox, which then, due to friction, flakes off (both head and lining), the two react together to produce a spark and a flame starts. You can cut the chain down to micro-events and you'll find each one to be consistently verifiable.

Or do an experiment - grab a mug and lift it. I claim that your arm motion while your hand had grabbed the mug caused it to ascend. We can again examine the basic physical events, cutting them as fine as you'd like, and we'll see that it really was you with your arm, who caused the mug to ascend.

This is a cause-and-effect link -- when you can show how event A lead to event B via a chain of events, each of which directly causes the next. At the least you need to show that there is a sensible progression from the cause to the effect, in order to claim a causation.

Correlation on the other hand is a description of past observations. You can observe that each time it snows people increase their energy expenditure in order to heat their homes. Does the increase in heating cause the snow? That doesn't seem likely, knowing how heating works. It could be that increased energy demands leads to more coal being burned, leading to larger clouds being formed and lowering the outside temperature. However, even though increased coal burning does lead to such effects, they're pretty small from just increased heating, so we can discount this hypothesis. On the other hand, you can show that cold weather causes snow. Water that falls in cold air crystallises and becomes snow. So there is a definite causative relationship between cold weather and snow. Could there also be a causative relationship between cold weather and heating? Well, assuming that people wish to live at a constant surrounding air temperature of about 20-30 degrees, that seems very likely. There are of course individuals who don't fit that profile, but they are very few (I can't actually provide a study which supports that claim, but just assume it for the purposes of this demonstration). In the end, we can say that while snow and heating are correlated, heating does not cause snow. They simply have a common cause - cold weather.

And finally, if you keep your home at a constant temperature, the setting on your heater does not correlate with the temperature inside your home, because the temperature is constant, while you have to change the setting up and down to counteract the more or less cold weather outside. It does correlate perfectly with the outside temperature, but I'll leave proving that putting your heater on high doesn't make the temperature outside drop by 10 degrees as an exercise.

<comment><USER17> Any fact that depends on observation is uncertain, no matter how small or trivial it may seem. This whole physical world could be an incredibly elaborate computer simulation. We can never know the true nature of our observations, so nothing we observe can ever prove anything.

</comment></comment></comment><comment><USER19> In reality, causal mechanisms for the phenomena we most care about (economic and social phenomena particularly) are so fantastically complicated that you never really "figure it out". Nevertheless, if you can exploit the hypothesized causation to optimize some function, you've won, regardless of the metaphysics.

Even in the context of physical processes, like the biological processes Gwern mentions, the notion of "establishing causality" is much more of a regulatory artifact than anything else.

You might call this the machine learning approach (optimize some objective function, regardless of mechanism) as opposed to the statistics approach (generate some measured claim about a process itself).

</comment><comment><USER20> Statistics serves as a tool to overcome our cognitive biases. But what if these biases are at the center of learning?

Take for example the Gambler's Fallacy where a player believes she can predict the outcome of a coin toss with greater certainty than is possible. Obviously she cannot. But if I had to design a Machine Learning algorithm, I would certainly want it to always assume that a pattern existed. That way, if the data were predictable, the algorithm would be able to take advantage of it.

<comment><USER21> A smarter algorithm would refrain from betting on the coin.

<comment><USER20> That's great if you know a priori that you cannot predict the outcome of what you are betting with.

<comment><USER22> I don't think it would be necessary to know _a priori_ that something is unpredictable; failing to find a pattern after some number of observations should allow an algorithm to determine that the event is not predictable by that algorithm.

<comment><USER23> The trouble is that you can't reliably distinguish random data (or data determined by factors you're not considering) from patterned data.

<comment><USER24> How do you think random number generators are tested?

</comment></comment></comment></comment></comment></comment><comment><USER25> Correlation is due to one of the 3 C's, coincidence, confounding, or causality. I just like the 3 C's.

</comment><comment><USER26> Another cracking article from gwern.

For those wondering, that snazzy looking `choose` combinatorial function is from clojure, and I assume other lisps:

[http://clojuredocs.org/incanter/incanter.core/choose](http://clojuredocs.org/incanter/incanter.core/choose)

<comment><USER27> No, it's just binomial coefficient. It's common to say "10 choose 2", it's the standard way of reading binomial coefficients out loud. See [http://www.wolframalpha.com/input/?i=10+choose+2](http://www.wolframalpha.com/input/?i=10+choose+2).

</comment></comment><comment><USER28> The problem is that causation is an oxymoron.

The measurement problem is the same as the problem of induction.

Max Planck understood this, read his quotes on matter.

No amount of correlation increases the probability of one event followed by another. Therefore, all scientific analysis is unverifiable. Knowledge of the world is completely unjustified. Not to mention our immense presumption of consistency in world phenomena when we really have no basis for asserting uniformity of nature. Read Hume on causality.

"Belief in the Causal Nexus is superstition" \- Wittgenstein

If knowledge requires certainty, and our method of determining certainty is an infinite regress of reduction, it is like a recursive function which never returns a value.

Logic is inherently a comparison operator, and if logic is the only mechanism available to us, we are trapped in a purely relational analysis and it is impossible for the mind to conceive of certainty beyond a non-reasonable emotional preordained state of knowing. A feeling so powerful it is personally indubitable.

The premise of a singularity is invalid since it is inconceivable and the idea that causation is at any time inferred via comparison is fundamentally incomprehensible. If the premise is unclear then the argument is invalid.

If I say that an airplane functions on fairy dust and you make an argument about propulsion and lift, and you claim that my assertion is wrong because I have not properly performed a rational reduction and that a detail I do not understand could invalidate my entire theory, well guess what neither of us is technically correct to any degree since the test for invalidity applies equally to both of our assumptions. The concept of proof is also an oxymoron.

Hence, the very concept of phenomenal causation is an oxymoron. The notion of knowledge about the world is a disguise for a coping mechanism based on hope and desire.

<comment><USER4> I do agree with you that science doesn't show causation, but I think your interpretation of science is incorrect:

> Therefore, all scientific analysis is unverifiable.

I disagree. It's verified by experiment. (Here I'm using "verify" to mean that contradictions have not (yet) been found by experimental observation).

> Knowledge of the world is completely unjustified.

I disagree. Again, it's justified by experiment. If that's not enough for you, too bad, because that's all we have.

> Our immense presumption of consistency in world phenomena when we really > have no basis for asserting uniformity of nature.

I disagree. This is only asserted insofar as it's 1) useful, and 2) justified by observation.

I don't think you understand the significance of scientific results. They don't say "at a fundamental, irreducible level, this _is_ how <some system> operates." Rather, they're saying something more along the lines of "as far as we can tell, this is a description of how <some system> operates, but it's entirely possible that we're wrong. However, we don't have any data that shows that we're wrong (yet), and because this description is useful for understanding <some system>, we're going to use it."

We don't need to know how a system works at a fundamental level, if understanding it at a less-detailed level is useful. I do agree with you that we can't determine "actual causality", but we don't need to, if we can instead find a bunch of really good correlations. The problem is that people find lots of crappy correlations, and/or can't tell the difference between good and crappy correlations.

<comment><USER28> I agree that we do not disagree.

Your post is simply applying a different definition to the terms I used.

Your reply is a case for righteousness.

I don't think you have considered the implications of the use of such words as "good". I agree with you that experimentation is futile in the absence of ethics.

Why are you presuming common application of the use of the word good and useful? For example, the science behind the atomic bomb and it's usage, was it useful? To whom was it useful, those devastated by the blast or those who set it off?

I urge you to reevaluate your basis for righteousness.

Don't you understand that morality requires certainty?

<comment><USER4> At no point did I touch on morality or ethics. I do not know why you think that I did so.

> I agree with you that experimentation is futile in the absence of ethics.

I do not agree with that.

\</comment></comment></comment></comment><comment>--

Dictionary.reference.com:

> useful: being of use or service; serving some purpose; advantageous, > helpful, or of good effect

> good: having admirable, pleasing, superior, or positive qualities; not > negative, bad or mediocre

I urge you to reevaluate your understanding of the words "good" and "useful". Don't you understand that I'm not talking about morality?

</comment><comment><USER29> Hey Amarok, Image Occlusion addon author here. I'd like to meet you in person (can't reply to the comment in which you mentioned me). My email is in my profile.

</comment><comment><USER30> _Perceived_ correlation is usually not _actual_ causation. That should help explain why not.

</comment><comment><USER31> Disagree with the title. Correlation does imply casuation a lot of the times (especially for simple systems). But not always. Therefore, the caution is not to assume it apriori, but pursue further investigation to confirm or reject it. Even when it is rejected, a lot of those cases result in a third variable being the cause behind the correlated "effect" variables.

<comment><USER32> The problem with using "correlation does not imply causation" outside of a mathematical context is that in math, saying A implies B means that if condition A is true, then condition B is true. In common vernacular, implies means to suggest, hence the confusion.

<comment><USER31> 12 days late but still:

I just realized "Correlation does imply casuation a lot of the times" is a completely meaningless statement that I made (if correlation is not because of causation even some of the times then that's simply 'correlation doesn't imply causation').

What I really had in mind (which I think is correct) is that 'correlation ends up being due to causation a lot of the times'.

</comment></comment></comment><comment> dj-wonk This article starts off with a common mistake.

It is ok to create the 3 categories:

> If I suspect that A→B, and I collect data and establish beyond doubt that > A&B correlates r=0.7, how much evidence do I have that A→B?

> you can divvy up the possibilities as: 1. A causes B 2. B causes A 3. both A > and B are caused by a C

So far so good, but here is the problem:

> Even if we were guessing at random, you'd expect us to be right (at at least > 33% of the time...

No. It is not valid to assume each possibility is equally likely. If you do so, you are bringing your own assumptions to the problem.

If you ever find yourself assuming a distribution, pause and consider testing your assumption.

<comment><USER33> First, keep reading.

Second, when gwern says "you'd expect 33%", he [1] does not mean "the abstract 'we' mathematically expect 33%", but indeed "the generic person-on-the-street has an intuitive belief that we should get 33%". If you check the context I think you'll see this fits.

[1] So far as I know, anyhow.

<comment> dj-wonk @jerf Why did you assume I did not read the article? I read it, and I find the writing to be unclear. Maybe "you'd expect" means what you think; maybe not.

Independent of my or your opinion, (or precisely because reasonable people may disagree over something so simple) the writing is unnecessarily unclear. It would be simple to add a quick note saying, more-or-less, that the author will revisit the point later.

I also disagree with another commenter who says that the later writing clears up this issue. The point about 33% not being a fair assumption isn't addressed head on in the way that I think matters most.

Here is my main point, which was not received in the spirit I intended. Many smart people, even programmers, sometimes incorrectly assume a discrete uniform distribution (e.g. 1/3, 1/3, 1/3 in the above example), probably because they think it makes the fewest assumptions.

I'll put it another way with a related example. Alexa gives Bart a two-sided coin, tells him it may or not be fair, and asks the expected probability of getting heads. Bart reasons as follows: "I know that the coin may be unfair, ranging from, say, 0%/100% to 50%/50% to 100%/0%. By symmetry, for each 1%/99% coin there is a 99%/1% coin. So, in the end, the coin will average out to 50/50, so the best answer is 50%."

Chuck comes along and says, "Bart got it wrong. Any point estimate must be incorrect, because we don't have enough information. The correct answer must be a distribution. Since we know nothing, the best answer is a flat (uniform) distribution ranging from 0% to 100%."

Danielle chimes in and adds "Yes, Bart was wrong, and Chuck improved on the analysis, but Chuck did not go far enough. Both Bart and Chuck assumed symmetry, without justification. Chuck's act of assuming a uniform distribution is tantamount to knowing the process that generated the coin. Does Chuck really know the chances of a 0-10% coin are the same as a 10-20% coin, and so on? No, he does not. Therefore, even answering the question with a distribution is incorrect. The only correct answer is that the probability of heads ranges from 0 to 100%, but no distribution can be given."

Danielle is correct. The author may well know this, but it is not communicated clearly in the article.

I would be willing to wager that many people, when confronted with Alex's question, would answer like Bob or Chuck did.

<comment><USER34> > I also disagree with another commenter who says that the later writing > clears up this issue. The point about 33% not being a fair assumption isn't > addressed head on in the way that I think matters most.

How would you want it addressed? A large part of the article is exactly about how the - to some - "intuitive" idea of 3 evenly split categories is incorrect. E.g. later in the article he writes:

"It turns out, we weren't supposed to be reasoning 'there are 3 categories of possible relationships, so we start with 33%', but rather: 'there is only one explanation "A causes B", only one explanation "B causes A", but there are many explanations of the form "C1 causes A and B", "C2 causes A and B", "C3 causes A and B"…', and the more nodes in a field's true causal networks (psychology or biology vs physics, say), the bigger this last category will be."

Presumably this is why two people separately assumed you did not read the whole article.

<comment> dj-wonk Ok, to move this forward, let's move past the who has read what part, which doesn't seem to be helping. Anyhow, my point is relatively subtle, I think, and getting lost in communication somehow. I get what the article is saying, but I'm not sure that everyone here is getting what I'm saying.

Let me try to elaborate. My points are:

1\. The article does indeed criticize the 1/3, 1/3, 1/3 split, but not for the same reasons that I am.

2\. I would prefer that the 1/3, 1/3, 1/3 split not be used as a prior at all. (Yes, I get that Bayesian priors don't matter that much _if_ you have enough data to update them. The article, on my reading, does not offer a way to get the needed observations, so I think the priors dominate.)

3\. Yes, later, the article dismisses the even split as naive but does not go far enough when suggesting a better way. You see, the "better way" still boils down to the naive assumption of counting bins: this is to say that if we have more bins, then we should have a higher expected probability. In my opinion, the "better way" is still falling into a conceptual trap: this is not a combinatorics problem. For combinatorics problems to work they have to rely on real, countable things, not arbitrary notions of how you slice and dice a problem. Yes, I get that as you add more variables to the DAG, you can certainly say that there are more possible paths through the graph. But we cannot assume that just adding more nodes in the DAG will change reality. You see, the DAG is a mental construct. We should not assume that adding more mental bins (an arbitrary way of slicing reality) will affect the distribution.

Here is an example. Francis tells Greg there are three marbles in a jar (green, red, blue) but does not reveal the distribution, so Greg cannot rightly assume a 1/3, 1/3, 1/3 distribution. But let's say Greg does anyway. Next, Francis tells Greg, "Sorry, I was oversimplifying before; really there are 4 types of marbles in a jar (forest green, lime green, red, blue)." What should Greg do? Update the prior to 1/4, 1/4, 1/4, 1/4? Well, that would be inconsistent, since 1/4 (forest green) + 1/4 (lime green) does not equal 1/3 (green). Should Greg suggest a 1/6 (forest), 1/6 (lime), 1/3 (red), 1/3 (blue) split? Nope. My point is this: down that road lies arbitrariness and contradictions.

Hopefully this helps explain what I'm talking about. I like the article, but I don't want people to get into the habit of assuming a distribution. I also don't like people counting up concepts that are arbitrary and turning that into a distribution.</comment></comment></comment></comment></comment>


What every programmer needs to know about game networking (2010) - <AUTHOR> http://gafferongames.com/networking-for-game-programmers/what-every-programmer-needs-to-know-about-game-networking/ <comment><USER2> TIL that some programmers have to be taught what others learned first hand by encapsulating IPX packets inside TCP/IP[1] packets inside PPP packets modulated over the POTS network.

(Not Kali, the OTHER one. :) )

Oh, and there used to be different types of game engines. You used to be able to SEE whether or not your engine was interpolating missing data, whether it was maintaining state on your end, their end, or both, etc--by observing how your opponents' ships would react when your mom picked up the phone in the kitchen.

Descent had a pretty darned good method.

Alice and Bob play a game. From Alice's point of view, the visual of Alice's weapon fire striking Bob's ship is rendered on her machine based on her machine's understanding of the location of his ship. But the AUDIO of weapon impact is "rendered" on Bob's end. So if you see impact, it means nothing--but if you hear impact, you know that 1) your fire collided with Bob's ship on his end and 2) it happened approx. n/2 seconds ago.

This resulted in a kind of meta-game wherein the lagscape was just part of the terrain. There was a 'general relativity' component to it: since neither side was master or slave... No point of view was privileged[2]. Alice had to keep two gamestates in mind at all times, the local to protect her hitpoints, and the remote to reduce his hitpoints.

In my opinion, both the "sound/visual disconnect" and the "no privileged POV" were necessary ingredients in order for the lag to be "part of the landscape"\--ie, fair and fun.

[1]: _Wait, did we already use UDP at that point? I can 't recall..._

[2]: _Some pieces of gamestate were given to the client that started the game or the client with the highest FPS. I think the reactor countdown was finalized in one client, for example, resulting in the famous "3... 2... 1... 0... ... ... ... ... BOOM!"_

<comment><USER3> I miss playing Warcraft 2 over Khan.

<comment><USER2> "Your sound card works PERFECTLY!" ... "It doesn't get any better than THIS!"

I was Hobbes in those days and there are non-zero chances that we have played together.

</comment></comment></comment><comment><USER4> _Distributed Virtual Reality – An Overview_ , Bernie Roehl, 1995: [https://ece.uwaterloo.ca/~broehl/distrib.html](https://ece.uwaterloo.ca/~broehl/distrib.html)

 _Characteristics of UDP Packet Loss: Effect of TCP Traffic_ , Hidenari Sawashima, Yoshiaki Hori, Hideki Sunahara, Yuji Oie, 1997: [http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM](http://www.isoc.org/inet97/proceedings/F3/F3_1.HTM)

 _I Shot You First: Networking the Gameplay of HALO: REACH_ David Aldridge, 2011: [http://www.gdcvault.com/play/1014345/I-Shot-You-First- Networ...](http://www.gdcvault.com/play/1014345/I-Shot-You-First-Networking)

</comment><comment><USER5> 403 for me as well.

Web archive to the rescue:

[http://web.archive.org/web/20141118054424/http://gafferongam...](http://web.archive.org/web/20141118054424/http://gafferongames.com/networking- for-game-programmers/what-every-programmer-needs-to-know-about-game- networking/)

</comment><comment><USER6> This article ends right when you expect him to start telling you everything you need to know about game networking.

It would be better titled 'a brief history of game networking', and funny enough on the site that actually has everything a programmer needs to know about game networking this article has that subtitle. And that site is made by the author as well!:

[http://gafferongames.com/networking-for-game- programmers/](http://gafferongames.com/networking-for-game-programmers/)

Cool hn discussion with game network programmers here:

[https://news.ycombinator.com/item?id=8399767](https://news.ycombinator.com/item?id=8399767)

<comment><USER7> Web archive: [http://web.archive.org/web/20141015220426/http://gafferongam...](http://web.archive.org/web/20141015220426/http://gafferongames.com/networking- for-game-programmers/)

</comment></comment><comment><USER8> (2010)

Thus far I'm having a pretty easy time ensuring that my games are deterministic functions of sequences of user inputs. For people who don't want to do that, this talk about Halo multiplayer is pretty informative: [http://www.gdcvault.com/play/1014345/I-Shot-You-First- Networ...](http://www.gdcvault.com/play/1014345/I-Shot-You-First-Networking)

</comment><comment><USER9> This is a great series of articles - I had searched for such information a month or so ago, as the subject was made interesting and relevant to me due to being a Beta-backer of Elite Dangerous, which is using a combination of client-server plus P2P networking whilst playing in the "All-Online" mode (multiplayer), and just uses client-server in the "Solo-Online" mode.

I have my doubts that Frontier Developments will be able to get the P2P side of the netcode working smoothly and properly by the 16th December when they officially release the game. I'm also having doubts that model will ever work smoothly for them, but, time will tell, and good luck to them for all that.

</comment><comment><USER10> What is the client sending to the client exactly? "I pushed down W" and "I released W" or send "I am holding down W" every X ms?

If the latter, how would you prevent the client from spamming it more than every X allowed ms? I'm thinking it would build up a queue if commands came in too fast but not sure how that would be handled exactly. Also, even with a queue someone spamming would gain an advantage since it would bridge the latency.

Example: If the client sends "move 1px to the right" every 20 ms and someone manually started sending it in every 5 ms instead, it would still only move every 20 ms on the server. However, wouldn't this still be better than sending it every 20 ms from the client since latency and other things (slow processor?) could mean the real interval is more than 20 ms?

<comment><USER5> And there you have it: speedhack :)

This is a topic of where to process commands (client vs server), what data to send, which restrictions to impose, what rules to enforce, etc. Give client too much say in the simulation and it can ruin the experience of others, leave it all on server and enjoy your lack of speed.

</comment></comment><comment><USER11> [https://developer.valvesoftware.com/wiki/Source_Multiplayer_...](https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking) The technology from Valve referenced in the article. Goes into a bit more technical detail about the prediction.

</comment><comment><USER12> People who are interested in this may also be interested in Mauve's posts on rollback-based netcode for fighting games, and how he handled some issues related to that:

[http://mauve.mizuumi.net/2012/02/22/investigating- rollcaster...](http://mauve.mizuumi.net/2012/02/22/investigating-rollcasters- timing-issues/)

[http://mauve.mizuumi.net/2012/03/08/how-i-didnt-clip-the- sou...](http://mauve.mizuumi.net/2012/03/08/how-i-didnt-clip-the-sound- effects/)

[http://mauve.mizuumi.net/2012/07/05/understanding- fighting-g...](http://mauve.mizuumi.net/2012/07/05/understanding-fighting- game-networking/)

</comment><comment> 97-109-107 Whover likes this would also enjoy [http://www.gamasutra.com/view/feature/131781/the_internet_su...](http://www.gamasutra.com/view/feature/131781/the_internet_sucks_or_what_i_.php) \- The Internet Sucks: Or, What I Learned Coding X-Wing vs. TIE Fighter

<comment><USER13> I found great joy in reading about Quake3's networking model [1]. It is build on top of the assumption that the network is unreliable: most of the packets will be dropped and those that make it are not in sequence.

[1] [http://trac.bookofhook.com/bookofhook/trac.cgi/wiki/Quake3Ne...](http://trac.bookofhook.com/bookofhook/trac.cgi/wiki/Quake3Networking)

<comment><USER14> It's interesting how similar that approach is to the one taken by Mosh:

[https://mosh.mit.edu/#techinfo](https://mosh.mit.edu/#techinfo)

</comment></comment></comment><comment><USER15> Site is back up. Sorry for the downtime. Thanks site5.com for pulling the plug on my site arbitrarily! -- gaffer.

</comment><comment><USER16> This link now 403s for me.

<comment><USER17> Entire server is 403'd: [http://www.gafferongames.com/](http://www.gafferongames.com/)

Rather puzzling. Traffic strategy?

</comment></comment><comment><USER18> Link is down ....

</comment><comment><USER19> I too browse reddit.

Front page post on reddit from 19 hours ago: [http://www.reddit.com/r/explainlikeimfive/comments/2n9rbj/el...](http://www.reddit.com/r/explainlikeimfive/comments/2n9rbj/eli5_how_doom_1993_had_online_multiplayer_on/)

Top comment: [http://www.reddit.com/r/explainlikeimfive/comments/2n9rbj/el...](http://www.reddit.com/r/explainlikeimfive/comments/2n9rbj/eli5_how_doom_1993_had_online_multiplayer_on/cmbvd4w) with content:

> Here's a longer writeup I found: [http://gafferongames.com/networking-for- > game-programmers/wha...](http://gafferongames.com/networking-for-game- > programmers/what-every-programmer-needs-to-know-about-game-networking/)

<comment><USER20> What's the point of this comment? Some of us don't browse reddit.</comment></comment>


How United Onboards New Users (design) - <AUTHOR> http://www.useronboard.com/how-united-onboards-new-users/ <comment><USER2> I guess we're going to see this posted every day until United falls out of the headlines?

[https://news.ycombinator.com/from?site=useronboard.com](https://news.ycombinator.com/from?site=useronboard.com)

It's interesting that every account that has posted this story submits multiple times a day.

<comment><USER3> It was painful though

</comment></comment><comment><USER4> I was happy to find that after the 149 slides I only needed to click my browser's back button once

<comment><USER5> Yep, I was sure to check that a few slides in heheh.

</comment></comment><comment><USER6> Oh the irony, criticising the mobile app about its design on a painful to use on a mobile phone slide show.

<comment><USER7> You should ask for a refund.

<comment><USER8> Not quite understanding this. The United app is also free.

And at least it has buttons that are large enough to touch with my finger without encroaching into some other div causing an unexpected zoom.

<comment><USER9> I laughed and then tried to invite you on HN mobile... Joke's on us I guess.

</comment></comment></comment></comment><comment><USER10> Curious why the user didn't click back instead of on the home button around slide 70-75. It would have saved them about 50 slides or so and 10-15 minutes of real time. They clearly saw the back button later around slide 120. The micro-pessimization of looking at the account creation flow before attempting to book a flight also seemed somewhat unnecessary.

While it is not an excuse for the poor design, I am not sure _why_ you would search award travel if you don't have any points/miles to redeem nor even have an account. They even leave the boxes unchecked by default. Clearly a better UI would have process equally streamlined both for new users and for customers with United miles/points to spend.

<comment><USER11> Right, along with complaining that 'Show Nearby Airports' doesn't work when you've forbidden location services.

<comment><USER12> The question asked was "Allow United to access your location even when you are not using the app?". It's not clear that answering "no" would prevent the app from accessing location when you _are_ using the app.

<comment><USER11> That's a failing of iOS, not the app - that is a system dialog that allows an app to ask for foreground only location services, or background location services (and this is the dialog presented, no customization allowed).

</comment></comment></comment></comment><comment><USER13> This is exactly why whenever I have to deal with something more complicated than Skype credits, I use a desktop.

Typing a lot of information in on a smartphone in your shitty app = waste of my time and I'm not taking that risk again after dozens of tries over the years.

</comment><comment> m-j-fox I've seen worse. The CVS Android app. In order to enter my birthday (why do you need my birthday CVS? You won't let me order alcohol in your app anyway. For those who don't know, CVS is a popular chain of liquor stores where you can also get prescriptions filled. Like a small, neighborhood Costco.) I had to click a little back arrow month by month 40 years to get to when I was born. Twice.

<comment><USER14> If only you were born yesterday.

</comment></comment><comment><USER15> I'm a frequent flyer with status on United Airlines.

This "onboarding" demonstration is an eye-opener for me. While I didn't experience any of the issues that this slideshow demonstrated, I now realize that my experience and knowledge with flying United helped a ton to start working with this app. On a related note, I also did not share the same general resentment for what United did on the recent flight to Louisville. Not because of lack of empathy, but again because of experience both good and bad, and also because if I were to be the doctor, I would know my options for getting to my destination in time. After a while, you start forgetting that other people do not share that level of experience and knowledge, and to make this relevant for Hacker News: this is an issue that affects me as a developer too.

That being said and in United's defense, this app has incredibly powerful functionality that's not mentioned at all here. This app allows you to scan your passport and get it "pre-approved" for boarding an international flight, something that can save you A LOT of time at the airport. Furthermore, in case you are affected by "IRROPS" (delays and cancellation) the app can offer you alternatives and rebook you immediately. And there's a lot of other helpful stuff this app can do. Granted, it is still limited in what it can do, and sometimes it can't help you out, but it is a huge timesaver if it does. In short, the United iOS app can be an incredibly powerful travel companion, but it still requires a user that is more educated about air travel than the average Joe.

So as a developer, I do agree that the onboarding experience for inexperienced users should improve and I will look at the onboarding process in the applications that I write. But at the same time, as a frequent flyer, I hope any changes do not constitute merely "dumbing things down" at the expense of the powerful functionality where this app really shines today.

</comment><comment><USER16> This is probably a result of having a huge team of disempowered developers, each working on delivering a tiny slice of functionality, with a product owner lacking any kind of organizational clout. Most likely the app team is beholden to a much larger, powerful and entrenched backend division that dictates what APIs they have available and when they can use them.

<comment><USER17> Most likely this was outsourced to shop that doesn't care at all about anything but checking off "requirements" in a contract.

<comment><USER18> That sounds about right.

</comment></comment></comment><comment><USER19> It's good that someone is pointing out these basic mistakes. It's really easy to see which companies go the extra mile.

</comment><comment><USER20> Slide 123 is hilarious! It's exactly how I feel after I've lost pages of information due to a badly-designed flow.

[http://www.useronboard.com/how-united-onboards-new- users/?sl...](http://www.useronboard.com/how-united-onboards-new- users/?slide=123)

</comment><comment><USER21> I gave up ~35 slides in.

The app wants to notify you to give you updates on flights. It's a handy feature. I've received Delta notifications 5-10 minutes before it's announced at the gates.

The "Favorite" prefilled locations are United's two major hubs. A significant amount of their travelers start out or end up at those two locations, so it is saving a lot of users time.

The location and notification messages they complain about are Apple's messages. United I'm sure would want to say something more informative.

This review is apparently based on a 10 year old's understanding of traveling and tech.

<comment><USER22> I think the issue isn't that it's asking for notif permission at all, it's that it's asking at the wrong time. Most mobile app developers and designers should know that you wait till the moment before it's needed before asking the user, which for this case would probably be _after_ you book a flight successfully. This is for the good of both the customer and the company. The customer actually knows what it's being used for, and the company successfully establishes a communication method with the customer for more engagement. You don't ask for notif permission right at the beginning. That's an amateur app developer mistake.

If "Favorite" is United's favorite, maybe it should be renamed "Popular". The term "Favorite" is usually used to indicate something the user set themselves.

As for location and notif permission prompts, if as an app developer you think the message isn't informative, typically you explain in your words in a custom UI prompt in your app (about why you want to send notifs) first, and when they click yes, you make the system prompt. Two steps, yes, but higher allow rates for the company, and more information for the customer. I think it's a good practice.

I don't think the review is based on a 10 year old's understanding of traveling and tech. I think it's quite valid overall.

<comment><USER23> I am 100% in agreement with you.

(I'm the person who made the slideshow)

:)

<comment><USER8> Is it your website as well? If so, the mobile experience needs a few tweaks. Especially when viewed in portrait orientation.

The arrows to go back and forth are tiny, and very close to the slide window. Which, when your finger overlaps onto, jumps around and zooms. I thought maybe swiping would work, but it does not.

The top menu links are pretty tiny as well.

I thought it worth mentioning because the amount of work to fix it seems pretty low.

<comment><USER23> Yes, it's my website and the mobile experience completely sucks. That's the highest item on my to-do list for site updates.

</comment></comment></comment></comment></comment><comment><USER24> Well this brings back memories of sitting in their plane trying to become a milageplus member. After filling in the whole form, I find that I cannot submit because the app cannot use the airplanes wifi. I know my bad for assuming that if united let you browse their webpage for flight status that surely they would let their mobile app work over the wifi as well. Worse I assumed when I got home, I would submit the form and be done but no, the session timed out and all my data entry was lost.

</comment><comment><USER25> I tried the United app last year. I was in Austin, about two hours before my flight.

I signed in and expected my flight info. Nope. Instead, they asked for my confirmation number. Remember, they know who I am and the current time. Why else would I open the app?

I checked the "airport maps" page assuming it would show Austin. Granted, as my home airport, I probably don't need a map but maybe my destination? Nope, Cleveland.

Delete.

</comment><comment><USER26> Given United Airlines' consistent poor service and arrogance for many years, it's time to dismiss the company, re-accommodate and onboard their employees (especially executive managers) to other businesses, perhaps new airlines started by people like Richard Branson.

</comment><comment><USER27> The app sucks, but apparently the critique is from someone who has never flown before.

<comment><USER12> How so?

<comment><USER28> Because everyone who has flown knows you need to choose these basic things when flying: Departure, Arrival, and two Departure Dates. So it's not that bad if the app won't hold your hands and tells you that you need these basic things.

<comment><USER29> People who fly, like, once or twice per year (leisure travelers or travelers visiting friends and relatives (VFR)), will not know these things. There are many, many people who have never flown on a plane, too.

They make up a significant portion of total passenger volume.

Also, every other airline app _gets this right._

</comment></comment></comment></comment><comment><USER30> Funny. Although I didn't see the option to specify which hospital to take you to after United has dragged you off the plane when the flight is overbooked.

</comment><comment><USER31> OK the UA iOS app definitely has problems.

I wonder how it stacks up against other airlines' apps?

How do the airlines' apps stack up against 3rd-party apps like Kayak?

<comment><USER8> They all suck. It's a hard space to get right, especially to get right across 4 different UIs (desktop, mobile web, Android, iOS). And, pieces of the functionality across even more UIs, like the airport kiosks.

It's basically a thousand tiny cuts. For example, if the user selects an outbound flight, then changes the return date to one day earlier...it may have invalidated the pricing on the outbound flight.

Similar if they pick flights and then input a birthdate past a certain range. They are now an "unaccompanied minor". Prices are different, and they can't be on any connecting flights...nonstop or direct only.

Just two examples, as mentioned, there's a lot of this sort of complex business process.

And, most invested in their backend services during the "desktop mostly era". So a fair amount don't have reasonable services layers...the business rules are locked up in some layer that isn't easily exposed to all 4 UIs. Most are just now coming out of that barrier, piece by piece. Legacy is a bitch.

</comment></comment><comment><USER29> There is a lot of talk on United's folly out there. This is definitely worth a read.

</comment><comment><USER5> Surprised it made direct mention of the recent event.

</comment><comment><USER32> Probably A/B tested for maximum customer annoyance /s</comment>


Is Alexa spawning companies in the same way that Slack/Facebook Messenger are? - <AUTHOR> It seems like there has been an explosion of companies creating text bots for messaging platforms. Amazon's Alexa/Echo platform is similar in concept (Natural Language interface into a streamlined service), but I haven't a similar explosion. Am I missing something? Does anyone know of any companies building specifically for Alexa (the Alexa fund companies I've seen are all hardware focused)? <comment><USER2> Check out some of the companies that have presented at the NYC Amazon Alexa meetup - www.meetup.com/NYC-Amazon-Alexa-Meetup/

Often, it's not that companies are building only for Alexa, but that the Alexa integration is powering a new product offering they weren't able to fully provide previously.</comment>


New York City turns to dry ice to control its rat population - <AUTHOR> https://www.marketplace.org/2019/05/23/new-york-city-turns-to-dry-ice-to-control-its-rat-population/ <comment><USER2> I am curious if this really works. I assume rats will outsmart pest control and will just compensate their losses by breeding more and somewhere else.</comment>


Ask HN: Is there a "solve my programming problem" website? - <AUTHOR> StackOverflow isn't suited for "here's my code, what's wrong with it" questions, but is there a site or forum that is? <comment><USER2> Codementor.io solves for exactly this problem

</comment><comment><USER3> If the code works and the goal is to get feedback: [https://codereview.stackexchange.com/](https://codereview.stackexchange.com/)

</comment><comment><USER4> That could be an interesting take on a marketplace, where a fee is set and people solve larger problems for payment.

</comment><comment><USER5> Are there similar services or sites for guidance or mentoring on devops and system administration problems? Or could the options already suggested here: [https://www.codementor.io/](https://www.codementor.io/) and [https://www.airpair.com/](https://www.airpair.com/) also be used in this way.

</comment><comment><USER6> Google - a big part of it is framing the query to get the responses you need.

</comment><comment><USER7> [https://www.fiverr.com/](https://www.fiverr.com/) it's not free though

</comment><comment><USER8> For free or for compensation? For compensation, maybe AirPair.com?

</comment><comment><USER9> algorithmia.com, it has codebounty where you can pose your problem

</comment><comment><USER10> StackOverflow for sure. Not sure why you disagree.

<comment><USER11> StackOverflow strongly prefers "Short, Self Contained, Correct (Compilable), Example"s [sscce.org] that showcase a specific problem.

If you go and paste several files with a few hundred lines of code, you're unlikely to get any help. People will not usually make the effort to set your project up locally (because you likely didn't paste the project configuration), debug it, and send you the diff to the fixed project.

But more importantly, any answer to your question is hard to transfer to problems other people have. It can't be used as a reference or solution repository anymore, because there's so much distraction around the essence of your problem.

The help center also has an entry about this: [http://stackoverflow.com/help/mcve](http://stackoverflow.com/help/mcve)</comment></comment>


How a few lines of code greatly improved a crucial part of UX - <AUTHOR> http://blog.self.li/post/23668063348/few-lines-of-code-improved-ux

<comment><USER2> Another even easier and quicker way to greatly improve UX in many products is better copywriting (possibly by hiring an actual copywriter).

Quite frankly, I know I'd be quite puzzled if a message popped up out of nowhere telling me "You might be logged out, please, back up your content and refresh the page!".

What do you mean by I /might/ be logged out? Am I logged out or not? And why would I be logged out? What does this have to do with anything anyway?

And what on earth am I supposed to do to "back up my content"? What does this even mean?

And what is this "refreshing the page" business all about?

To be honest, being a developer myself, I would actually figure out what you actually mean. But I can guarantee you that 99.9% of the people out there won't have the faintest clue of what you're talking about. They'll just do what they always do in this case: they'll click OK, or Cancel, or whatever they think is going to make the annoying message go away.

<comment><USER3> > what is this "refreshing the page" business all about?

Look at a calender and note the year. Now realize that NSCA Mosaic was released in 1993. Eternal September was in 1993 as well. That is almost 20 years ago. Do you really think that 'refreshing the page' is beyond 99% of peoples knowledge?

That error message does need cleaning up but your target audience are most likely not fetuses, so don't treat them like they wouldn't be able to put their pants on in the morning if you didn't tell them how to in excruciating detail, despite doing it every day, when they visit your site.

Oh excuse me, a 'website' or a 'site' is a virtual location accessed through a 'web browser'. A 'web browser' is a program that is used to access 'web sites' over an interconnected series of networks called 'the Internet'. I know you said you're a developer and may be able to figure this out for yourself but as you said 99.9% of people wouldn't, I wasn't going to take that chance. Was it too advanced for you?

Also depending on the issue 'might be logged out' may be completely correct. Obviously something has occurred that is making part of the webapp believe something has gone wrong, but it probably still has "Welcome [username]" or "Logged In" at the top of the page. Reloading the page is really the only way to verify something really bad is going to happen.

</comment></comment><comment><USER4> I agree that there are better ways of solving the problem. Getting back to the root cause would be good. But, over the years I've seen too many cases where users have to live with a poor experience while the development team is working on the "right way." It's nice to see the experience of the user taken seriously.

</comment><comment><USER5> This strikes me as odd: "After a complaint from one of our users [...], we realized that the very same thing could happen to someone else"

One user complains about something that might have happened by his own doing and the OP starts implementing a feature, that not circumvents the end result (can't save the content) but notifies the user that the software is broken somehow.

Anyway, I'm genuinely interested in the following:

What are these "universe glitches", when one gets logged out by the browser? Session Cookie vanishes? Server restart invalidates some kind of access token?

<comment><USER6> That's what I was wondering, they solved user's problem, but not their problem.

To me, it sounds like they might be expiring the session due to inactivity, while the user is actively writing something.

EDIT: Typo

<comment><USER7> Yep, that's probably exactly what's happening, I've had it happen too.

And the solution is the ping that he proposes. So if that was the problem, the ping is resetting the session cookie every 60 seconds and the problem should go away.

One thing I would add is a logging feature that tracks how many times this actually happens (is happening) so they have metrics to back up any decision in the future.

<comment><USER6> Then why isn't this ping _already_ part of the auto-save feature, and not an extra request.

</comment></comment></comment></comment><comment><USER8> We did something similar while building the CMS for the Yahoo! home page. Now the way auth works (and should work), is if you try to access a restricted page and you're not logged in or don't have the necessary privileges, you're redirected to a login page. This is all done automatically by apache and your back end app has no control or knowledge about this.

On the front end, auto-save is done using an XHR call from JavaScript. If we detect a 302 response from the server, and the Location: header was set to a known login pattern, we'd inform the user of this, and allow them to log in in a new window/tab.

On login, the new window/tab redirects the user back to a page in our app. The page in our app does two things. 1\. It calls window.opener.someCallback() 2\. It calls window.close()

The callback function then makes a call to the server to fetch a new csrf token (you do use csrf tokens, don't you?) and then re-attempts the save.

This reduced user frustration significantly.

I do not know if this app is still in use.

</comment><comment><USER9> I don't think this is a good solution. If you have 1000 users logged in you have a 1000 extra requests a minute just checking the status of the user. This is to solve an edge case.

My company had a problem with clients being logged out after 20 minutes. You would think this is enough time but some people would spend hours editing content without pressing save. We implemented 2 solutions. The first was an auto-save. This pinged the server and removed the primary cause of the issue. The second thing was a warning which triggered after 15 minutes of inactivity. If they pressed OK some data would get sent to the server and the timer would start again. We are not concerned about random "universe glitches"

I would have thought a better way would be to add a listener for form submissions. Check if user is logged in. If so - send data. Otherwise login form can pop up or data can be put into a session or something and reappear after the client logs back in.

This means the user doesn't need to know how to copy and paste.

<comment><USER10> IMO if you have 1000 users logged in, an extra 1000 requests/minute is in the noise. And I tend to be paranoid about such things (e.g. I would add a small random number to the 60s delay to avoid synchronization).

</comment></comment><comment><USER11> Here's another way to deal with this. Upon each successful am-i-logged-in call, the server returns a relogin-as-user-X ID back to the browser (this can continue to be the same cached value or have timestamp/TTL if you want). If since the last call to am-i-logged-in, the session is lost for any reason (server reboot, client changed IPs), the browser will simply post this ID back to the server, which will perform the authentication and relogin.

To make this secure, make am-i-logged-in no-cache, don't store the ID in cookies/localStorage, encrypt/decrypt the relogin-as-user-X only on the server-side, and timestamp it with short TTL.

If you are already able to detect that the user has logged out unintentionally and just minutes prior to that you knew they were logged in, then you might as well just log them back in again. Of course, if the user logs out intentionally in one tab, then do not log them in automatically in the other tab. Simply stop returning any valid relogin-as-user-X IDs.

</comment><comment><USER12> Why not just give a random session key or something equivalent? That way, the user can post to /save/?key=<userkey> to save, regardless of sessions.

</comment><comment><USER13> I think the probleme is elsewhere, maybe some cookie or memcached problem, I never heard of browser logging out people, maybe the cookie expire time is incorrectly set.

<comment><USER11> Browsers do not log out people, but servers do. My web host randomly drop all sessions once a week. It is cheaper/easier for me to build auto-re-login script than deal with a new host or pay current host to debug my server issues.

<comment><USER6> Sounds like your webhost is deleting PHP's file based session store.

</comment></comment></comment><comment><USER14> Another way to make sure work isn't lost is to save it in browser storage when available. This would also work in case connection is lost.

<comment><USER15> Yes, the proposed solution sounds a lot like working around a shortcoming of the design with a hack (that needs constant connection and, if I didn't miss the part, not actually saves the content, just displays a warning.

Why not save the content as a 'semi-anonymous' user, together with the user's session cookie and on the next login tell there's some ghost session lurking around and whether it should be kept?

</comment></comment><comment><USER16> Does the auto-save function they've already implemented not return a value representing failure? Is that value ignored?</comment>


Tell your House Rep to vote against letting the FBI access web history - <AUTHOR> https://www.privateinternetaccess.com/blog/tell-your-house-representative-to-vote-against-letting-the-fbi-access-internet-history-without-a-warrant/?aff=TELL37 <comment><USER2> Does not go nearly far enough. The FISA needs to be scrapped unless they figure out a way to have actual accountability and prevent misuse.</comment>


Will they be laughing at this in 42 years? - Imagining the tech world in 2050 - <AUTHOR> http://www.news.com/8301-13953_3-9933345-80.html?tag=nefd.top

<comment><USER2> I for one, will welcome our robot overlords. Did I say overlords? I mean protectors.

Joking aside, I have no idea of most of the ideas as I'm no expert in them.

I DID find it striking that not one mentioned climate change or global warming and its impact on us and the information technology needs we will face to meet the challenges of an environmentally changing world.</comment>


The many lives of John le Carré, in his own words - <AUTHOR> http://www.theguardian.com/books/ng-interactive/2016/sep/03/tinker-tailor-writer-spy-the-many-lives-of-john-le-carre-in-his-own-words <comment><USER2> Not complaining, just observing: this is native advertising done really well.

<comment><USER3> It's called publicity. Books have been serialized and excerpted in periodicals for hundreds of years.

And yeah, to expand on "done really well" it benefits all parties: the periodical, the book, and the reader

<comment><USER2> They could have added a small notice at the top that somehow explained that all of this was a about a newly published book and that there was a link to buy it at the bottom.

In my mind this would have given them (The Guardian) more credibility in the long run. But I guess they're a bit desperate for cash.

<comment><USER4> Even though you're downvoted, you're correct.

Also, le Carre's books aren't that well done, imho.

To parent: Appeal to tradition is dangerous.

</comment></comment></comment></comment><comment><USER5> My favorite le Carré book, 'A Perfect Spy', is largely based on his unconventional early life. I'm looking forward to reading his memoir.

</comment><comment><USER6> Can't stop reading it

</comment><comment><USER7> I never had the desire to read le Carré, even after seeing that movie that came out. But after reading this, I surely do.

</comment><comment><USER8> Just realized one of my favorite writers hangs out at the same park as I do... OMG! :D</comment>


Utility lib to generate tokens for MapKitJS in Rust - <AUTHOR> https://github.com/Schr3da/mapkitjs-token-gen <comment><AUTHOR> This utility library provides wrapper functions to generate MapKitJS valid jwt tokens for your rust program.</comment>


Ask HN: What does Privacy by Design exactly mean? - <AUTHOR> <comment><USER2> It's a framework of ideas, so the "exactly" is hard to pin down if you're asking about how to do it. At the core, the idea that Privacy (like security and many other concerns) should be considered from the first steps of a product instead of being retrofitted after the design process. The latter leads to questions like "How can we make practice X more secure/more private?", when the better answer might be "X isn't a good idea, how can we achieve the same goal differently"

I quite liked Heather Burn's article fro Smashing about it: [https://www.smashingmagazine.com/2017/07/privacy-by- design-f...](https://www.smashingmagazine.com/2017/07/privacy-by-design- framework/) (written in the context of GDPR)

<comment><AUTHOR> Thanks for the reference. I'm wondering how this principle may be applied in the field of AI where user data is often an essential asset. In particular, what are the AI companies that can legitimately claim to be private by design?

</comment></comment><comment><USER3> To me, it means that the thing being designed is private in a way that can not be meddled with.

I.e. encrypted messages, that a company couldn't give to the government even if they wanted to.

In practical terms, this might look like building a system where you purposely don't store or give the ability to access personal content, instead relying on systems that are either local (i.e. not online), encrypted or otherwise fully obfuscated.</comment>