When Does Intelligence Peak? - <AUTHOR> https://blogs.scientificamerican.com/beautiful-minds/when-does-intelligence-peak/ <comment><USER2> This[1] is the paper they quote. The paper is pretty confused, mostly by their own fault. For example:

 _Recent evidence shows that whereas short-term memory for names and inverted faces peaks around the age of 22 years, neither short-term memory for faces nor quantity discrimination peaks until around the age of 30, a fact difficult to assimilate into the fluid- /crystalized-intelligence dichotomy (Germine, Duchaine, & Nakayama, 2011; Halberda, Ly, Wilmer, Naiman, & Germine, 2012). Whether face memory and quantity discrimination are exceptions to the fluid/crystalized rule or represent more systematic and previously unrecognized patterns of age-related difference is an open question_

First, there's no "fluid-/crystalized-intelligence __dichotomy __" , since the two factors are highly correlated with each other, to the point where psychometricians prefer to derive the higher-order order factor, g.

Second, clearly the authors will be very confused by results like this if they don't actually calculate g-loadings, and at the very least correlate the vector of g-loadings with the vector of age declines.

Now compare the above paper with a paper studying basically same question, but using proper factor analysis[2]. In particular, if you're low on time or background, look at the graphs in Fig 2. Clearly, being able to separate g-loading from group factors and test-specific measurement error gives us much clearer picture of age-related decline.

[1] - [https://sci-hub.tw/10.1177/0956797614567339](https://sci- hub.tw/10.1177/0956797614567339) [2] - [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3637652/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3637652/)

</comment><comment><USER3> FWIW I turned 55 this week and feel smarter and more capable than any time before in my life.

I feel I write better code, and faster, than before. I understand articles and books better.

I can't ski, bike, run, or backpack like I used to in my 20s. This is not a huge surprise.

<comment><USER4> Lucky you!

I feel like my coding peak was at 35 and is slowly declining ever since.

I'm slowly getting better in dealing with people though.

<comment><USER5> I don't believe that "coding" is a single axis of quality, either. I'm now better at some types, and worse at others.

When I was young, I was much better at reading or writing complex functions that worked correctly. As I get older, I'm much less able to, but I also wouldn't want to even if I could.

I don't think that's just sour grapes. I don't want to inflict complex code on my coworkers, either. And there's many times (when I'm tired, when I'm fixing an emergency bug, etc) that I'm not at 100% mental capacity and I value not having to work right at my brain's complexity limit.

16 year old me would probably not have believed that.

<comment><USER6> It is orders of magnitude harder to write a "simple" solution. Developers should not be proud of complexity at any age.

<comment><USER5> Are you suggesting one should be proud of something just because it's harder?

<comment><USER6> No, one should be proud of creating an elegant solution.

</comment></comment></comment></comment></comment></comment><comment><USER7> > _processing speed and short-term memory for family pictures and stories peak > and begin to decline around high school graduation_

I am skeptical that "short term memory for family pictures" is a discrete cogntive function.

I'm even more skeptical that you can extrapolate anything generally useful about cognitive decline, since this result is exactly what I'd _expect_ if there _were_ no cognitive decline -- people are better at remembering things they can contextualize to the familiar, so _of course_ they'll be able to remember new information about their families better when they still live with them than after they move out.

<comment><USER8> agreed. I'd even be inclined to take it further and suggest that once leaving high school, pressures and stresses related to self sustainability (being employable, financial concerns, housing, etc etc) would become more real than any previous family memory.

Also, wouldnt an understanding that materializes as one gets older of the back story into each memory story (with the frame that memory stories here as having a positive impact) have the potential to change the frame of the memory, which decreases the likelihood of it being remembered? As in, certain memories are pretty great until ypu understand how it came to be that way, in which case either the back story becomes more signigicant or it becomes more mediocore than remembered. And mediocrity is easily forgotten...

I'd also suppose that as people move on with their lives, people relate to each other via shared stories... family stories have little meaning to those not involved, and so have little impact in "fitting in" in the rest of society. As that dawns on individuals leaving home, so are such stories repressed until they more actively need to be remembered.

</comment></comment><comment><USER9> Come now,this is silly.

There are people in their 60s that are so physically fit they can win a fair fist fight against a 20 something year old who ever so rarely works on his/her muscles.

Why is there a presumption that intellectual capacity is any different? A person that trains their mind will do better than those that don't,that's obvious.but how many people reach anywhere near their intellectual maximum capacity?

My theory is that as people get older the items of life take up much more of their time than in their earlier years which steals their potential to maintain or exceed their earlier intellectual capacity. Those that manage to make time for a mental workout in latter years would likely feel as competent as when they were young if not more.

Just a theory based on observation.

<comment><USER10> I hear you. But consider this. So you're making a case that you can exercise muscles and cardiovascular system, so therefore you can exercise the brain.

But what about other organs? Can you keep skin youthful through exercising it? What about hair? Eyes?

I think it's equally as plausible that no, you can't exercise the brain just by using it well. (though I don't personally believe this)

<comment><USER11> There is also clearly a peak and decline, even with exercise. The article does seem to cover this given the wide error bars for some of the cognitive tasks.

</comment></comment></comment><comment><USER12> PSA: Being determined is a superior predictor of success than intelligence. With age, you have a better idea of what you feel is worth doing, and why, than the easy enthusiasms of youth.

Of course, those with wisdom, intelligence and youth will beat you. It's rare.

</comment><comment><USER13> You are probably the smartest at around 5 to 10 years old. At least when it comes to creativity and learning. Peak performance of brain capacity is probably around 20 to 25. After that you aren't getting particular less intelligent per see but stress and lack of sleep are deterrent. The risk of stroke also increases as you get older. There are probably a peak of intelligence at around age 50 when your brain is still healthy and you have acquired experience and knowledge.

</comment><comment><USER14> Degree of change would be nice. These traits could all be very flat over time and therefore small peaks make things statistically significant.

</comment><comment><USER15> Disappointed by the tasks they chose. This is because the data is collected in a lab but we have so much more data these days.

Does anyone work for Udacity/EdX/Coursera? I would be interested to know if/how performance on those courses decline/improve with age.

It would also be interesting to see if performance is correlated because of lurking third factor. (My money is on exercise, or time).

<comment><USER16> Performance in MOOCS (or any educational setting) != Intelligence or ability to perform

<comment><USER15> I guess I'm more interested in whether my ability to learn machine learning (or whatever the equivalent is in 10 years) will decrease over whether I'll be able to remember a 7 digit number.

Call that what you will.

I am confused about what you consider "ability to perform". Education is a context for performance.

<comment><USER17> The theory is that your ability to memorize a 7 digit number (working memory) IS fluid intelligence.

The problem is that memorizing an x digit number is proven to be trainable, without much carry over to g or even other working memory tasks. I.e. - it's hard to train fluid intelligence rather than train the specific task which it is applied to.

</comment></comment></comment></comment><comment><USER18> I feel my ability to think quickly and see intuitive connections has decreased, but on the other hand I've become more logical and rational in my thinking. I think the latter has become more valuable to me, because it is key for communicating with others. And clear communication is much more useful right now than rapid thought.

</comment><comment><USER17> Fluff article with misleading quote: "In this view, knowledge does not compensate for a declining adult intelligence; it is intelligence!"

This doesn't introduce anything of interest to people who already know that fluid intelligence decreases and knowledge can continue increasing.

Depressing.

<comment><USER15> What if the article is aimed at people who didn't know that?

<comment><USER17> Then you got a decent introduction sprinkled with misleading commentary.

</comment></comment></comment><comment><USER19> Humans have an odd focus on individual intelligence, even though most of the achievements that we have accomplished as a species have been through collective intelligence – rarely would I say that people function and create alone.

<comment><USER20> Our obsession with individual intelligence is usually about selection for (leadership of) elite teams.

</comment></comment><comment><USER21> For a brief moment I thought this article will talk about the point where humanity itself peaks in intelligence and knowledge.

On the other hand looking at the USA, I'd definitely say it already happened.

</comment><comment><USER22> And when does the definition of intelligence stabilize?

</comment><comment><USER23> This is a classic example of Betteridge's law.

Edit: QED

>It seems that the question "When does intelligence peak?" is actually a rather meaningless question. Not only do our various cognitive functions peak at different times, but past a certain age it might make more sense to view adult intelligence not through the lens of youthful general processing speed and reasoning, but through the lens of expertise, wisdom, and purpose.</comment>


Defending the Right to Be Forgotten - <AUTHOR> http://jacquesmattheij.com/defending-the-right-to-be-forgotten <comment><USER3> If you look at the fact, that Google CEO Eric Schmidt actually recommends to teenagers, to change their name at the age of 18 just to escape the online artefacts of their young years[1], there are actually strong arguments to have a "right to be forgotten".

As we are moving forward to a society, where everything we do is shared online, not only by us but also by our friends, we suddenly loose the control over how we are portrayed. Of course there is the argument that this has been true with classic media, but from personal experience you always have the chance to rerecord something you said on a interview if it's not live and I pre read a majority of all print articles and interviews, because newspaper have high quality standards and try hard to get everything right.

As copyright holders are (quite successfully) lobbying to enforce their rights to their content globally, we as a society should enforce our rights on how we are portrait globally. This should not be exclusive to europeans and european cyber space, everyone should enjoy the freedom to decide which parts of his past should be forgotten.

[1][http://www.huffingtonpost.com/2010/08/16/google-ceo-eric- sch...](http://www.huffingtonpost.com/2010/08/16/google-ceo-eric-schmidt- s_n_684031.html)

<comment><USER4> That's not quite what he said:

 _Mr. Schmidt is surely right, though, that the questions go far beyond Google. "I don't believe society understands what happens when everything is available, knowable and recorded by everyone all the time," he says. He predicts, apparently seriously, that every young person one day will be entitled automatically to change his or her name on reaching adulthood in order to disown youthful hijinks stored on their friends' social media sites.

"I mean we really have to think about these things as a society," he adds. "I'm not even talking about the really terrible stuff, terrorism and access to evil things," he says._

[http://www.wsj.com/articles/SB100014240527487049011045754232...](http://www.wsj.com/articles/SB10001424052748704901104575423294099527212)

</comment></comment><comment><USER5> The article brushes aside _the_ major issue with how the French are trying to enforce this, which is that the French government is asserting that its law should apply to all global properties of an international corporation simply because it has a subsidiary operating in France.

A quote from Google on the matter:

        "we respectfully disagree with the idea that a national data
        protection authority can assert global authority to control the
        content that people can access around the world."

The author brushes this aside with the assertion that:

        "search engine results aren't 'free speech' to begin with in my
        opinion, they're re-hashings of someone else's speech."

And doesn't address the inherent problem with a global company being subject to the combination of the censorship regulation of all the countries in which it operates.

What is Saudi Arabia decides to start punishing Google because its image search shows "revealing" images of women, is that just fine because search engine results aren't free speech anyway, and any nation on earth should be able to make laws about search results that apply globally?

<comment><USER6> Google collect this data on French soil, and then export it abroad.

Why is it so hard to understand that the condition to allowing them to collect the data in the first place is that they respect the local law about that data?

<comment><USER7> Interesting thought about data being exported, I had never looked at it that way. From the French perspective, data is like a new "resource", and Google is the mighty data baron, in the era of the "data rush".

</comment></comment></comment><comment><USER8> This whole concept is ludicrous.

I can understand why it appeals to people - their sense of unfairness is triggered, and that is a useful heuristic for finding unjust behavior.. but the actual _implementations_ discussed amount to "make it like it used to be". On the surface it seems like you can design something that might work.. but the trick is, it'll involve a ton of fiddly bits of law. Those fiddly bits are things that lawyers (and therefore corporations and rich people) are very good at playing with, which will inevitably result in a evolving double-standard, just like in finance, health-care, politics, and employment law.

There is a form of the 'right to be forgotten' that makes total sense. Your creations - your journal, your twitter feed, your facebook notes and photo blog - these belong to you in a sensical way, and determining who has access to them might be a reasonable right you possess (though I'm unaware of any legal system that enforces that right).

But the conceptions of this 'right' I hear discussed all hinge on some variant of 'noteworthy' \- that's because a moment's thought of the obvious approach tells you "wait, that will gag the news services!", which we've trained ourselves to think of as representative of goodness and openness. But 'noteworthy' is a context-based definition! The news that is noteworthy to one group is not to another, and that distinction is inherent in society - <USER2> can't wrap a semantic game around to solve the problem.

The reality is that the nature of accessible information is _changing_ , and society will need to change with it. It's hardly the first time technology has forced society to change! The right to be forgotten is the right to exist in the world circa 1960 forever.

<comment><USER9> > Your creations - your journal, your twitter feed, your facebook notes and > photo blog - these belong to you in a sensical way, and determining who has > access to them might be a reasonable right you possess (though I'm unaware > of any legal system that enforces that right).

That would be "Copyright".

<comment><USER8> Would be, if copyright were implemented that way. But it's not - you signed over the right to control the stuff you put on Facebook as soon as you wrote it.

<comment><USER10> > you signed over the right to control the stuff you put on Facebook as soon > as you wrote it.

You still retain copyright on it, though.

<comment><USER11> But you've also given them a non-exclusive and irrevocable right to use it as they want (with limitations, you'd need to read their exact policy to be sure).

So while you may retain the copyright, you could very easily have signed over permission to whatever company who's website you used to post it to use it however they want.

<comment><USER12> > But you've also given them a non-exclusive and irrevocable right to use it > as they want

Such things are not actually irrevocable in the US, although they do last a long time.

Starting 35 years after the grant of a transfer or license of copyright, there is a 5 year window during which the author can give written notice of termination, and the transfer or license ends. The author may do this regardless of any agreement to the contrary. This is codified at 17 USC 203 [1] for those who want the gory details. "copyright termination rights" is a good search term to find discussion and articles on this.

Probably not much use in getting embarrassing content removed from a website that you unwisely licensed it to because after 35 years people have probably long forgotten about it unless it is really bad, in which case it has probably already harmed you as much as it can.

It will be interesting to see if the termination right ever affects free/open source software.

[1] [https://www.law.cornell.edu/uscode/text/17/203](https://www.law.cornell.edu/uscode/text/17/203)

<comment><USER13> > It will be interesting to see if the termination right ever affects > free/open source software.

I wonder if this could be tested with code assigned to Oracle? I bet there are contributors to OpenSolaris under CDDL that wouldn't mind testing it out.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I know of many contractors in the UK who registered a LTD Company. Since then they have taken permanent jobs so dissolved the company. What's left on the internet if you google their name is their personal details and home addresses. Companies House say there is nothing they can do because the company has been dissolved. However, companies house have sold the data to hundreds of other companies who splatter this private information all over the Internet. These other companies have no processes in place for hiding private information.

IMO this is not acceptable and a good application of the right to be forgotten principle. If companies house were not so greedy in selling the data to companies which employ heavy SEO tactics to get hits, the problem would not exist.

</comment><comment><USER15> I also agree with the right to be forgotten principle.

However regarding its current implementation, I don't necessarily agree with the fact that private entities are the one judging the requests (and they never asked for it!). If it is some sort of legal requirement, in my opinion, it should be reviewed by some kind of judge, administrative body or peers, that would apply consistently the rules.

Finally regarding the EU vs Global application of the rule, I have to side with Google : you can't comply globally with every specific request from every country or sensibility, because it would mean banning a lot of things (from blasphemy, homosexuality to various disputed historical events, or territories...). I can't see that working practically.

<comment><USER16> QUESTION: Why should you have 'the right to be forgotten'?

<comment><USER17> Technically, because the EU countries are sovereign and can decide what rights their own citizens should have.

*EDIT: wrong word

<comment><USER18> That's why people may _have_ such rights, not why they _should_.

<comment><USER19> Nope, that's pretty much all it takes. Society decides itself what rights its members have which is why they are not the same everywhere.

Or can you suggest a better authority to define them?

<comment> V-2 So if society decides that redheads have no right to live, then they have no right to live, end of story?

This is a dangerous school of thought, authoritarian in spirit - the state as the source of ethics and the ultimate moral authority (Hegel's view).

The opposing concept is called [https://en.wikipedia.org/wiki/Natural_law](https://en.wikipedia.org/wiki/Natural_law)

<comment><USER19> OK, let's take the third paragraph of your link as an example, where it basically describes how Natural law forms a philosophical bedrock of US society. Why were documents resulting from it accepted by society (eventually, more or less)?

Clearly from the same page there is not just one and societies existed and still do that don't share the same underpinning. They could have turned it down and go with something else.

My point is that they didn't because this was close enough to existing shared values. That doesn't mean it matched anyone's perfectly or that you can't move them over time, but they are not imposed.

Well, they may be for a while under occupation of a stronger force, but surely that is not what you argue for.

In your hypothetical example it would mean yes, they don't _in that society_. Like converting to a different religion can get you killed in some places even if it is completely unimaginable where I am from.

I am certainly not agreeing with or defending choices of every society under sun.

<comment> V-2 > In your hypothetical example it would mean yes, they don't in _that society_

So you mean "rights" in descriptive sense, you're talking about the _de facto_ rights. This obviously excludes rights that are legally violated, but which are still there (from ethical point of view).

If someone asks why people _should_ have a certain right (as nthcolumn did), they clearly mean _rights_ in the latter sense of the word. Otherwise it's not an answer because it's circular logic: why would society grant you these rights? Because society grants you these rights. Well, like every tautology it's true ("technically", as you put it), only it's of no use :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> The logic fails on this premise:

> On the other hand, from the view of an individual that has done something > wrong at some point in his or her life and that has through either a fine or > a jail sentence paid their debt to society it is kind of annoying that > they've been declared pretty much un-employable by google or one of their > lesser competitors.

The error here is in the behaviour of the society in general. If you use the right to be forgotten as the correction mean, then you prevent society from evolving and changing this wrong behaviour.

We are entering an era of increasing information, society needs to adapt, and we must allow it to adapt, painful as it may be.

<comment><USER21> > We are entering an era of increasing information, society needs to adapt, > and we must allow it to adapt, painful as it may be.

I've a nagging feeling that I've misunderstood your post somehow, if so let me apologize in advance for the remainder of my post:

That's exactly what society is doing: adapting. Previously information wasn't as accessible as it is nowadays so society adapted to the new situation, in this particular case by enacting laws that further protect people's privacy.

<comment><USER20> No. The right to be forgotten law is not an adaptation, it is an attempt to cling on to the past.

In the past your personal history was difficult to dig. Nowadays it is archived on the Internet for anyone to search and read. These laws try to emulate the past environment in the new information age.

Instead, I think we should let information flow. People will adapt to having lots of information available, and the effect of negative signals, that people are so afraid of today, will probably be minor. As an example: who cares if you smoked pot in college when 60% of people also have an online record of smoking pot.

<comment><USER19> No, it is an adaptation, but time will show how successful. Just as it does for every other. Adaptations are not only changes that succeed or that conform to the way you would like to see society evolve.

</comment></comment></comment></comment><comment><USER22> At what point are you remarkable enough to lose the right to be forgotten? What counts as being in the public eye? Do you have to desire to be in the public eye?

How do you manage the case of an unremarkable person removing something, then becoming remarkable enough to lose that right?

<comment><USER23> This is why we have courts, to answer these sorts of questions on a case by case basis.

<comment><USER22> I agree with the other commenters here - it can be a bit 'fuzzy', which is disagreeable to people who like definite answers. Case by case is probably a good way of tackling it. The cases, however, do need to be held in reference to the law, and those laws need to be appropriate for the scope and scale of our communication technology.

If we are arguing that someone needs to be 'remarkable' then I'd be interested in what people think counts as remarkable. You may be remarkable in a certain community, job or geography, but not in others.

</comment></comment></comment><comment><USER24> Even with the rules the author speaks of the entire law is still one big, stinking gray area:

\- Should single owner corporations be treated as corporations or persons? A singer with his own company in his own name might want an article about him removed. \- Should a low level government employee be able to have a forum rant about his handling of a case removed? What if the mishandling of a case wasn't the employees fault? \- Can a corporation get (pay/threaten) another person to remove an article if that person is mentioned in the article? \- What level of politician should be unable to have a link removed? Member of a political party? City counsilman in some small town? \- Does the article about a politician have to be about politics? What about a personal/non-work related link about a politician? \- Famoust pianists shouldn't be able to have a review removed, but what constitutes a 'review'? \- What about people who don't want to be in the public eye - it's not always their choice? \- So EU's laws should apply everywhere - what about Iran's? Why are they different?

This just scratches the surface.

</comment><comment><USER16> Why should you have 'the right to be forgotten'? And why should it only be Google that has to do this?

edit:Having to click 'accept cookies on every site' is such EU nonsense too - does anyone really think you can't be tracked without cookies?

<comment><USER21> > Why should you have 'the right to be forgotten'?

Why should anyone have any rights? Rights are granted by sovereign nations to their citizens. Sometimes (and this is a good example, the US "right to bear arms" is another good one) different nations give their citizens different rights. When these rights are given by democratically elected governments, it's fair to assess that the people collectively decided they wanted said rights and enacted them.

<comment><USER25> But these rights almost always have a philosophical justification – they aren't just things that we want, they're bigger than that.

<comment><USER23> Perhaps people should be able to move on with their lives after they make mistakes.

<comment><USER15> Moreover, in some circumstances, it is not even a mistake, but you've been the victim of something terrible, and your name will be associated forever with it.

"I am not sure I want to employ/rent to someone who's been victim of X..."

</comment></comment></comment></comment></comment><comment><USER26> I have a newspaper clipping from when a car hit me when I was 5 years old... the dark truth of the matter was _I hit the car_ by bolting across the road and smacking my head into the passenger side door as it was trying to drive through an intersection. This little page 6c clipping is all that exists of that moment. No cameras, videos, blog or twitter posts.

If that happened today the driver's name might be permanently searchable and could even cause problems when looking for a job "They hit a 5 year old, that doesn't sound like the type of person we'd want to hire as a delivery driver."

Maybe we can't fix this problem, or maybe it isn't a problem at all but just something we should accept. One thing is certain, we've never been in a situation where everything was recorded so thoroughly before. Can we adapt?

</comment><comment><USER27> It's really 'The Right to Pretend to Be Forgotten.'

Google isn't the only search engine or gateway to the web. Every year there are more avenues.

If the EU were serious they'd force newspapers to edit and republish back issues - but they're not.

<comment><USER28> A citizen of the EU can also ask a newspaper to remove online articles about him, and in fact the requests usually start there. Why that doesn't apply to hardcopies has also been discussed at length.

Newspapers are not required to retroactively publish their articles, because the EU directive is about data accessibility, not about rewriting history. The EU is simply codifying that the natural shelf-life of (newspaper) articles also applies to online content.

Yes, you can still find those articles if you're willing to spend the effort. Search engines removed the effort requirement, and the EU decided that this placed a disproportionate burden on their subjects' personal lives.

</comment></comment><comment><USER29> The problem with the "right to be forgotten" is it's like a "right to have a nice day." It sounds great in principle and certainly nobody will say that people shouldn't have nice days, but any actual implementation requires the elevation of one person's rights over another. And that's not acceptable.

<comment><USER21> > requires the elevation of one person's rights over another. And that's not > acceptable.

I'm sorry but that is senseless. Elevating certain rights over others is effectively the basis of any society. To use a crass example: it's widely accepted that your right to not be assaulted is elevated over another's right to assault you.

<comment><USER30> Does the person who assaulted the other person in your scenario have a right for any record of it to be removed from the internet?

<comment><USER31> In Western European societies: YES! Something like a public register of sex offenders is UNTHINKABLE in Germany. You get a sentence for your assault, but public shaming is not part of the punishment. That's not inherently worse or better then the American approach, it's just what most people here consider right and just. We consider people who break the law as human beings and citizens, and we strive to make them functioning members of the society.

<comment><USER30> Public shaming is distinct from hiding all the evidence of someone's past actions.

</comment></comment></comment></comment></comment><comment><USER32> The Right to be Forgotten if enforced globally will create a blackmarket for this type of information.

Downloading copyrighted material is illegal. Technical solutions exist and individuals are making money off of it. Buying drugs is illegal too. Solutions exist and individuals are making money off of it.

Do not fool yourselves, you won't be forgotten.

<comment><USER33> The point is not to actually be forgotten, but to create a limited barrier against casual discovery.

In Norway basic income tax information is publicly accessible, and have been "forever" at the tax offices. This was considered important for accountability, while the limited access was considered to make it an acceptable loss of privacy. With the advent of the internet this information was suddenly available online, and "everyone" looked up their friends and neighbours. Suddenly the privacy impact was drastically different, and recently there has been new restrictions added to find a balance more in line with the original.

This is what the right to be forgotten is about too: Practices that previously had little privacy impact, and public benefits, now have vastly different privacy impact due to the internet. As a result we need to find new balances. In some cases the benefit of the information will outweigh privacy considerations. In others they won't.

Even then, so far EU courts have made it clear that the original publication is protected: Nobody has been given the right to have information wiped from newspaper archives, for example. But indexing and making that information extremely easily available is a separate issue.

There's no reason for a black market: The information is still there and easily accessible for those specifically looking for that information. It is just not pushed in front of someone "just" doing a regular search for someones name.

</comment></comment><comment><USER34> I would like to add a few points to the discussion:

* from my experience with the Dutch cookie law, the interpretation is that if you serve or target a Dutch audience, you need to comply; it would only make sense if the same metric is used for the Right to Be Forgotten;

* one interesting moral dilemma I have is whether this right to be forgotten should also apply to the audience in a different country (USA, for example). In other words, when someone from France requests a removal of a search result which was found on a US server, would this still be visible to people from the USA ? I think the answer to this _should be_ yes, according to the laws that currently exist, but would allow someone who really wants to have "unfiltered" results (say, a recruiter) to simply use a VPN or proxy server to circumvent the law.

All in all, an important but difficult topic to discuss.

<comment><USER35> > * from my experience with the Dutch cookie law, the interpretation is that > if you serve or target a Dutch audience, you need to comply; it would only > make sense if the same metric is used for the Right to Be Forgotten;

The first question is going to be "or else what?". Outside of the USA, Ireland and the UK, firms are going to simply choose not to operate there; but due to the nature non-discriminatory nature of the internet, their services will be available there anyway. There's not much that (e.g.) the Dutch government can do, and trying to fight it with blocking will just end up turning themselves into some sort of internet backwater; which I don't think anyone wants.

<comment><USER34> You're missing the point I was trying to make: Google wanted to implement the Right to be Forgotten only on the .fr domain, which seems like a cheap trick. The way legislation works around this is by simply saying "if you serve people from our country, you obey our laws", which makes sense.

Also, both this law and the cookie law are EU-wide laws, and I don't think Google will be retreating from all EU countries any time soon. That's how these things work when you're a small country, you work together with others and suddenly they take you seriously.

</comment></comment></comment><comment><USER36> The EU already has laws regarding the correctness of information which protect against keeping and reporting inaccurate information. We also have laws which protect your right to a certain level of privacy.

The only valid reason for the right to be forgotten is because those existing laws are not enforced - news outlets can print falsehoods and leaked celebrity nudes, companies can publish data they know is outdated, and laws that exist to punish them for these things are not enacted.

If such laws were to be enforced, what need is there to be forgotten? Only factually true public scandals seem to remain, and we have laws to protect their existence (freedom of the press, etc.) for a reason.

<comment><USER17> It's not about "public scandals", it's about small reports that got printed on page 34 twenty years ago on a regional newspaper, and that would have been history if Google didn't pull it out when you search for the person's name.

The landmark case for this issue was _Costeja_ : in 1998, a guy had some debts to Social Security, which forced the sale of some of his property to get reimbursed. More than a decade later, what would have been a record in a library basement, now appears as one of the top results when you search for his name.

[https://en.wikipedia.org/wiki/Google_Spain_v_AEPD_and_Mario_...](https://en.wikipedia.org/wiki/Google_Spain_v_AEPD_and_Mario_Costeja_Gonz%C3%A1lez)

<comment><USER36> I apologise, "public scandals" was poor phrasing. My point was that the right to be forgotten aims to limit/remove access to information that is legally publicly available.

In the Costeja case the court ruled that Google could be blocked from displaying the information on the grounds that it was no longer relevant (and did so under _existing_ data protection laws, even if I believe that their application of those laws was misguided). The court also ruled that the newspaper could not be forced to remove the information.

What bothers me about the right to be forgotten is that difference. The _information_ was accurate and legal, but _Google_ displaying that information was not. Why the split? If the information was truly irrelevant, why not have the newspaper remove it (and solve the Google issue in the process)? Why only Google (especially in the wider right to forgotten laws)?

</comment></comment></comment><comment><USER37> Having others forget your more embarrassing moments sounds so reasonable on the the surface.

Yet our profoundly good memories allowing us to remember almost everything we experience is one thing that separates from the rest of the animal kingdom. It makes the prisoners dilemma not a dilemma for us at all, but rather turns the history of decisions we make over time into a single fabric into which can be woven all sorts cooperative behaviour.

Fortunately a world where people can demand things they have done be forgotten has been explored by others in much greater depth than is displayed here. "Eternal Sunshine of the Spotless Mind (2004)" is a good starting point.

</comment><comment><USER38> Does the Right To Be Forgotten apply to those citizens (French in this case) that have done or had bad thing happen in other places, outside of their home country? What if that French citizen goes to e.g. Korea and, I don't know, bad mouths a local politician in a blog post that maybe ends up as a local scandal as well? The person is probably still "unremarkable." Citizens of other countries that haven't enacted such laws or agreed to the removal have a right to that knowledge, then.

My example isn't great but hopefully you get what I'm saying.

</comment><comment><USER39> It might be naive but wouldn't it be easier for the individuals to change their name ? It's like burned HN accounts, you don't keep the karma but a new account gives you a fresh start.

<comment><USER40> Unfortunately, as a general solution, this doesn't really work. The enormous paperwork hassle aside, in some jurisdictions (like Germany), you can't change your name except in very limited circumstances (I think here the situations are marriage, naturalization, or you can demonstrate some hardship from it, like being named "Assbutt McGee" or whatever).

<comment><USER39> In either case some change to regulation seem to be necessary though. "can demonstrate some hardship from it" could be extended to the right to be forgotten.

</comment></comment></comment><comment> V-2 If humans have a "right to be forgotten", there's no reason why it should only be enforced on Google or search engines.

Bringing the story up again in print, or on a talk show or wherever (in public) should be made illegal as well.

</comment><comment><USER41> What is the motivation behind his affectation of spelling "google" with a lower case "G"?

<comment><USER21> Completely unrelated to the article: your post made me wonder whether "to google", as a verb, was accepted by any dictionaries and if it'd be capitalized or not. According to Merriam-Webster it's indeed accepted as a verb and is not capitalized (though they say it often is)[1]

I was just curious and wanted to share :)

[1] [http://www.merriam-webster.com/dictionary/google](http://www.merriam- webster.com/dictionary/google)

</comment></comment><comment><USER42> Before delving into a view on the case for a "right" to be "forgotten", I want to touch on some points about the article, the legal aspect, and some meta points.

Regarding the article, I disagree with the "evil corporation" test, and also disagree with the "unremarkable person" test. These test seems too specialised, and too representative of a particular worldview, to be of general applicability or usefulness.

Legally, it is an exciting and fascinating case, and really emblematic of the present. Given their history, it's cool that France is the nexus where this is going on. I'm fascinated to see what the courts adjudicate and what the enforcement results are.

At a meta level, making regard to Google's strategy, if I ran Google, I would never ever have given in to any of this, because as I will come to, this is not really about a "right to be forgotten at all." Europe wants to censor our search results? Ignore. Europe threatens fines? Ignore. Europe makes some effective threat? Respond with a threat to shutter Google services for Europe, and market it, just to further incent the populist outrage, as adhering to principles of freedom for individuals and anti-state censorship ( as they did in China, even tho that decision was, because of different reasons, incorrect ). Then see where the regulators loyalties are, with the "cause of a lifetime" or with their soon to be rescinded tenure?

The antitrust case, and now this, it worked for Google to never have budged an inch on either of these things, but because the legal advice they listened to was all cowardice, they did. A reason contributing to the workability of an unflinching attitude for Google is that this is not really about a "right to be forgotten" for individuals at all, it is really about a "right for Europe to forget the reason their tech sector failed to produce a Google".

Europe is bitter, that an American company has, again, taken over the world, and left "poor Europe" with the scraps. Rather than owning this, the Europeans would rather pretend that Google has done something wrong, so they can pretend that their failure was, in fact, the result of some injustice perpetrated against them by what they delude themselves is an evil corporation. Europe indulges this fantasy of victimization instead of owning their failure and creating improvements to make their tech sector work. Obviously their attitude is self-defeating and their delusion isn't helping them create any improvements. It is giving them the fake pay off of pretending they are righteous fake victims of an "evil", American company, that they feel in their delusion is "too successful to be moral".

Because the antitrust case and now this is driven by Europe's meta desire for a catharsis from owning responsibility for its economic impotence, Google should never have aided Europe in indulging their fantasy of victimization. It doesn't help Europe, and it doesn't help Google, because, as Ivan Vanko in Iron Man II says, "If you could make God bleed, people would cease to believe in Him. There will be blood in the water, the sharks will come. All I have to do is sit back and watch as the world consumes you." Europe is poking the Googlebear because Europe, like Ivan, incorrectly blames Google for its own legacy of failure, and is now obsessed with fabricating a narrative that fake justifies their crusade against the American giant.

To paraphrase Tony Stark, "Where will you be watching the world consume me from? Oh, that's right, economic meltdown. I'll send you a food voucher."

Google should never have budged because there now is blood in the water, and Europe has found a weakness.

To get even more meta, Europe actually was testing Google's strength. Is Google strong enough to be an inspiration for leadership here, to us? If Google had showed true backbone, and said, "No, this is ridiculous," Europe would have stepped back, confident they were in the presence of something that really deterministically knew what it was doing, take off their hats, bowed, and said, "Okay, Google, show us the way." This would have led to greater integration and European collaboration with Google, which would have been great for everybody. Unfortunately, Google caved at the crucial moment, a trend that is disappointingly common in present-day American leadership.

Moving now to the actual ethics, I think there are two basic principles under contention in a case for a "right to be forgotten" :

1) does anyone ( beside the state, which is fine ) have the right to control the contents of another's mind?

2) what information about a thing ( person, company, entity ) is it lawful to keep secret ?

The answer to the first is no, no one has a right to control the contents of anyone else's mind beside their own. Every person has perfect mental freedom to discover, learn, think and believe whatever they choose or whatever random chance leads them to. This is obviously to say nothing of expressing those beliefs, or creating consequences from those beliefs in the world. These latter two things are ignored in this present discussion.

The answer to the second is important to define. Some things are lawful to keep secret, and laws are already being or have been introduced to protect people in these cases ( such as the privacy of clinical information, the illegality of "revenge porn", "sealed" legal proceedings, trade secrets, financial information of private companies, information that incites terrorist violence ( in Britain, at last ), and other things ). There is a lot of scope here for creating improvements, and likely a lot of future caselaw to adjudicate what different places mores and standards are about these things.

So, in light of this, and in light of the internet being a "public space", and in light of Google being a major gateway to access information, it's clear that there are __some__ things which are lawful to keep private, and illegal to publish publicly, and therefore, if they have appeared on the internet, have appeared on the internet illegally, and therefore are subject to similar treatment as now effected by DMCA takedown requests. So there is a right to remove some of these things, and that is simply the normal legal apparatus also finding the expression of its effects on the Internet. These things are not legal off the internet, and eventually the law will enforce them on the internet as well. In time, the coordination to strike a balance between different jurisdictions will lead to some "enclaves of permissiveness and iniquity" and also some new cooperative legislation between different jurisdictions to work together to remove illegal content.

So the basic consideration here is that while no one can limit what an individual puts into their head, there are some things that it is not legal to publish, and so in effect, individuals will not be able to put them into their heads.

Calling it a "right to be forgotten" is really just abstruse, and incorrect. You can't force someone to forget something particular ( unless Eternal Sunshine is real ), and you can't really even force the internet to forget. You can make some types of information illegal, and effect their removal or prevent their display. Practically tho, just as you can not unsay something said, nor put the dropped tumbler back together in the same way, you can't rewind thermodynamics and you can never remove information once you have unleashed it online. All you can do, is limit the fallout, and try to limit the scope of dissemination, and try to dissuade people from releasing in the first place. One of the things this difficulty of enforcement suggests is that the penalties, when they are written, will come down on the side of being exaggeratedly deterrent, since their excess must compensate for the lack of a preventative complement.

<comment><USER15> I can't manage to reconcile what you are explaining here with your previous post on Turkish censorship.

[https://news.ycombinator.com/item?id=10005009](https://news.ycombinator.com/item?id=10005009)

It sounds like you are advocating the exact opposite in both posts regarding how a global company should treat local laws and culture.

</comment></comment><comment><USER16> Being downvoted just for asking a question is not cool. Can somebody please tell me the/your argument(s) for a 'right to be forgotten' or state the 'right to be forgotten principle.'?

</comment><comment><USER18> Honest question: why doesn't google just stop serving France for a day, with a message such as "we can't provide service to you because France is bothering us, contact your (whatever the French equivalent of Congressman is)"? Or at least do what they did with SOPA and put a link on the homepage.

Does anyone think that wouldn't work, or that the cost to Google would be too much? I don't think France would be as willing as China to part with Google.

<comment><USER17> Because they'd get as much, or more popular opposition as support. SOPA was seen as a corporations vs people issue; this isn't.

And even from those who don't particularly like the Right To Be Forgotten, many would not take kindly to an American company trying to teach us how we should organize our society. Anti-capitalism, and anti-American-imperialism are still strong ideas in Europe.

<comment><USER18> >And even from those who don't particularly like the Right To Be Forgotten, many would not take kindly to an American company trying to teach us how we should organize our society.

This isn't saying "organize your society like this", it's saying "we refuse to operate under these laws we view as restrictive, so as a society, decide whether you prefer to have these laws, or have Google". They have leverage, why not use it?

As for popular opposition, their case is that they shouldn't need to censor things around the world that are only illegal under French law. That seems likely to be well supported (especially if the alternative is "we can't use Google at all anymore").

<comment><USER17> _As for popular opposition, their case is that they shouldn 't need to censor things around the world that are only illegal under French law._

It's not clear that France wants to censor things around the world. The problem is that Google doesn't censor results _even to French connections_ , if the user accesses Google.com instead of Google.fr.

<comment><USER18> I don't think Google would particularly care if they only needed to censor French connections . They're clearly understanding it as a global demand. See [http://googlepolicyeurope.blogspot.com/2015/07/implementing-...](http://googlepolicyeurope.blogspot.com/2015/07/implementing- european-not-global-right.html)

>We believe that no one country should have the authority to control what content someone in a second country can access.

Additionally, if this was all a misunderstanding, I believe it would have been cleared up by now, so the fact that nothing like that has come out in the last ten days tells me that they did understand correctly.

<comment><USER17> I think Google is purposefully being misleading about that. If you read the notice they link to, it says:

 _Although the company has granted some of the requests, delisting was only carried out on European extensions of the search engine and not when searches are made from "google.com" or other non-European extensions._

 _In accordance with the CJEU judgement, the CNIL considers that in order to be effective, delisting must be carried out on all extensions of the search engine and that the service provided by Google search constitutes a single processing._

Nowhere does it say that the problem is that the results are still accessible from other countries; it's specifically about still being accessible on the .com and other domains.

<comment><USER18> I don't see any motive for Google to do that. Does it make sense to you? Why would they agree to censor the .fr, but not the .com for French users? I think they do understand the notice correctly.

And if it was that simple, why hasn't CNIL come out and said that Google is misinterpreting it?

Also, what they link to is not the notice, which can be found here: [http://www.cnil.fr/fileadmin/documents/approfondir/deliberat...](http://www.cnil.fr/fileadmin/documents/approfondir/deliberations/Bureau/D2015-047_MED_GOOGLE_INC.pdf) I don't read French (and putting it through a translater didn't help much), so can someone who can maybe tell us what it actually says?</comment></comment></comment></comment></comment></comment></comment>


Questioning the Clampdown - <AUTHOR> https://www.wsj.com/articles/questioning-the-clampdown-11584485339 <comment><USER2> The last 24 hours have seen some dueling arguments, from respected sources, regarding how to model this.

This one (pdf) has been highly influential to US and UK governments and is pretty sobering and alarming: [https://www.imperial.ac.uk/media/imperial- college/medicine/s...](https://www.imperial.ac.uk/media/imperial- college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI- modelling-16-03-2020.pdf)

This one is way on the other end: [https://www.statnews.com/2020/03/17/a-fiasco-in-the- making-a...](https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as- the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable- data/)

I'm just hoping that we discover effective treatments soon, because with those we might be able to start treating this like a normal cold before any of the worst case scenarios would come to pass.

<comment><USER3> That first link isn't loading for me. Looks overwhelmed. I checked the Wayback archive, and it says it has been archived a lot of times, but it isn't working. Anybody have a working link?

<comment><USER4> the first link is a pdf. try checking your downloads folder?

<comment><USER3> I know it's a PDF. I got it eventually. Did you check archive.org for the url? As of yesterday it was simply broken.

</comment></comment></comment></comment><comment><USER5> About every two weeks since January, it seems there are people who say that it's too expensive to take bold action. And roughly every two weeks later it gets much more expensive, and the options become narrower. This, despite having China, and then Italy, to give us a sense of the severity, and urgency.

These are hard choices. There are no easy answers at this stage. I think there's also a large degree of cognitive dissonance at play, where people aren't really able to grasp that this is all actually happening.

Personally, I think we should have done this weeks ago. I think it's good we're doing it now, but we lost time and it's gonna cost a fortune. Rolling the dice on this virus feels reckless. That seems to be the opinion of people like Bill Gates, Taleb, Fauci, etc.

</comment><comment><USER6> The number we need to know is the death rate in the absence of hospitalization. Because if we let it ride, and the hospitals fill up, that's the number we're looking at.

If it turns out to be only 1%, then shutting down the economy is probably a mistake. But if it's more like 10%, then without quarantine we would get an economic collapse anyway, plus a much higher death toll.

<comment><USER7> A 1% CFR will be catastrophic if we let it ride. Over 7 billion people worldwide. Assuming a simple 1% infection rate (extremely unlikely consider the R0 of SARS-CoV-2), that would result in over 700K dying worldwide. And that ignores the hospitalization required for non-fatal cases. Currently it looks like 18% of all infections require multi-week hospitalization. So again, with 1% infection rate, that means over 12 million hospitalizations. And that's just a 1%/1%/18% assumption.

<comment><USER8> A completely collapsed economy could trigger a society breakdown and will have deadly consequences as well, like violence (fighting over food, possesions) and bad healthcare

<comment><USER7> True. But imagine a more realistic scenario based on the CFR we've seen worldwide. Start with 7B people. Assume 50% get infected because we don't want to do a lockdown. 3.5B infections means 630M multi-week hospitalizations and 70M fatalities worldwide. And the follow-on effects will kill even more since these hospitalization rates will consume all our medical resources, so other illnesses will have higher CFRs as well.

I don't know about you, but avoiding 70m fatalities and over 600M hospitalizations is the optimal choice. Pursuing this choice would be the greatest human calamity, surpassing the destruction of WW2 in terms of lives.

<comment><USER2> Is there a way to derive infection rate from the R0 value? Because we don't even get close to 50% infection with the flu, and covid's R0 isn't like measles or anything.

<comment><USER9> There's no way to derive infection rate at all, unfortunately. Even for pandemics in the past, where we can go back and look at all the epidemological data, estimates of infection rates tend to vary by at least 10 percentage points.

</comment></comment></comment></comment></comment></comment><comment><USER10> I believe that at this point, any quarantine or 'shelter in place' order must be voluntary. We stand to lose too much by giving up fundamental rights, and it is questionable what we would gain from...what, arresting the vanishingly small number of people who are still blasé about the measures?

Like the article notes, our goal is now to slow the virus, not stop it. Normalizing heavy-handed authoritarian measures will only breed future despotism.

<comment><USER11> > must be voluntary

Must not be voluntary. Humans are erratic and generally selfish.

> now to slow the virus, not stop it

Stopping it is ideal and when the plague is leathal enough, the only option. It is not draconian to institute measures that are effective. They will be instituted eventually (eg if a plagie was ebola with a 60% mortality and 2wk incubation), regardless of what philosophy you subscribe to.

<comment><USER12> If you have a complete mandatory lockdown that works to stop the spread, it's highly likely that there will be a resurgence of infections as soon as the lockdown is lifted.

</comment></comment></comment><comment><USER13> Here's what it comes down to then; Which option is cheaper:

1\. Spend money to temporarily increase hospital beds and ventilator capacity, but keep everything else running normally, or;

2\. Shutdown everything to slow the spread so that we don't need to increase bed and ventilator capacity.

I'm tempted to say that adding bed and ventilator capacity is cheaper. I don't think that option costs trillions of dollars, like option 2 does.

<comment><USER7> The first assumes that increasing vents, ECMOs, and hospital beds is all that's needed. You'd still need trained staff to man them. Staff that is extremely vulnerable to this disease. Also, if you don't shut things down, you'll overwhelm the system, even if you were able to double the number of beds...

<comment><USER13> Do those things cost trillions of dollars? I don't think so. Each country could probably commit a couple hundred million to this and be better off than a complete shutdown.

<comment><USER7> In a world with perfect planning, yes it would have made more sense for us to expand our healthcare so that we could withstand something like this. But we didn't. We made our bed, and we have to use what we have in hand.

We can't create enough bed space, vents, ECMOs, in the next 6 months. We can't train enough doctors, nurses, etc in 6 months. So we play the hand we're dealt, and choose a mitigation strategy that doesn't result in 70M worldwide fatalities (6M in the US).

<comment><USER13> Why 6 months? How much money does it take to do it in 1 month? Is that amount of money less than trillions of dollars?

<comment><USER14> It's not a cost issue. It's supply & capacity issue. You cannot create instant supply immediately, even with a trillion dollars.

<comment><USER15> Lower requirements by having virus treatment staff with rapid specialized training

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> [https://archive.md/KMBQc](https://archive.md/KMBQc)

</comment><comment><USER17> In the final analysis, it will be said that Trump sacrificed his crown jewel, the American economy, to save lives.

I think better of him for it.

</comment><comment><USER18> [https://en.wikipedia.org/wiki/Holman_W._Jenkins_Jr](https://en.wikipedia.org/wiki/Holman_W._Jenkins_Jr).

Born 1959. 61 years old.

A quick search for covid mortality by age brings me to [https://www.businessinsider.com/coronavirus-death-age- older-...](https://www.businessinsider.com/coronavirus-death-age-older-people- higher-risk-2020-2?op=1) which puts him in the 3.6% mortality bracket.

If we assume his parents were in their twenties when they had him, they are in the 80+ bracket, with 14% mortality.

As to what we are getting from this shutdown? Does "a major boost to Medicare For All and maybe even basic income to substitute for the fact that the minimum wage has not budged since 2009" count as a thing we get? Probably not, he got his already.

Or in summation, ok boomer.

<comment><USER19> You can't post personal attacks to HN, nor generational slurs, nor flamebait in general. We ban accounts that do these things.

If you'd please review [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html) and use HN in its intended spirit, including right now when stress, fear, and anger are at higher than usual levels, we'd be grateful.

<comment><USER20> There isn't consensus on whether "ok, boomer" is a slur vs a protest or criticism. It seems inappropriate to me to ban accounts for using that phrase.

<comment><USER19> We don't ban accounts only for using that phrase. If someone just said "ok boomer", I'd either not reply or do a "please don't do this here" ([https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...](https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20%22do%20this%20here%22&sort=byDate&type=comment)) In practice, though, accounts that do that kind of thing tend also to do other things. Combining it with personal attack is already an escalation.

In any case, it is generational flamebait, which is off topic on HN regardless of generation.

</comment></comment></comment></comment><comment><USER21> Anyone concerned with individual rights should be concerned at the government responses. H1N1 caused 10K+ deaths and infected tens of millions in the US, and we had nothing even remotely approximating this response.

This is not to say that we shouldn't be concerned and diligent in our efforts at containment, but that we should also be concerned and diligent about our liberty. Right now I see a lot of emphasis on the former and almost none on the latter, and anyone who voices concerns is labeled a crackpot or worse.

<comment><USER22> The 10K+ deaths is over the entire course of H1N1/09\. The US had ~4K deaths from 2009-04 to 2009-10 [1]. Italy has had 3K deaths in just 2020-03 [2]. You can't directly compare the early state of COVID-19 to the end state of H1N1/09.

[1] [https://www.cdc.gov/H1N1flu/estimates/April_February_13.htm](https://www.cdc.gov/H1N1flu/estimates/April_February_13.htm)

[2] [http://www.protezionecivile.gov.it/media- communication/press...](http://www.protezionecivile.gov.it/media- communication/press- release/detail/-/asset_publisher/default/content/coronavirus- sono-28-710-i-positivi)</comment></comment>


Why is processing a sorted array faster than an unsorted array? - <AUTHOR> http://stackoverflow.com/q/11227809/399268 <comment><USER2> Modern architectures really are quite complex, and it's important to keep your internal machine model up to date with what's happening. Examples: It used to be on a 486 even correctly-predicted taken branches had a two-cycle penalty. It used to be important whether you arranged your code so the most commonly taken code was on the fall through path or not. Today, correctly predicted branches are free--the processor stitches together the current path and the new path so there are no bubbles in the pipeline. Virtual function calls also used to be more expensive back in the day, because CPU's didn't try to predict the target address of the branch. Today, CPU's have a buffer that maps from a program counter to a target address, so if your virtual function call predictably hits the same target each time, the only overhead will be the extra memory accesses to indirect through the v-table.

At the same time, things that are expensive today may not be so in the near future. Haswell is supposed to make uncontended synchronization operations almost free. It'll make a lot of algorithms, particularly lock-free algorithms, much more practical than they are today. For example, C++ smart pointers are slow in multi-threaded situations because the reference count manipulation needs to be done under a lock. But the lock is almost never needed (except when it is). Cheap synchronization should make it much more practical to use reference counting more pervasively in C++ programs.

<comment><USER3> "For example, C++ smart pointers are slow in multi-threaded situations because the reference count manipulation needs to be done under a lock."

Several architectures support atomic increment and decrement operations, so there's no need for any lock. Libaries like OpenSceneGraph use them for their reference counting.

Also Haskell isn't magical in this regard. Their lock free implementations will also presumably just use safe atomic operations.

<comment><USER2> Atomic increment/decrement operations still generate a locked memory transaction of some form or the other at the processor level. Haswell is Intel's new CPU that is supposed to implement some level of hardware transactional memory, leveraging the infrastructure used for cache coherency. In theory, this will make transactional operations (atomic increment/decrement is just a tiny transaction) as cheap as a regular memory operation so long as there is no contention.

<comment><USER4> Sort of, but it's not really a "lock". If, as is very likely in performance- sensitive-but-uncondended multithreaded code, the CPU already has the cache line in "exclusive" (in the MESI sense) mode then the atomic operation needs to be no faster or slower than a standard read/write sequence. On x86 it is serializing, however, which can have performance impacts for some workloads.

<comment><USER2> An atomic operation on a cacheline owned exclusively by the local CPU avoids any bus lock operations, but still serializes the pipeline to avoid conflicts caused by memory operations on the same CPU. This basically destroys your memory parallelism if you use it in a situation where the atomic instruction happens often (e.g. every time an object reference is loaded or stored).

<comment><USER4> Isn't that exactly what I said? Note that "destroy memory parallelism" is often not a high penalty in typical workloads where all in-flight accesses are hitting L1 cache only. It's not nearly as high as the "full round trip to DRAM" latency implied by calling it a "lock".

<comment><USER2> I'm not really disagreeing with you, but I think we're using the terminology differently. To me, "memory" is the whole memory pipeline--everything from the load/store units, through the load/store buffers, the caches, to DRAM.

When I said it was a "locked memory transaction of some sort" I was including the effect of a serializing instruction which prevents concurrent load/store operations even if they hit the cache. I wasn't trying to imply a full round trip to DRAM.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> Ah this.. the highest voted Q on SO. Not again plz - [http://www.hnsearch.com/search#request/all&q=Why+is+proc...](http://www.hnsearch.com/search#request/all&q=Why+is+processing+a+sorted+array+faster+than+an+unsorted+array) (interesting case of same page diff urls bypassing HN duplicate url check though)

Go to <http://stackoverflow.com/questions> sort by votes and read from top if you like.

<comment><USER6> I also saw it a few weeks ago here on HN and was wondering why it is here again... At least I wasn't the only one who thought so ;-)

Sometimes I wonder how old 'news' can be and still climb up to the top of HN. Is there somewhere a search engine which can tell the author if something was on HN already? Or if it is about good websites an not about news, why nobody posted a link to google.com during the last weeks? ;-)

<comment><USER7> Wasn't there something from the 80's recently?

<comment><USER8> A week or two ago we had a text from 70's about diamonds, AFAIR. :).

</comment></comment></comment></comment><comment><USER9> Several months old, but still worth a read - especially for the end of Mystical's comment, where he discusses the different optimizations performed by different compilers. Particularly interesting is the Intel Compiler, which effectively optimizes out the benchmark itself: something to keep in mind when testing your own code, if you want your results to make sense.

</comment><comment><USER10> One take away point from this for non-c coders is that higher level languages will always be significantly slower than C because it's far easier to make these kinds of fiddly pipeline optimizations when you don't have another abstraction layer in between mucking things up. There's few java programmers for instance that both intuitevly understand how both the machine architecture, and the JVM compiler will affect the code they write. There's too many quirks. While in theory it's possible to write comparably fast java code(assuming your willing to do sketchy things for memory manipulation related tasks). In practice it's significantly more difficult to get all the juice out with higher level languages.

<comment><USER11> And assembly would be faster again?

Both the C compiler and the JVM JIT target machine code. I don't buy that one is automatically & universally superior to the other. You can equally argue that the JIT will always be superior because it's run-time aware.

<comment><USER12> In the counterfactual world where the JVM's JIT commonly won bechmarks against C, that would be a great point.

<comment><USER2> What kind of benchmarks? The benchmarks C folks always want to use are the ones that favor static compilation (matrix multiplication, etc). The best any language can do in that situation is tie--it's basically just an exercise of how much money has gone into the compiler's low-level optimizer.

Consider a benchmark implementing a common situation in real programs--a performance intensive loop calling some code located in a plugin that isn't known until runtime. The JVM will wipe the floor with a C compiler in that situation--since the JVM can do inlining at runtime between module boundaries, while a C compiler cannot do inlining at runtime at all.

Or, when memory management is involved, they always want to test situations where everything can be easily pool-allocated, instead of complex situations involving lots of dynamic allocation. But in situations where code is really allocating lots of short-lived objects (e.g. functional code for optimizing compiler trees), the GC in a JVM is going to wipe the floor with any malloc() implementation.

<comment><USER13> But, in "real programs", all this doesn't usually matter b/c clever JIT tricks with loops and branches fail to offset 5-10x the memory use and the abysmal corresponding CPU cache hit rate.

See "branch misprediction" vs. "fetch from main memory" on norvig's rough timings list: <http://norvig.com/21-days.html#answers> .

<comment><USER14> 5-10x the memory use? Only if coders smoked something. I agree that JITed runtime has some overhead, but in a great majority of cases it is far below what you write here.

</comment></comment></comment></comment></comment></comment><comment><USER15> Mystical mentions that Intel's compiler uses loop interchange to gain an extraordinary speedup. There's actually a great answer further down the page that details this and was great as I wasn't aware of how it worked: <http://stackoverflow.com/a/11303693/300908>

<comment><USER16> Meh, GCC can do the interchange as well, it just isn't on by default.

</comment></comment><comment><USER17> Isn't HN amazing? Here is the same article submitted on no less than four previous occasions:

* <http://news.ycombinator.com/item?id=4185226>

* <http://news.ycombinator.com/item?id=4170972>

* <http://news.ycombinator.com/item?id=4355548>

* <http://news.ycombinator.com/item?id=4167834>

Total upvotes: 29.

Total discussion: 0.

No wonder people try to game the system and find optimal times to submit things.

No wonder people think HN is broken.

<comment><USER18> Colin, I find it amazing that you did not confess that you were one of the people who submitted a duplicate of this story, your duplicate being the most recent:

4167834: zmanji 105 days ago

4170972: VeXocide 104 days ago

4185226: moobirubi 101 days ago

4355548: ColinWright 63 days ago

Especially given your fight against duplicated stories:

 _"My intention in doing what I did was always to try to create value by reducing duplication (and thereby reducing wasted effort)"_

Glass houses?

<comment><USER17> I was going to reply privately, but could find no contact details. I see you espouse HN Notify, so I'm hoping you'll see this response.

My duplicate detector found the initial pair almost immediately. I did nothing, because there was no discussion, so I didn't know which one to point to. Then the third came. Still no discussion. I watched in considerable dismay as a submission that I thought was absolutely excellent, and deserving of significant discussion, sank without trace.

Was it just unlucky? Did no one read it? There were a few desultory upvotes - a few had read it. But was I so completely wrong about what HN would find interesting?

I couldn't believe it, so I submitted it again _over a month later._ Still no upvotes, still no discussion. It seemed I was wrong about the audience here.

And now I can see that I was right about HN being interested in deeply technical issues like this, and I'm happy. I'm less happy that a brilliant item like this took four submissions, sorry, _five_ submissions before it got noticed.

That's why now I generally try to point out duplicates in two circumstances. One is when they come close together, and that's to help stop a split discussion. The other is when there was a significant discussion on an earlier occasion, and I want people to benefit from previous comments.

So no, I don't think it's a case of glass houses.

<comment><USER19> > But was I so completely wrong about what HN would find interesting?

I don't think so.

Whether a good submission gets noticed and upvoted early depends quite a bit on what time of day it is submitted. I have formed the opinion that 0700 eastern might be the best time, as the east coast sees it, UK sees it, and if something gets a few upvotes within a few minutes, it is likely to hit the front page, at which point it much more likely to be evaluated on its merits.

Keep in mind that upvotes come from two sources. First is if someone is viewing an HN page, perhaps /newest and likes the article, clicks uparrow. The other way is if another user submits the same link, it gets an automatic update. So if a popular twit is noticed and submitted, the chance of a handful of submissions in a few minutes is higher.

And I have a suspicion that if one were to tally the number of times the uparrow was hit on the front page as opposed to the /newest page, the front page would be vastly higher.

</comment></comment></comment></comment><comment><USER20> _You are the victim of branch prediction fail._

"Failure" is a perfectly serviceable word and the cutesy "fail" is just annoying in a technical discussion like this.

<comment><USER21> > _"fail" is just annoying in a technical discussion_

I am sorry but you know what is REALLY annoying? Nit-picking at what must be the atomic level considering the depth and detail of this perfectly brilliant reply.

Technical discussions need a LOT more people like Alexander Yee aka "Mysticial" - and technical discussions sure as hell do not need ANY people like you complaining about the choice of words or slang as long as it is perfectly clear and understandable for the target audience and on that account he more than delivered.

<comment><USER20> _Nit-picking at what must be the atomic level considering the depth and detail of this perfectly brilliant reply._

Can't have it both ways. The more depth and detail a discussion tries to present, the _more_ important it is to get the details right. The medium (StackExchange) even recognizes this and has editing, correcting, and collaborative fixing built into its DNA, for that very reason.

The more famous this response becomes, the more ESL readers there will be. Why trip them up this way? There's just no reason to use "fail" like this here.

"Branch prediction fail" is not actually a very clear or precise phrase. It could mean the failure of a single branch prediction attempt; it could mean the failure of the technique in a particular case, and it could refer to an author's opinion that it was a failure in general. Why muddy the waters?

<comment><USER21> You are being pedantic to an extreme I haven't even encountered amongst the worst "grammar nazis" and you are using nothing but FUD arguments. So his reply was "muddied" up by that? The amount of upvotes and comments and people linking to the reply very strongly beg to differ. It is a brilliant reply and I am not a native speaker, even I understood it perfectly well too so he must have done something right.

You do not have a single reason to complain and the sooner you realize that, the better it will be for your own sake.

<comment><USER20> So, to recap: I voiced my irritation at a particular word choice, and the conclusions are:

1) I am therefore _worse than the Nazis_

2) _I'm_ the one who's being unreasonable

3) If something is _popular_ it follows that it is also _perfect_ and cannot be improved in any way.

Thanks for clearing that up.

</comment></comment></comment></comment></comment><comment><USER22> If you're looking for a good online course on how computer systems work from AND gate scale - up to data center size, the iTunesU course <<"Computer Science 61C", UC Berkeley, w/ Daniel Garcia>> is really informative and fun to watch.

[http://itunes.apple.com/de/itunes-u/computer- science-61c-001...](http://itunes.apple.com/de/itunes-u/computer- science-61c-001-fall/id461056991)

</comment><comment><USER23> Not to hate on the great top-voted answer, but isn't it a bit strange that it was posted a mere five minutes after the question?

<comment><USER24> Not really, a very common pattern on SO is to jump on a question to which you know you have a good answer, put something very short (but correct) and then edit in a better, longer response. This avoids "losing" initial votes from people looking at responses while you're fleshing out the description, adding benchmarks, etc...

It's definitely gaming the system, but it's not really suspicious (in the sense of "guy asks a question and immediately puts a response he had pre- written in a text editor" suspicious)

<comment><USER25> The "suspicious" case you mention is explicitly supported by the system; you can actually write the question and answer at the same time if you want. The normal content rules still apply, of course; if the question isn't about an actual problem, you're more likely to win a bundle of downvotes.

</comment></comment></comment><comment><USER26> The fact that someone would take the time to help someone in such depth restores my faith in humanity. With a picture to boot.

<comment><USER21> The fact that someone would complain about the use of the word "fail", on HN no less, should nicely even out whatever faith was restored, unfortunately...

<comment><USER27> Nope, you can look away from the troll and focus your attention on the good stuff. You have the power to weigh good stuff more heavily.

</comment></comment></comment><comment><USER28> The answer focuses on branch misprediction but there is also another factor - CPU data prefetch.

In order to offset increasing CAS latencies (largely due to the increasing number of channels) modern CPU's are greedy (in a good sense) and fetch more pages per memory controller read request than are necessary.

If your data is laid out continuously (in this case - sorted), chances of the pages you need next being primed via prefetch are greatly increased and further read requests to the memory controller may be reduced or avoided.

I benchmarked the effect of data prefetching on modern processors with large data structures (usually limited to L1d cache lines):

[https://github.com/johnj/llds#l1-data-prefetch- misses-200000...](https://github.com/johnj/llds#l1-data-prefetch- misses-200000-per-hardware-sample)

A plausible analogy for CPU prefetching would be "readahead" mechanisms implemented by modern filesystems and operating systems.

<comment><USER29> While prefetch is an important factor, it is not the case here since the data array is accessed sequentially in both the sorted or unsorted scenarios. Also the initial number generation loop already primes the data in the memory for either scenario.

</comment></comment><comment><USER30> Alexander J. Yee, who answered the question is a student who calculated Pi to 5 trillion digits:

[http://www.geekosystem.com/pi-five-trillion-digits- alexander...](http://www.geekosystem.com/pi-five-trillion-digits-alexander- yee-shigeru-kondo/) _The main challenge for a computation of such a size, is that both software and hardware are pushed beyond their limits. For such a long computation and with so much hardware, failure is not just a probability. It is a given._

</comment><comment><USER31> Sorted arrays make it easy for a simple branch predictor guess correctly, if the branch predictor guesses correctly then you avoid a pipeline stall.

Assuming the median value in the array is 128 then for the sorted case the first half of the array will have one cache miss and the second half will have one cache miss.

If the OP optimized their code, sorted/filtered the array and then summed only those values greater than 128 it would be obvious why sorting is faster.

</comment><comment><USER32> What about the time taken to do the sorting? Is there some rule of thumb that hints at whether taking the time to sort the data is going to be a net win?

<comment><USER33> I would expect sorting to take a lot longer than just summing.

</comment></comment><comment><USER34> Interesting test, but it's not news if you know something about code optimization.

Caching of the data may also play a part on similar cases (but not this one - or better, the effect is negligible)

Here's a little test that can be tried. Make the test result (inside the if) be true or false alternately (for example, sum only if the index is even), see how long it takes.

Spoiler: modern branch predictors can detect and predict cyclic conditions

</comment><comment><USER35> TL;DR Would it not be great that we change the branch prediction implementation to switch on/off automatically(with every new process)? This would be difficult to trace or even implement but I'm just guessing. The CPU stops using branch prediction if a lot of it's predictions are failing but starts again after some time.

</comment><comment><USER36> Branch prediction is not the only reason this is faster and probably not even the biggest reason.

Since the array is sorted,

        if (data[c] >= 128)

will evaluate to true consecutively. When the cache requests something from memory, it will request blocks containing multiple words at a time. Since every data[c] that needs to be added to sum is in a contiguous piece of memory, the code is minimizing the number of times a block is transferred from memory to the cache. This is the concept of spatial locality[1].

[1][http://en.wikipedia.org/wiki/Locality_of_reference#Locality_...](http://en.wikipedia.org/wiki/Locality_of_reference#Locality_of_reference)

<comment><USER37> No. The "true" or "false" value of a simple expression isn't stored in memory. It's stored as a bit in the status register as a side effect of the "cmp" instruction†. It is no faster for "cmp" to set "true" than it is to set "false".

† _(CMP is actually setting carry, overflow, sign, zero, &c; "truth" or "falsity" is decided by the specific conditional jump, here JL, which checks if sign != overflow)._

<comment><USER36> That's not what I was saying. I was indicating that more memory accesses would be required in unsorted data for the `sum += data[c]` computation, obviously overlooking that data[c] is already in a cache, and probably in a register as a result of the comparison with 128.

<comment><USER38> But that is true whether the array is sorted or not; the indexes (and therefore memory accesses) in the loop are still in order even if the contents of the array are not sorted.

<comment><USER36> I know. Let me state it another way.

"I was wrong, but not in the way that you(tptacek) understood me to be."

<comment><USER37> For the record: I acknowledge that you were not wrong in the way I understood you to be wrong; I responded because I got the sense that the thread was now discussing the storage of boolean expression results, and klaxons inside my nerd brain started going off.

</comment></comment></comment></comment></comment></comment><comment><USER39> a very broad intuition is that less entropy in the input, more predictability in executing algorithms on that input, more regularities to exploit

</comment><comment><USER40> Where can I learn more about this?</comment>