On the cruelty of really teaching computer science (1988) - <AUTHOR> https://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036.html <comment> vezzy-fnord Previously on Hacker News (in no particular order):

[https://news.ycombinator.com/item?id=8909529](https://news.ycombinator.com/item?id=8909529)

[https://news.ycombinator.com/item?id=8872085](https://news.ycombinator.com/item?id=8872085)

[https://news.ycombinator.com/item?id=2122826](https://news.ycombinator.com/item?id=2122826)

[https://news.ycombinator.com/item?id=8368788](https://news.ycombinator.com/item?id=8368788)

[https://news.ycombinator.com/item?id=6701607](https://news.ycombinator.com/item?id=6701607)

[https://news.ycombinator.com/item?id=6838434](https://news.ycombinator.com/item?id=6838434)

[https://news.ycombinator.com/item?id=3553983](https://news.ycombinator.com/item?id=3553983)

[https://news.ycombinator.com/item?id=1666445](https://news.ycombinator.com/item?id=1666445)

[https://news.ycombinator.com/item?id=43978](https://news.ycombinator.com/item?id=43978)

[https://news.ycombinator.com/item?id=2090256](https://news.ycombinator.com/item?id=2090256)

[https://news.ycombinator.com/item?id=1989473](https://news.ycombinator.com/item?id=1989473)

<comment><USER2> If only we could remember history...

</comment></comment><comment><USER3> Most of Dijkstra's writing is overly melodramatic and this is no exception. Now that computing is even more ubiquitous his elitist attitude about programmers is even less relevant. Driving a car does not require a degree in mechanical engineering and thermodynamics and similarly writing programs should not require a degree in mathematics and logic. Also, reasoning by analogy and intuition is one of the better ways to solve problems. If it is good enough for people like Einstein and Feynman then it is good enough for me and Dijkstra in this case is just plain wrong.

</comment><comment><USER4> > Carl Friedrich Gauss, the Prince of Mathematicians but also somewhat of a > coward, was certainly aware of the fate of Galileo —and could probably have > predicted the calumniation of Einstein— when he decided to suppress his > discovery of non-Euclidean geometry, thus leaving it to Bolyai and > Lobatchewsky to receive the flak.

There's a long history of mathematicians and scientists suppressing their results.

Copernicus didn't publish about heliocentrism until he was on his deathbed.

Newton started developing calculus in 1665 but didn't publish his theories and methods for calculus until 1693.

Darwin didn't publish until 1858, when Alfred Wallace had begun to rediscover Darwin's theories. Darwin had already theorized about natural selection for himself and privately among naturalist friends since he returned from the Beagle in 1836.

I am not a physicist, but my understanding is that Einstein's formulas showed him that the universe was expanding. He didn't trust what his own formulas, which were radical at the time, were showing him, so he modified his equations so that the universe would be stable - adding a cosmological constant. Observations by astronomers like Hubble later showed him that his original formulas were correct (my understanding of the field equations are shallow so I may have mischaracterized this).

<comment><USER5> _> Einstein's formulas showed him that the universe was expanding._

More precisely, they showed him that there was no solution describing a static universe; the universe would have to be either expanding or contracting. Nothing in the math gave any reason to favor expansion over contraction; it's just that, since we now know the universe is expanding, that's the possibility we always focus on when talking about this episode.

 _> He didn't trust what his own formulas, which were radical at the time, were showing him_

I don't think it was that he didn't trust the formulas; he had already explored other solutions that were giving correct predictions. It was just that he thought the universe should be static, so he didn't like the fact that there wasn't a static solution to his original equations.

 _> Observations by astronomers like Hubble later showed him that his original formulas were correct_

More precisely, they showed that the universe was expanding, so there was no need to insist on having a static solution to the equations. But, as FreeFull pointed out, we now know that there is a nonzero cosmological constant, so Einstein's original equations, without that constant, weren't quite correct anyway.

</comment></comment><comment><USER6> "It is probably more illuminating to go a little bit further back, to the Middle Ages. One of its characteristics was that 'reasoning by analogy' was rampant; another characteristic was almost total intellectual stagnation, and we now see why the two go together. A reason for mentioning this is to point out that, by developing a keen ear for unwarranted analogies, one can detect a lot of medieval thinking today."

Nice.

</comment><comment><USER7> This speech was so quintessentially Dijkstra. Here's where it got _super_ -Dijkstra:

> Right from the beginning, and all through the course, we stress that the > programmer's task is not just to write down a program, but that his main > task is to give a formal proof that the program he proposes meets the > equally formal functional specification.

And so we produce programmers who are helpless without formal functional specifications, and are therefore incapable of writing any software in the real world, with conflicting, underspecified, and post-hoc requirements.

But more importantly: this sucks all of the joy out of it! Who here got into programming out of a passion for translating formal functional specifications into formal proofs? How can any of the _magic_ of software survive such a mindset?

<comment><USER8> Honest question, isn't it a historical bias ? maybe in his days programming projects were devoted to huge, complex and costly problems that were always pretty strictly defined, unlike the late 70s ~Apple/Microcomputer days ?

> But more importantly: this sucks all of the joy out of it! Who here got into > programming out of a passion for translating formal functional > specifications into formal proofs? How can any of the magic of software > survive such a mindset?

Sure, but I do like the bliss of watching a tiny piece of code knowing it's sound by induction.

<comment><USER9> > maybe in his days programming projects were devoted to huge, complex and > costly problems that were always pretty strictly defined, unlike the late > 70s ~Apple/Microcomputer days ?

Huge? Not by today's standards. Complex? Again, not by today's standards, but they didn't have today's libraries to build on. Costly? Definitely.

Pretty strictly defined? They tried to be, but the gaps always showed. Certainly they were _not_ strictly defined by Dijkstra's standards.

</comment></comment></comment><comment><USER10> I think Dijkstra's viewpoint is a bit "elitist" here. He talks about what are the best things students should learn (formal methods etc.) instead of talking about what would be the best ways to teach those to them. He takes the viewpoint of the professor, unsurprisingly, not the viewpoint of the student.

<comment><USER11> > I think Dijkstra's viewpoint is a bit "elitist" here.

I don't think that there's any need for scare quotes; Dijkstra was consistently and unapologetically elitist, and probably would (and maybe even did?) say as much himself.

</comment></comment><comment><USER12> If you have a hammer, everything looks like a nail. If you're a mathemetician, everything looks like a math problem.

.

We're not really that special.

.

.

 _The programmer is in the unique position that his is the only discipline and profession in which such a gigantic ratio, which totally baffles our imagination, has to be bridged by a single technology. He has to be able to think in terms of conceptual hierarchies that are much deeper than a single mind ever needed to face before._

I disagree.

A city planner does not need to know or care how to make bricks. An application developer does not need to know or care how HashMap<> internals work (well, not after they pass the interview at least).

An identical program can run on 1MB of data or 1TB of data, without changing it or adding layers of abstraction. A city with 4x as many people is not just 4 1x cities stuck together.

Complexity and the number of abstraction levels _do not_ necessarily scale with system size in elementary particles.

.

 _To this I should add that, to the extent that we view ourselves as mechanisms, we view ourselves primarily as analogue devices: if we push a little harder we expect to do a little better. Very often the behaviour is not only a continuous but even a monotonic function: [...]_

Of interest is that while this view may be _popular_ , I think it's beginning to become apparent that it's not entirely _accurate_. Besides more well-known effects like overtraining, there's also reduced productivity with too-high work hours, reduced creativity in the face of excessive pressure, etc. And it's not just the "monotonic" part that's questionable -- things like burnout and the traditional "mid-life crisis" and mental breakdowns hint that we're not even particularly continuous.

 _In the discrete world of computing, there is no meaningful metric in which "small" changes and "small" effects go hand in hand, and there never will be._

This also applies to complex mechanical systems, and to anything with DNA. Whatever mutations cause cancer can't be that big a change, or they wouldn't happen so often.

.

 _The nice thing of this simple change of vocabulary is that it has such a profound effect: while, before, a program with only one bug used to be "almost correct", afterwards a program with an error is just "wrong" (because in error)._

This is very silly. Not all bugs/errors render a program wholly useless.

.

 _[Don 't anthropomorphise programs/computers.] ... ...deal with all computations possible under control of a given program by ignoring them and working with the program. We must learn to work with program texts while (temporarily) ignoring that they admit the interpretation of executable code._

This depends entirely on what you're doing. Are you trying to study computation using some program as an example, or are you trying to accompilsh some real-world effect with that program?

 _A programming language, with its formal syntax and with the proof rules that define its semantics, is a formal system for which program execution provides only a model. It is well-known that formal systems should be dealt with in their own right, and not in terms of a specific model. And, again, the corollary is that we should reason about programs without even mentioning their possible "behaviours"._

Since the current context is the evils of anthropomorphization, what is the assumed use that "formal system" is a better (more useful) model than "intentional actor"? Insistence on formality becomes especially weak when the system _can 't_ be formally defined, either because the behavior depends too much on (unknown) input data or because the formal spec is intractable.

.

.

 _Right from the beginning, and all through the course, we stress that the programmer 's task is not just to write down a program, but that his main task is to give a formal proof that the program he proposes meets the equally formal functional specification._

We've since discovered that all this will accomplish is to pass the buck to the people who wrote the invariably-buggy (I mean, erroneous) spec. Which since it's fully formal, could just be mechanically translated into a program anyway.

.

.

I think some of the things I learned were pretty good (and in a pretty good order): use of mathematical induction to prove that recursive algorithms work (introductory); complexity analysis (same introductory course); lambda calculus and compilers (4xx level). That's certainly better than trying to focus on formal methods, when all they can accomplish is to _move_ the informality. I'd have also benefited from something on creating and refining models, or turning a problem into a spec (because really, building the right thing is always harder than building the thing right). For CS rather than CompEng, I'd also suggest more depth on discrete math than I took, and whatever kind of math is appropriate for distributed systems (queues and graphs and flow rates and stuff).

<comment><USER9> > This depends entirely on what you're doing. Are you trying to study > computation using some program as an example, or are you trying to > accompilsh some real-world effect with that program?

I don't think Dijkstra was actually interested in producing programs that accomplished some real-world effect.

> That's certainly better than trying to focus on formal methods, when all > they can accomplish is to _move_ the informality.

Correct, and very insightful. (Roger Penrose made the same point in "The Emperor's New Mind - at some point, it all starts with an informal idea of what the program is supposed to do.)

</comment></comment><comment><USER13> This is a good read.</comment>


Hate the peer-review process? Einstein did too - <AUTHOR> https://theconversation.com/hate-the-peer-review-process-einstein-did-too-27405 <comment><USER2> > In his brilliant career, the only time his work was subjected to blind peer > review – the authors don't know the reviewers and vice versa - he showed > contempt for what is now the gold standard of science ... The story reminds > us that double-blind peer review is only a relatively recent invention.

I believe the author has confused anonymous (or "single blind") peer review with double blind peer review. That is, I don't think the Einstein-Rosen paper was subject to double blind peer review, which didn't really come into use for another ~20 years, and in sociology.

Double blind peer review is not the gold standard for peer review. It is the gold standard for medical trials. I suspect the author has confused the two.

> He then send it to a reviewer for comments – his selected reviewer was > probably the famously gossipy Howard Percy Robertson, one of Einstein's > colleagues at Princeton.

"Probably"? What is the source for this doubt? [http://www.geology.cwu.edu/facstaff/lee/courses/g503/Einstei...](http://www.geology.cwu.edu/facstaff/lee/courses/g503/Einstein_review.pdf) is a copy of the September 2005 Physics Today article on the topic. It shows a copy of the Physical Review logbook. The caption for Figure 5 is:

"""The Einstein–Rosen article was received by the journal on 1 June 1936. After a delay of more than a month, John Tate sent a referral to Howard Percy Robertson on 6 July, finding him in Moscow, Idaho, on vacation after a sabbatical at Caltech. Robertson returned the manuscript and his review to Tate on 17 July. Six days later the package was sent back to Einstein."""</comment>


The NHS is selling your private data – here's the price list [pdf] - <AUTHOR> http://www.hscic.gov.uk/media/12443/data-linkage-service-charges-2013-2014-updated/pdf/dles_service_charges__2013_14_V10_050913.pdf <comment><USER3> I'm not worried, there's absolutely no way anyone could identify you with only your NHS number, your date of birth, your postcode, your gender and ethnicity, your medical diagnoses (including cancer and mental health) and any complications, your referrals to specialists, your prescriptions, your family history, your vaccinations and screening tests, your blood test results, your body mass index (height/weight) and your smoking/alcohol habits....

Oh... wait....

</comment><comment><USER4> A comment on one of the Guardian articles:

J_smudger 24 January 2014 3:00pm

I think there is confusion amongst some commenters here. This comes from reading a large amount of literature from the relevant pages on the NHS / Health and Social Care Information Centre (HSCIC). The HSCIC are basically a repository in Leeds, where all this information will be stored.

Your GP records are going to the HSCIC as pseudoanonymised information, which as has been said does indeed include your NHS number, date of birth and postcode. The HSCIC will then build up a database of this information. They can indeed pass on certain of this information to certain external interested parties, although when they do this the data becomes truly anonymised as opposed to pseudoanoymised. You can read about this in the NHS published guidelines (although not in the rather patronising leaflet), as well as from the documentation of the HSCIC and the government itself.

To quote the HSCIC:

        we take out details that could identify you before we make any information available

At the NHS:

        there are no personal details such as your date of birth and postcode included... We would never publish this type information because there is a risk that you might be identified.

The HSCIC can only release identifiable information when (1) you specifically ask them to, or (2) hypothetically, when there is a national emergency such as a highly virulent pandemic. This would require a legal process.

[http://www.nhs.uk/NHSEngland/thenhs/records/healthrecords/Pa...](http://www.nhs.uk/NHSEngland/thenhs/records/healthrecords/Pages/care- data.aspx)

[http://www.hscic.gov.uk/article/3399/Rules-for-sharing- infor...](http://www.hscic.gov.uk/article/3399/Rules-for-sharing-information)

Or if you have a hour to spend read this:

[http://www.hscic.gov.uk/media/12931/Privacy-Impact- Assessmen...](http://www.hscic.gov.uk/media/12931/Privacy-Impact- Assessment/pdf/privacy_impact_assessment_2013.pdf)

... or perhaps just sections 3.3.4. and 3.3.5.

<comment><AUTHOR> Regulation 5b allows the Secretary of State for Health to disclose confidential patient information for any medical purpose. No need for a national emergency.

[http://www.legislation.gov.uk/uksi/2002/1438/regulation/5/ma...](http://www.legislation.gov.uk/uksi/2002/1438/regulation/5/made)

:edit: gohrt pointed out that I'd overlooked the restriction to medical purposes in regulation 5, thanks.

<comment><USER5> Not "for any reason".

Please edit your commment for correctness:

""" General

"confidential patient information may be processed for medical purposes in the circumstances set out in the Schedule" """

</comment></comment></comment><comment><USER6> I would just like to point out that even if they "anonymize" the records it's generally not that hard to de-anonymize data.

During the Netflix prize, randomly generated IDs were eventually matched to people based simply on movie ratings and matching public information in other public sources:

[http://www.wired.com/politics/security/commentary/securityma...](http://www.wired.com/politics/security/commentary/securitymatters/2007/12/securitymatters_1213)

With medical data, it will probably be trivial (maybe easier or more appealing to insurance companies?).

We're a lot more unique than we think. Reminds me of this EFF project:

[https://panopticlick.eff.org/](https://panopticlick.eff.org/)

Some companies will probably resell this information to potential employers, banks (there goes your loan), etc.

Well, that's going to suck for people in the UK.

<comment><USER7> Companies are TERRIBLE at this. We used a company to do an employee feedback survey. They promised us that the data would be delivered in a 100%, completely anonymized format. We sit down to go over the results and slide number one is "Men were a 100% approval while women were 73%". I'm the only male on my team. How in the world is this anonymous?

<comment><USER8> I know in my C.S. courses most of us guys/girls wont put down our gender on surveys so the 4 or 5 females in the room can maintain anonymity.

</comment></comment></comment><comment><USER9> This would be straight up illegal in the US due to HIPAA, which guarantees a patient's right to privacy.

HIPAA = [http://en.wikipedia.org/wiki/Health_Insurance_Portability_an...](http://en.wikipedia.org/wiki/Health_Insurance_Portability_and_Accountability_Act)

<comment><USER10> No it isn't. U.S. health data can be sold if it is de-identified or with a patient's permission.

<comment><USER11> But this data is not de-identified. They explicitly list prices for data with confidential, patient-identifying information.

<comment><USER12> The main page, [http://www.hscic.gov.uk/dles](http://www.hscic.gov.uk/dles) , states

> The data we supply is normally pseudonymised. We only provide identifiable > data when there is a lawful basis to do so i.e. with patient consent, > approval under section 251 of the NHS Act 2006 which enables The Health > Service (Control of Patient Information) Regulations 2002, or where > appropriate statutory regulation is in place.

This "About Section 251" page, [http://webarchive.nationalarchives.gov.uk/20130513181011/htt...](http://webarchive.nationalarchives.gov.uk/20130513181011/http://www.nigb.nhs.uk/s251/abouts251/index_html) , states

> Section 251 came about because it was recognised that there were essential > activities of the NHS, and important medical research, that required use of > identifiable patient information but because patient consent had not been > obtained to use people's personal and confidential information for these > other purposes, there was no secure basis in law for these uses. [NB. There > are a few exceptions where there is a legal basis for disclosure e.g. > reporting of notifiable diseases]. Section 251 was established to provide a > secure legal basis for disclosure of confidential patient information for > medical purposes, where it was not possible to use anonymised information > and where seeking consent was not practicable, having regard to the cost and > technology available.

<comment><USER13> >" _and where seeking consent was not practicable,_ " //

As they have your contact details and next of kin details then seeking consent must be quite practicable.

I expect them mean "commercially financially viable". Those people the NHS can't simply look up a phone number for (from doctor's surgery records) and ask (or ask their guardian/parent) must be in the few hundreds [of those they have sufficient medical information for to be used in a scientific study].

<comment><USER12> I guess that's what it says, practicable with regard to cost.

To look at one random example, the first study on the list of approvals in 2013 was the "ETPOS: European Transfusion Practice and Outcome Survey". Apparently they took data from 10,000 patients who had blood transfusions and tried to see if there was any correlations between practices such as "ratio of red blood cells to other blood component therapy, such as plasma and platelets" and health outcomes. I guess it was deemed that phoning each of the patients was impractical.

<comment><USER13> Robot Dialer: The NHS wish to sell your medical data for use in a study of people who had blood transfusions. Press 1 to accept, 2 to refuse, 3 to speak to an agent.

Robot Dialer: You pressed 1 to accept; can we use your data for future studies? Press 1 to accept, 2 to refuse, 3 to speak to an agent.

Robot Dialer: You pressed 2, can we contact you to ask about using your data in specific studies in the future? Press 1 to accept, 2 to refuse, 3 to speak to an agent.

Umpteen marketing companies appear to be able to afford to do this sort of calling (yes even though it's against the law for them to contact me as I'm on the no-call database [which wouldn't apply to the NHS]).

If it's too costly then the studies can hardly be worthwhile? Remember the NHS wasted £10 Billion on a single IT project over the last 10 years. What would this auto-dialer have cost? £10k in "management", couple of thousand in IT staff and set-up (join study NHS numbers with main database ID table and contact info tables, select phone numbers; set-up dialer script, test, initiate) maybe £500 in direct call costs. They most likely already have systems in place to do auto-dialed calls for disease outbreaks [UK Environment Agency use one for flood warnings].

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I fully expected to read the details and see that the headline was some sort of hyperbole, as these things nearly always are. I'm still hoping someone will tell me this isn't real.

This seems downright evil. Disgusting. There is no justifiable reason for this data to be available in any sort of unanonymized form. Everything that is justifiable that can be achieved with it in anonymous form can be achieved with it anonymized.

The terrible part is that there is a good reason for a program like this. There are real reasons to collect and know this kind of data - it can make a huge difference to human health and well being. And that is why this is so bad. It's going to set back participation in any sort of electronic health record all around the world, if people see such a high profile program manifest as a privacy disaster.

<comment><USER15> >I fully expected to read the details and see that the headline was some sort of hyperbole

It really is, see the source page for more details:

[http://www.hscic.gov.uk/dlesaac](http://www.hscic.gov.uk/dlesaac)

The misunderstanding going on in the comments seems to be stemming from a failure to distinguish between personal identifiable data and personal confidential data.

The former: "This includes patient identifiable data, such as:

NHS number Name Address Postcode Date of Birth Date of Death"

and the latter: "Personal confidential data also includes sensitive data which may include items such as:

Racial or ethnic origin Political opinions Religious or other similar beliefs Physical or mental health condition Sexual life Criminal record"

The patient identifiable needs explicit permission from the patient in order to obtain, patient confidential needs a good legal reason + reviewed application.

<comment><USER16> _The patient identifiable needs explicit permission from the patient in order to obtain_ \- please could you link me to the official document that states this, specifically in the context of care.data?

</comment></comment></comment><comment><AUTHOR> A good explanation of the scheme, from Dr Neil Bhatia can be found at [http://care-data.info/](http://care-data.info/)

It includes details of how to opt-out.

<comment><USER17> It's a long site so I'd just like to highlight the following:

"The data extracted - your Primary Care Dataset - will include the following:

Your NHS number Your date of birth Your postcode Your gender and ethnicity Your medical diagnoses (including cancer and mental health) and any complications Your referrals to specialists Your prescriptions Your family history Your vaccinations and screening tests Your blood test results Your body mass index (height/weight) Your smoking/alcohol habits"

\---

Go to that site. Opt out here [http://optout.care- data.info/](http://optout.care-data.info/). It's really simple.

<comment><USER16> If you prefer, you can just write a letter to your surgery:

\- State that you wish to opt-out of care.data

\- Request that both the 9Nu0 and 9Nu4 codes are added to your GP records

\- Remember to include full names and DOBs (and your address if you are happy to)

<comment><USER18> The conundrum I have is knowing this has been done.

I wrote a letter with all of this, took it to the GP surgery. The receptionist was bemused. There has been no acknowledgement. Can I tell if I've been opted out, without hassling the already-overworked reception staff?

<comment><AUTHOR> Try talking to your practice manager - mine is very helpful and (s)he'll hopefully be much more clued up about this than the reception staff.

</comment></comment></comment></comment></comment><comment><USER19> Whoever gets their hands on this data should build a "20 Questions" game, to identify a person's NHS number. Knowing something of my neighbour's recent medical history, I'm pretty sure it'd take 10 questions or less.

Are there any restrictions on publishing the data? I can't find licensing terms.

<comment><AUTHOR> Not exactly licensing terms but page 3 of the price list states that extracts subject to an annual fee will continue to be charged that fee until it is certified that all hard and soft copies of the data have been destroyed.

Other than that I'm guessing they will enforce some pretty draconian restrictions on publishing and sharing the data since doing so would undermine their ability to sell the data.

<comment><USER5> So the UK govt aren't even claiming that this data is public information, and charging a processing fee for it -- the govt is claiming that citizen's personal health information is _owned by the goverment_ with privileges to restrictively license it to commercial customers!

</comment></comment></comment><comment><USER20> I think the thing that's missing from much of the discussion is that all released information is subject to a very clear contractual agreement and for specific purposes. The agreements limit the ability to link supplied data with anything else. These contracts and use of data are subject to privacy group oversight, managed by the NHS.

The intended use is not that insurance companies can link your medical data against you and then charge you more (or any variant on that). Instead, the intended use is that companies with clear information controls can perform useful research more cheaply, and stop guessing at cause and effect. I personally support that intent, and am interested to see what comes out of it.

What's to stop the companies just doing whatever seems to get them the most money? In my opinion, it'd be the fact that failing to stick within the agreement would cause existential risk to the company. I think that courts, government, the NHS, and UK society at large would come down VERY heavily on any company contravening their contracts. Companies are going to spend significant effort ensuring their company doesn't disappear overnight in a storm of lawsuits with the directors in jail.

Companies wouldn't do this for the same reasons that Seagate doesn't sell the data off RMA'd hard drives on the open market.

I trust the relevant public bodies in the UK to protect my interests here. You may not, of course.

<comment><USER21> Let's be clear, the intention is for the UK Government to make money off your medical data.

If Seagate wanted to make money off your RMA'd hard drive and they thought the data on it would do the trick, you can bet it would be for sale on the open market.

If the law says that is illegal, Seagate does not have the option to change it. However, the Government can simply change the law to make whatever they want to do 'legal' and their problem is solved. That's essentially what they've done here.

Large 'healthcare' companies interested in this data are more than just health providers, they have multiple divisions with multiple competing and tangential aims and targets. Just because a piece of paper says it can only be used in one way, that is not going to stop the re-use (and leaking) of the data.

Remember the UK had bankers totally screwing the country and got rewarded with massive bail-outs - I don't recall any jail time for their bad behaviour [in the UK]; quite the reverse. Any social science student will be able to cite many examples of companies shielding individuals from the consequences of their bad behaviour - it's a whole subject area.

The UK government sets up QUANGOs specifically to shift liability and risk to prevent consequences; a Scottish care home where elderly people were burned to death escaped prosecution as the legal entity was simply shut down and dissolved prior to the court case starting [this did bring about legislation changes to close that avenue in Scotland [http://www.bbc.co.uk/news/uk- scotland-17740645](http://www.bbc.co.uk/news/uk-scotland-17740645)]. There are dozens of ways to get away with abusing the data and walk away free - <USER2> you're going to make a lot of money, you can afford good lawyers to help you prepare well ahead.

Why would it be different for your health data ?

<comment><USER20> Sorry, but I don't agree with the premise and the first line of your comment. Much of the rest I do agree with.

The NHS (not "the government" \- which is an emotionally charged noun in this sort of circumstance) is selling the data. They are in financial difficulty, yes, but they are also responsible for broad social-health in the UK.

The NHS is in an almost unique position world-wide, in that they have access to high quality data that can dramatically improve health at an international level. They aren't, however, a research group. Companies just do research better than government departments, and finding a balanced way to improve access to the data and improve social health is critical to the NHS's future as the population ages.

This is why they are selling health data, imho.

I think there's a balance to be struck. The global and NHS specific improvements in health need to be balanced against individual privacy.

Unfortunately, the only way to do this is through "pieces of paper" (again, an emotive term).

It's also worth mentioning that many of these pieces of paper have already been in place for years, where they have been sharing hospital data. So to some degree this extends an exiting structure that is already working. It's just more emotive to many people since it involves a centralised location, and their local GPs.

I'd rather have a centralised location with oversight fighting down a multinational, than my local GP trying to manage legal contracts with them.

It's difficult to respond to your specific examples. Some are completely valid. Some are (imho) not. "Mistakes were made" and mistakes will be made in the future.

It's complicated, and it's a balancing act. Personally, I think it's the right balance.

</comment></comment></comment><comment><USER9> Opportunity here for the motivated - the prices seem high, so arbitrage the price by reselling the data at lower prices to multiple buyers.

Bonus: you could set up a system where a person's data gets cheaper as more people query it!

(Not for me thanks.)

</comment><comment><USER22> What would be interesting would be to write up this 'opt out' procedure, slightly disguised, as part of a research proposal and submit it to the UK ethics committees. I'd be shocked if they don't all reject it. Any UK academics - with enough tenure that it won't bork their career - up for that?

</comment><comment><USER23> I await the shit storm when a public figures medical records are mined by the media divulging something controversial.

<comment><USER13> People with rare diseases, especially multiples, must be reasonably easy to deanonymise. Also joining the data with newspaper reports of crimes (perhaps only ones that are pertinent are mentioned, eg harassment of hospital staff) or hospitalisations would seem likely to deanonymise quite a few records.

<comment><USER3> Post code. Date of Birth. Gender.

What more do you need?

</comment></comment></comment><comment><USER24> [http://medconfidential.org/how-to-opt-out/](http://medconfidential.org/how- to-opt-out/)

<comment><USER25> Hmm. Interesting. I remember when I recently registered with a new doctor, I was asked directly if I wanted to opt out from my medical information collected by my GP being digitally accessible by hospitals, etc. I'll have to admit: I chose not to opt out. Thinking "it's about time they join us in the digital age!" Plus, from the perspective of my health, this seemed like a positive move overall.

Of course now that I read a bit more into it, I am less sure. But I do find the above link a little fear-mongery.

Incidentally, this document has some interesting insight into the position of the hscic. [http://www.hscic.gov.uk/media/12931/Privacy-Impact- Assessmen...](http://www.hscic.gov.uk/media/12931/Privacy-Impact- Assessment/pdf/privacy_impact_assessment_2013.pdf). I am a little tickled by this statement about preventing the data falling into the wrong hands: "The Government itself could be considered a pair of 'wrong hands' with questions raised over whether it would have access and therefore would be able to misuse or exploit the data".

Not sure how they're mitigating against that risk...

</comment></comment><comment><USER26> Let's say I believe that this data truly is anonymised (read: cannot be traced back to an individual in any way). I still have a problem that MY data is being sold by SOMEONE ELSE. The opt-out nature of this process stinks. It feels as though it's someone else's data by default unless I kick up a fuss.

</comment><comment><USER27> You can choose to have your information not included, it's an opt out system.

Form is here:

[http://www.connectingforhealth.nhs.uk/systemsandservices/scr...](http://www.connectingforhealth.nhs.uk/systemsandservices/scr/staff/aboutscr/comms/pip/optout.pdf)

<comment><USER28> You are spreading misinformation, almost definitely not on purpose.

Read about the correct opt-out form here: [http://optout.care- data.info/](http://optout.care-data.info/)

<comment><USER29> And of course you're included automatically, but to opt out it is most painful way to do it.

And not to mention that you need to know about it in the first place.

</comment></comment></comment><comment><USER30> Would this enable the private industry to create services and analyses based off this cheaply available data?

Yes, it seems heinous at first, but are there legitimate, palatable Big Data opportunities here, assuming the data is properly anonymized?

<comment><USER21> Did you look at the price list ?

Standard extract – no personal confidential data £9,565

Alternatively for just under £1,000 more :

Standard extract – containing personal confidential data £10,453

They're specifically enticing people to purchase the confidental data version since it is only 10% extra to get all the juicy information.

Trouble connecting people to their parents, siblings, children, (ex)partners ? Simple, they'll even do that for you - look at Patient Tracking, Cohort Event Notification (!) etc.

The value of this data to marketers (e.g. health insurance, private hospitals - which do exist in the UK, etc. makes the price list charges trivial and insignificant to just slurp up everything they can and start targeting people). Want someone to try and sell you cancer insurance 2 weeks after your mother dies of breast cancer ? Cohort event notification report makes this simple.

Remember the toothpaste does not go back into the tube - once the data is sold, it's basically wild and free for all sorts of use and abuse. You have absolutely no guarantee it will only be used by benign 'good actors'.

edit:spelling

<comment><USER31> You can't just buy what you want from them without a legitimate legal basis.

[http://www.hscic.gov.uk/dlesaac](http://www.hscic.gov.uk/dlesaac)

<comment><USER5> Right, you "can't". Just like developers on Facebook's Graph API "can't" use social graph data for unapproved purposes. Because they signed an agreement.

Contracts are not the same as controls.

</comment></comment></comment></comment><comment><USER32> Does anyone know if this is England-only, or if it affects Wales/Scotland/Northern Ireland too? How does the opt out work if you've been in multiple areas/GPs/etc.?

<comment><USER33> It's [NHS] England only (currently).

<comment><AUTHOR> I live in Bristol. I wonder how quickly I can move to Cardiff?

<comment><USER13> Well in Cardiff you'll be opted in to organ donation automatically of course.

<comment><USER34> You say this is a bad thing?

<comment><USER13> I think government assuming ownership of [parts of] persons is a bad thing, yes.

IMO it makes logical sense to either not assume ownership of anything a person had in life, including their body and allow their family/executors to look after the remains of their person , effects and assets [the status quo]. Or you should say that ownership is void on dying and all assets, including one's body should pass to the state.

It would make far more difference for the state to assume ownership of the things a person had, houses, bank accounts, than it will make to only assume state ownership of a persons body _post mortem_.

There doesn't appear to be a consistent argument to compel one to take this middle ground.

</comment></comment></comment></comment></comment></comment><comment><USER35> I just threw up in my mouth a little.

Then I started thinking about how to make a dating service using this data-- find all eligible males with your blood type in a given area!

<comment><USER36> Oh don't worry, you can "opt out"

</comment></comment><comment><USER33> Will data be made available outside the NHS?* (PDF) [england.nhs.uk] [http://goo.gl/CxqFVa](http://goo.gl/CxqFVa)

* FAQ 39.

</comment><comment><USER37> You have to email them instead of querying an API?

</comment><comment><USER38> So happy I live in Scotland...

<comment><USER39> Once you're part of Scandinavia all your problems will be gone. [http://m.bbc.co.uk/news/magazine-16050269](http://m.bbc.co.uk/news/magazine-16050269)

</comment></comment><comment><USER40> £300 for a DVD copy? ...Huh?

</comment><comment><USER41> 2 words: thumb drive.</comment>


Pay attention to power law distributions - <AUTHOR> http://jackealtman.com/2013/01/03/pay-attention-to-power-law-distributions/

<comment><USER2> I agree with the point of you are making, but feel the need to question your use of the term 'wonder drug' when describing accutane, also known as isotretinoin. HN readers should know that isotretinoin is a profoundly dangerous chemotherapy drug which was only by coincidence discovered to cure acne. The side effects of isotretinoin can be permanent [1] to the extent that the drug's inventor Hoffman-La Roche stopped selling it in 2009 because it was losing so many lawsuits. Yet the FDA (and all the rest) refuses to withdraw this drug from market. It is a modern pharmaceutical scandal which is still in progress.

If a woman becomes pregnant while taking isotretinoin, her children WILL have birth defects. The drug also causes depression in far more cases than Roche's data suggest. The side effects are as diverse as they are serious – there exist entire forums dedicated to sufferers of this drug's permanent adverse effects.

This drug cures acne like nothing else we know. However, the same poorly understood mechanism that enables it to dry out one's skin also systematically dries one's joints of the hyaluronic acid that is so necessary for their lubrication. Since ceasing my treatment four months ago after researching causes for the cartilage damage in my right kneecap, I have been transformed from a healthy and sport-playing science undergraduate to finding even typing this message uncomfortable because of pains in my fingers. So much that I have taken for granted as a 21 year old has been taken away from me by this drug, and I'm not alone in this. I don't like to overdramatise things, and my situation should improve to some extent given time, but taking this drug is the worst decision I have ever made, and the young, impressionable people who are all-too-readily prescribed this drug must be made more aware of its risks.

Remarkably effective acne cure, yes. Wonderdrug, absolutely not. Calling it one is just disrespectful to those who have killed themselves [2] and whose lives it has damaged [3].

[1] [http://en.wikipedia.org/wiki/Isotretinoin#Permanent_side_eff...](http://en.wikipedia.org/wiki/Isotretinoin#Permanent_side_effects)

[2] <http://rense.com/general32/scu.htm>

[3] [http://jacquelinefox.wordpress.com/2011/04/01/ibs- sufferers-...](http://jacquelinefox.wordpress.com/2011/04/01/ibs-sufferers- agree-to-settlements-with-hoffman-la-roche/)

<comment><USER3> I've taken this drug, the potential side effects are definitely scary. I was lucky enough to only suffer from dry skin and increase sun sensitivity (which is still existing to this day).

I know that when I was taking it (that was 12 years ago), the dermatologist would only prescribe it as a last resort, and, in case of a girl/woman, would always proceed with a pregnancy test first.

I personally am happy I took this drug but I can fully understand it's not your case.

<comment><USER2> I'm genuinely happy for you. I know what it's like. When isotretinoin works — and it usually does — it has an amazingly powerful effect on one's self esteem.

Since you took the drug 12 years ago, Roche has been quite aggressive in its marketing of accutane for treating more mild acne – not necessarily as the last resort that you describe. I think it's reasonably well prescribed here in the UK, but I get the impression that it is handed out somewhat too readily in many nations, including even the USA.

</comment></comment></comment><comment><USER4> I am the dermatologist referred to here. I am sorry this poster had such a bad experience with this medication, but I want to correct some things he/she said. First of all Accutane was not found by coincidence - it was very intentionally synthesized in the mid-late 60s. It has been known for years that high dose Vitamin A clears acne, but at doses so high it was harmful to the liver. So Accutane was developed to maintain the good effect on acne (which is to shrink sebaceous/oil glands) and get rid of the hepatotoxity (ill effect on liver). But like high-dose Vitamin A it is a teratogen, which is a medication that has a bad effect on an embryo. Lots of drugs have this same issue - when you pick them up at the pharmacy they come with a sticker or insert that says "don't take if you are or may become pregnant" but the focus is on Accutane because acne is such a common condition and mostly occurs in teens and early twenties, a sexually active child-bearing population. Many of the other drugs with this same issue are used for less common conditions in much older patients. And yes, it can be sun-sensitizing, like many meds. As for the mood effects: lots of studies looking at this - all designed as retrospective studies comparing group of adolescent acne patients treated with Accutane with acne kids treated with antibiotics (NOT compared to kids who have NO acne, and so have one less thing to be unhappy about). Studies consistently show the Accutane treated groups have LESS negative mood issues than their antibiotic-treated counterparts - basically because their acne is gone for junior high/high school/college. AND this prevents scarring - we can always prevent scarring, and can never really undo it. In fact , there are recent reports of patients suing for scarring that resulted from NOT being prescribed Accutane! I am sorry about all the lawyer trolling of the internet for business and am aware of all the online support groups around purported ill effects of various drugs - nothing I can do about it. It is true that Hoffman LaRoche stopped making name brand Accutane in 2009 - but only because they couldn't compete financially with the four other companies who entered the market with branded generics. And in fact a fifth company has just come out with another competitor this past month.

After the patient and parent watch the video, I go back in the room and spend as much time as they need with whatever questions they may have - and then for girls I excuse the mom/parent to the waiting room for a careful and confidential contraception discussion!

</comment><comment><USER5> Hmmmm... I think the post is a bit fast and loose. I recall reading one study of happiness that concluded small purchases/investments were best because one some gets used to new things. Can't find a link right now, unfortunately.

Power laws have become a kind of popular science meme, but I don't think they are as common or powerful an explanatory tool as some would like I to believe. Or perhaps I've just been reading too much of Cosima Shalizi's blog.

<comment><USER6> "Power laws have become a kind of popular science meme..."

The blog post is essentially restating the Pareto (80/20) principle, first espoused by management consultant Joseph Juran in 1941. In the blog post he calls it a power law -- the Pareto distribution is a a power law distribution -- but it's an observation that has been made before and is known by many.

<comment><AUTHOR> Power law and the pareto principle are definitely very related, but I think one is an instance of the other. I'd argue that power law is one manifestation under the umbrella of pareto principle, as compared to a stepwise function where 80% of actions/people/whatever act one way, and then another class of 20% act dramatically different from the 80% but similar to each other.

VC fund returns aren't driven by 80/20 (although they may fit that description), but by the much smaller number of outliers.

<comment><USER6> You have it backwards. The Pareto principle is based on a Pareto distribution with specific coefficients. The Pareto distribution is a power law distribution. Ergo the Pareto principle is based on a power law distribution. There is nothing stepwise about a Pareto distribution.

Maybe VC returns follow 80/20 and maybe they don't. It isn't exactly a public dataset that we can readily get our hands on. The point I was trying to make in my original comment was that although power law distributions seem to be a popular topic of conversation for the past few years, they aren't exactly a new discovery. Given that wealth and income follow a Pareto distribution, perhaps it isn't that surprising that VC returns do too.

<comment><AUTHOR> You're right, we shouldn't be surprised that VC returns looks like this, and it's well known that they do. The point I'm making in the post is that more mundane things our lives may look like this than we expect.

Pareto power laws can't be stepwise, the point I was trying to make is that 80/20 distribution can be, and they are related to but not necessarily the same as power laws.

</comment></comment></comment></comment></comment><comment><USER7> To: johnconroy [dead] (I can't reply to that comment.)

Hi! I'm not a site administrator, so I don't know why you were hellbanned. I read your comments history list to see if I could found a cause (for example an extremely offensive comment, or spam) but I couldn't find anything annoying. [ _]

I saw that a lot of your comments are oneliners. In this community we generally don't like them and they can get you a downvote. Try to expand them and add more details. This information may be useful for the other members and it's more probable to get a few upvotes.

For example: <http://news.ycombinator.com/item?id=4161475>

Says: "Bookmarked ..."

A better comment could be for example: (It depends a lot on your background.) "I work as a web designer. We have a giant banner with the html symbols at the office, but it's a pain in the back to find the correct one. We were tried to make short versions with the most used symbols but that wasn't useful. I will bookmark this right now."

Or sometimes it is good enough to upvote to the history (without any comment).

[_]There are (probably) other reasons for hellbanning that can't be found reading the comments, for example massive sockpuppets.

</comment><comment><USER8> I was left wondering:

\- What the hell can be so complicated about "explaining an acne medication to a patient"? It can't be hard to use. Is it the risks associated? How to respond to numerous reactions and side effects?

\- Why did it take a dermatologist (which would put her in the ~90th percentile of intelligence) so long to realize she was repeating the same thing over and over and over, and that this could therefore be automated?

\- If a video -- not interactive tutorial -- fully replaces everything her in- person explanation provides, then she must not have been getting questions from patients, which makes me wonder how much of it they were actually following and was informative to them.

<comment><USER9> > Why did it take a dermatologist (which would put her in the ~90th percentile > of intelligence) so long to realize she was repeating the same thing over > and over and over, and that this could therefore be automated?

IANAL & IANAD, but I'd wager small sums of real money that there are tons of legal implications around a doctor explaining how to use a medicine vs handing it off to a machine or even a nurse or other medical practitioner. Think: "my doctor didn't properly explain how this medicine would interact with this other medicine and as a result I was harmed, now I feel that my doctor should be held liable." Were I a doctor, that would probably be the very first thought in my mind immediately following how to automate an otherwise time- consuming process.

There are also considerations about ease of access. To a technologically lay person, the perceived complication and costs around a professionally edited instructional video can seem insurmountable or not cost-effective.

</comment></comment><comment><USER10> A recent article in nytimes discussed research that estimates the cost of some government regulations on businesses. It's done by first noticing that company size follows a power-law distribution and then measuring deviations. I wonder if you were able to get enough data on investment returns if you would find any interesting deviations?

[http://economix.blogs.nytimes.com/2013/01/02/why-49-is-a- mag...](http://economix.blogs.nytimes.com/2013/01/02/why-49-is-a-magic- number/)

</comment><comment><USER11> Both Steven Johnson's "Where Good Ideas Come From" and Steven Pinker's "Better Angels of our Nature" have fantastic discussions on many other occurrences of power law distributions.

<comment><USER12> This book [1] also goes through a quite a few examples of power law distributions. It also goes through a large number of examples of random walks and the associated concept of Lévy flights [2].

[1] Barabási, Albert-László (2010). Bursts : the hidden pattern behind everything we do. New York N.Y.: Dutton.

[2] <http://en.wikipedia.org/wiki/L%C3%A9vy_flight>

</comment></comment><comment><USER13> The hard part is figuring out which 1% of our investments will yield 99% of our returns.

<comment><USER14> I would argue that no one actually figures that out. The obvious implication of the fact that VC returns follow a power law is that VCs have no real expertise in picking winners, but are merely riding along on the underlying structure of the market. They invest in X number of companies and one of them hits so big for reasons no one can consistently explain that it covers the losses on the remainder. The rest is just ritual and hand waving.

<comment><USER15> This is a really interesting question. Humans tend to be pretty good at accepting things that are mostly random, or mostly skill-based, but have a surprisingly hard time with things that involve a combination of skill and luck. For a great example of this, watch amateur poker players.

This is just as true in investing-people tend to espouse either "all skill" (e.g. Warren Buffett") or "all luck" (one common interpretation of the Efficient Market Hypothesis.) Even defining skill in investing is difficult. High-risk investments tend to have a high return, so skill is getting an outsized return compared to the amount of risk involved. But the simple measures of risk (e.g. Beta) are not good _actual_ measure of risk. For example, a company whose stock price suddenly plummeted is often a good investment since the market tends to overreact to bad news. Furthermore, the stock is now lower-risk because it has less to lose. But it will appear extremely high risk by most common measures.

I think it's a decent assumption in VC to treat all investments at stage X as equally risky. E.g. angel investments are one category, series A as another. This is obviously not 100% perfect but is a decent approximation.

Then we can ask the following question: how did Sequoia Capital's series A investments in a certain time period (say 2000-2005) perform compared to all other series A investments? It's important that you look at all of a VC's investments and not just a particular fund, because firms will often bury their bad funds.

I have no idea if that kind of data is available anywhere, though.

<comment><USER14> That's a good start, but even if no firm outperformed the average, there might still be skill involved. The question is, what is the a priori probability of success for a given class of investment? To answer that, you'd have to account for investment opportunities that weren't taken by anybody. But of course there's no way to know whether an unfunded startup would have been successful. So the VC community can rest easy knowing that they are insulated from objective evaluation, unless a VC emerges whose sole criterion for funding is that everyone else turned the investment down.

</comment></comment></comment></comment><comment><USER16> PG has a name for trying to capture the potential of the large-payoff potential of investments: "Farming black swans" [1].

[1] <http://www.paulgraham.com/swan.html></comment>


Scaling Memcached: 500,000+ Operations/Second - <AUTHOR> http://blogs.sun.com/zoran/entry/scaling_memcached_500_000_ops

<comment><USER2> Does anyone know what the comparable figures for an Intel server cluster are? Looking around the net, it seems like the cost of the hardware required for that half a million ops figure is in the neighborhood of $30k. The article had an 8 core T2 with 64gb of ram.

[http://www.google.com/products?client=safari&rls=en-us&#...
Take a look on Newegg, and it would seem that it'd be enough to buy you around 20 Intel boxes with 16 gigs of ram each. The configuration I used put a Core2 Quad on each machine, so our cluster would have 80 cores and 320gb of memory. Unless there's a huge gotcha somewhere regarding the system bus in the Intel architecture vs. the Sun architecture, the Intel cluster still seems to win out. The Sun box indicated that they used DDR2 chips but didn't indicate what speed, so I picked DDR2 800.

<comment><USER3> Splinting memcache over several servers has all of the classic scaling issues. Including the need for 10Gig switches which are still stupid expensive:

Cisco 16-Port 10 Gigabit Ethernet Module Our Price $27,433.91

PS: You could probably get close with a 32 port GB switch with a 10GB unlink, but it's still not free.

</comment></comment><comment><USER4> A single threaded Redis instance on a 300$ Linux box performs 100,000 operations/second. Assuming a quadcore and running N instances to approach 500,000 op/seconds in cheap hardware should be not hard, so I'm not very impressed by this numbers, especially given the hardware used.

<comment><USER5> Read the facebook engineering notes on memcached. Linux kernel needs to be patched to fully utilize all cores even at 200k qps with 173ms average latency. Also you didn't give average request size and latency numbers, which makes your number less helpful.

</comment></comment><comment><USER6> It does advertise Sun's hardware but scaling memcached to heavily multicore setups is interesting, irregardless of hardware used - same points would be valid on highend intel servers. Good to know it's possible to saturate 10G without much tweaking.

</comment><comment><USER5> A typical "drag race" benchmark. There is no latency numbers for the corresponding throughput. The facebook engineering notes was a little more helpful, with at least average latency 173ms at 200k qps with an intel 8-core box with 1GE NIC (they could push it to 300k but the latency was too high.) So a max 500k qps (with unknown latency) on a much more expensive T2 with 10GE NIC is not that impressive.

I'd like to see median and 99 percentile latency for any throughput numbers.

</comment><comment><USER7> also see the older <http://www.facebook.com/note.php?note_id=39391378919>

</comment><comment><USER8> is this an ad? serious question.</comment>


Snapchat's 'disappearing' videos don't actually vanish  - <AUTHOR> http://money.cnn.com/2012/12/28/technology/security/snapchat-security-flaw/index.html?iid=SF_T_Lead

<comment><USER2> This doesn't really seem like a privacy breach on the part of Snapchat since it's your own device that is the "culprit", it is more of a feature of iPhones than a flaw of Snapchat's.

Also, there are many other ways to "cheat" the Snapchat timed delete of videos and pictures, eg screenshot or recording with another device.

<comment><USER3> No this is on the other person's device.

</comment></comment><comment><USER4> You'd think they would flush the cache.</comment>


What do you take out of a story such as this? - <AUTHOR> http://money.cnn.com/magazines/fortune/fortune_archive/2007/04/30/8405482/index.htm <comment><USER3> In many ways, I'd love to be that guy. It'd mean I'd never be unhappy because I'm in place A doing thing X instead of in place B doing thing Y-- I'd be doing the thing I most wanted to do 100% of the time. No complicated issues with people and obligations and such. Simple, and exactly right.

My only problem with it would be meaning-- I'd have trouble doing something that would probably have no trace when I was gone, aside perhaps from whatever philanthropy resulted.

Excellent article, by the way; reminds me of the old Reddit.

However, speaking of Reddit, this is pretty tangential to startups-- beware of community drift!

<comment><USER4> _Excellent article, by the way; reminds me of the old Reddit._

Not terribly surprising, considering how many of the people on the leaderboard are refugees from it.

</comment></comment><comment><AUTHOR> I love what he says about Steve Jobs:

"By the way," says Charan, "the last time I talked about Steve Jobs, remember? I've now verified from two directors of Apple, and they say it's right on the button." Gomber and Bartridge perk up their ears. "I asked, Well, tell me his three God's gifts. And they think about it, and they say first thing, this human being has a talent to figure out what the consumer really wants. This is a very valuable thing! No. 2, he has the will and the talent to find - <USER2> matter where it is! - the right technology that will deliver what they want. Nobody said he invented one! And third, he has the talent to create demand at the right time. I say, Where do you find those human beings? But he is one."

</comment><comment><USER5> Reminds me a little of Paul Erdős, although Erdős maybe wasn't quite so extreme. According to Wikipedia, he voluntarily spent most of his life as a vagabond, showing up at people's houses to collaborate on Math papers. (csci type stuff: Combinatorics, Graph Theory, approximation, etc)

<http://en.wikipedia.org/wiki/Paul_Erd%C5%91s>

</comment><comment><USER6> It's a strange portrait. The way he's described, I picture a character from an Ayn Rand novel.

<comment><USER7> He is extreme in a sense that is really hard to imagine.

He clearly makes a great deal of money, but according to the article, he doesn't use it at all. His life is his work - no romance, no home. Does he ever take a vacation?

He focuses on purpose to explain his lifestyle, and has clearly achieved incredible success at his business. But what about forging intimate personal relationships, both romantic and with friends?

I get such great value from having close relationships, a "home", an "office" that to imagine life without them is almost impossible. Of course he has 30+ year relationships with his clients, so maybe he has developed close friendships with some of them.

If Rand had created him as a character, maybe the purpose would be to show the downside of total focus on only business. Life without romance - never!

</comment></comment><comment><USER8> Couldn't you have given this article a more descriptive title? This was an interesting read, almost skipped it because of the title.

</comment><comment><USER9> You can focus on externals or on internals. He seems to focus completely on internals. He worries about the quality of the work he does at any given moment rather than about where his work will lead him or what he will get for it. It is as though he has given up fear.

</comment><comment><USER10> Truly inspiring. I thank you for posting this link. Wonderful way to start a morning.

</comment><comment><USER11> It is a caricature of failure despite epic success; his life is a dire warning.

<comment><AUTHOR> For you and I, we may never be able to accept such a life. But this guy seems totally content with what he has. That is the more shocking part. You don't really sense much of wishing it wasn't like this from the tone of the story - but then again, I don't know him personally.

</comment></comment><comment><USER12> Fascinating. My basic take is that he has both a unique philosophy of business/life and a unique way of thinking/problem_solving, and he gets his enjoyment out of applying this to other people's problems. It would be interesting to make a meta-cognitive model of his problem solving process.

</comment><comment><USER13> He gets marketing.</comment>


Schedule up for Startup School -- includes a recent founders panel - <AUTHOR> http://startupschool.org/schedule.html <comment><USER2> Wow, awesome. Great lineup.

</comment><comment><USER3> Looks great. I can't wait.</comment>


Pas6502 – Object Pascal dialect compiler to 6502 - <AUTHOR> https://bitbucket.org/paul_nicholls/pas6502/src/master/ <comment><USER2> Also: [https://lemonspawn.com/turbo-rascal-syntax-error-expected- bu...](https://lemonspawn.com/turbo-rascal-syntax-error-expected-but-begin/)

([https://github.com/leuat/TRSE/](https://github.com/leuat/TRSE/))

<comment><USER3> Previous discussion of TRSE: [https://news.ycombinator.com/item?id=21299610](https://news.ycombinator.com/item?id=21299610)

</comment></comment><comment><USER4> Nice - I know languages such as C/C++ also get compiled into Assembly language code and hence are also used for computer hardware programming. Is Pascal used in that space in the industry?

<comment><USER5> I know it's not industry, but as an aside, Oberon (an OO successor to Pascal also designed by Wirth and his team) was used to write BlueBottle OS at ETH Zurich. All of userspace and most of the kernel was written in Oberon.

I've played around with BlueBottle inside Qemu. One interesting feature is that userspace programs are compiled to compressed syntax trees and JIT- compiled by the dynamic linker/loader at runtime. The very slow disk drives used by the ETH Zurich workstations, combined with the relatively little amount of optimization performed by Wirth-style compilers, meant that the savings in disk I/O time was greater than the cost of JITting, so it was a net performance win. One of Wirth's graduate students went on to develop this idea further by developing SafeTSA, a similar idea, but pushing more of the work into the ahead-of-time compiler by switching from compressed syntax trees to compressed static-single-assignment control flow graphs.

<comment><AUTHOR> Small correction, BlueBottle uses Active Oberon, not plain Oberon.</comment></comment></comment>


Data-based Recommendations - <AUTHOR> http://bytepawn.com/2008/08/04/data-based-recommendations/

<comment><USER2> Most sites do recommendations as a big Venn diagram. If my_upvoted_items intersection your_upvoted_items > x then recommend me your_upvoted_items union my_upvoted_items(complement).

I don't think this is a great approach. If you and I have similar tastes, chances are that I have intentionally avoided the things that you like but that I have not voted on.

I'm working on a music recommendation site that, I hope, will avoid this problem.

</comment><comment><USER3> That would be a nice improvement for Hacker News.

I assume that it is possible to track my votes and compare them with other users. If I voted for an article, I should get a recommendation for another article that people with similar votes have found interesting.

This would also give an additional incentive to use the vote button, since it would improve my experience.

<comment><AUTHOR> I agree. Similarly, if IMDb would have good recommendations, I'd cast far more votes just to get better recommendations.

<comment><USER4> Isn't this what the Netflix prize is all about? Based on a history of user rentals and ratings you want to find recommendations.

<comment><AUTHOR> Unfortunately, Netflix is not available where I live, so I can't comment on it.

</comment></comment></comment></comment><comment><USER5> data >> algorithms

Which is why I asked: <http://news.ycombinator.com/item?id=265728>

<comment><AUTHOR> My point in this regard is not so much about data vs. algorithms. It's the fact that even the sites that use the data they have suck.

<comment><USER5> I think that the much, much larger user base of Amazon that is voting with dollars rather than diggs is the big issue.</comment></comment></comment>


The ethics of digital disruption (why top tech companies are the most TOXIC) - <AUTHOR> https://www.squareweave.com.au/blog/2015/10/21/the-ethics-of-digital-disruption/ <comment><USER2> I'm 1000% in support of work-life balance, and of employers treating employees as humans.

But given that the Amazon that I worked at for a year and a half (in the AWS tree) emphasized work-life balance, and among the dozens of workers I knew personally, none had complaints anything like the one in the article (or the many in the previous Amazon-is-evil article) I do wonder about the other examples given.

Bezos sent out an email company-wide after the story about the stillborn child broke and said that is _not_ something that should ever happen at Amazon, and to email him personally if anything like that ever happens again.

I also have reports from an Apple employee who has worked there for years that he's not particularly overworked, and that he has plenty of time to spend with his family.

All of the referenced companies are big. In any big company you'll have pockets of crappy management. It's no more fair to describe the _company_ culture based on the experience of an individual than it is to describe a _human_ culture based on the actions of an individual.

That said, I'm sure there are companies where the culture is cutthroat all the way down. But I'm also sure that trait is not confined to tech start-ups -- and I would hope that the trait isn't highly correlated with company success.

<comment><USER3> It's sad that the amazon work environment kerflufle started from an article that was so flawed and focused on the wrong things. The "work life balance" at amazon isn't as horrid as it could be but it's not good. There's more to a good work environment than simply avoiding horrendous overwork. Amazon has a lot of very serious problems including a toxic company culture that treats people like cogs. I've known a lot of people who have worked at Amazon in many different areas (including myself) and the overarching story is one of people who have not been happy and left quickly. There are some exceptions in specific groups and teams but the overall situation is very problematic and is a big competitive disadvantage for the company (though it's not obvious yet because they have so few capable competitors).

<comment><USER4> "Leaving quickly" seems to be common. The guy I'm sitting next to spent two years at Amazon (he left there a year ago) and said that at 18 months he was one of the longest employed in his group.

I don't know how you get any semblance of a team with that kind of turnover.

<comment><USER2> The way my offer worked, at the end of the first year some stock grants vest come due, and I would end up working for less money per-month for the next year unless I stayed for my next stock grant a year later. I am _not_ generally happy working in a corporate environment, but that's more about me than about Amazon in particular (I prefer being an entrepreneur; that's why I read HN). And I wasn't willing to work for another full year, even though the compensation was amazing. But it wasn't because of poor working conditions; instead it was pretty standard fare for big corporate salary jobs, like not being able to work on my own projects in my free time without restriction.

After one year as a full employee I was deluged with job offers from just about every other major company. I'm sure many of them would have offered to increase my compensation. I said no to all of them because I wanted to go back to consulting. I assume that after your second year, you probably get a similar set of offers.

A typical employee in AWS sticks around for about 2 years. But at the same time, tenure at tech companies is getting short just about everywhere these days -- and I knew people in my group who had been at Amazon for 4-5 years. My own personal tenure at companies rarely exceeds two years, with one exception where I stayed for nearly six. So again, I don't see how Amazon is terribly different on this axis.

One of the most annoying things at Amazon is the "on call" policy where _all_ developers need to put in time "on call", which includes answering a pager in the middle of the night, sometimes for poorly designed alarms that are poorly documented. I knew one developer who quit, with no notice, right after the first week he'd been on call. The frustrating thing about it was that my team did _client_ work: Android, iOS, and Windows clients for an AWS service, so none of us knew the first thing about how to fix anything on the server side, so it was particularly painful to be put on call.

So no, not everything is perfect, and I'm sure that some managers are downright awful. But not everything is "Work harder! Damn your family!" and "We'll be watching your performance after having a personal tragedy strike!" either, which is what the articles _imply._ I did really feel that if I'd seen something like that I could have taken it to HR and it would have been addressed.

<comment><USER4> I've heard that the one thing an Amazon employee can buy without getting manager approval is a pager. Which tells me pretty much everything I need to know about working there.

</comment></comment></comment></comment></comment><comment><USER5> Union, yes! The tech industry needs to unionize. It's the only way to push back against this sort of thing.

There are unions in creative fields. One of the relevant ones for tech is The Animation Guild, Local 839, I.A.T.S.E., which represents artists and technicians in Hollywood animation. Here's their standard contract.[1] It's not spectacular, but it has time and a half after 8 hours in a day or 40 hours in a week, plus double time after 6 days. This discourages crunches. There are also clauses which prevent jerking around employees too much, including a formal grievance procedure with teeth.

As I've pointed out before, unionization is why film scheduling and budgeting is a serious discipline. Film projects which overrun their budget cost the producer, not the employees.

[1] [http://animationguild.org/wp- content/uploads/2014/10/2012-20...](http://animationguild.org/wp- content/uploads/2014/10/2012-2015-Master-Book.pdf)

<comment><USER6> I don't want an hourly wage. I want a salary. Punching time cards is a shit lifestyle that I don't want to go back to.

The way to push back is to quit. Any half competent developer should be able to pick up a new job quickly. There are plenty of companies that don't require as many hours.

<comment><USER5> Anyone in system administration or "DevOps" is effectively hourly.

</comment></comment></comment><comment><USER7> I think there is plenty of reason to be concerned about tech giants and startup darlings being run without a moral compass by sociopathic leadership. But it seems myopic to focus your concern on their employees, who'll endure a tough couple of years and then wash out into $100k+ jobs elsewhere.

What about the consequences on society at large of (a) letting these companies accumulate power and influence and (b) lionizing their leaders' pursuit of results at all cost?

<comment><AUTHOR> I agree. The wider social consequence is ultimately more damaging. But even more difficult to deal with, or to hold them accountable for in a "laissez faire" free market context??

</comment></comment><comment><USER8> The problem is that in these top tech companies you can actually advance because you're good. Yes that means "climbing over others", and of course, it's rather likely that you are not in fact the best.

The alternative is, put kindly, nepotism, put badly Machiavellian political games. You climb up by marrying the spoiled brat daughter of some senior exec and keeping her happy.

I've worked in the middle east, where nearly all companies are government controlled (not "directly" but through family ties) and the whole society falls into the "non-toxic" category according to this article. Nobody will be working 1 minute longer than they strictly have to (except the personal servants - God help those guys). There is absolutely nothing whatsoever you can do to advance (other than the spoiled brat marrying thing). Everyone who doesn't report to you is against you in every social relation you have and people are constantly laying traps for each other, just to gain a leg up in meetings where the higher ups are present. Truly simple projects get denied necessary firewall access just so the security boss could tell your manager that you're over time. You learn to avoid HR and only ever ask them to "make a contract for THIS guy", because if you ask them to find someone only family members will be allowed to interview. Things like that.

Given the choices, I think I'll be taking the "TOXIC" company, thank you. A lot of people tell me I'll change my mind when I turn 50 or so (used to be 40 but that's getting close and (I like to think) I'm still very sure I want to be in the competitive environment.

</comment><comment><USER9> Honestly I could see investment bankers laughing at this article. I definitely feel like technology is WAY more flexible, where you can do the "hardcore 12 hour days" 200k/yr track and also the "10-5 with lunch" lazy man's 60k/yr track.

If you want to spend a bunch of time with your family and work 35 hours a week, guess what man, you can't drive a nice car, own a big house in Los Altos, or send your kids to private school. Pick a lifestyle and accept the drawbacks that come with it.

There's seriously nothing wrong with moving to a second tier city and making 60k/yr. Let the people who love stress and success do their thing in the Bay Area

<comment><USER10> This is the most self congratulatory self-justification I've ever heard. Really? Lazy man's track?

I'm sorry but if you're GIVING away your life for a company that you don't hold SIGNIFICANT stake in for a regular salary (even at 200k/yr) then you're the chump. Guess what, your time isn't endless. Your 20s-30s (hell, even 40s) will pass away. All you've done is provide free labor so some executive can cash in on his/her 7+ figure income.

This is not even considering the romantic and naive assumption that working 12-hour days will give you success. Or that "stress and success" are in any way correlated. Oh well. This is how tech companies lure naive people into such hours.

This is speaking from someone that's had 9-6 with lunch "lazy man's" jobs that paid 200+k/yr.

</comment></comment><comment><USER11> Finally voices are being raised about this huge issue, finally!

We don't live to work, we work to live, companies exist to make life easier, it's not the other way around.

<comment><USER9> They exist to make life easier for their customers, which is usually directly proportional to the amount of effort being exerted by the company...

<comment><USER11> This is simply not correct in the age of automation.

You are basically saying that your apartment does not get vacuumed because Roomba is doing the job.

And the amount of effort being exerted by the company is not the same thing as the amount of effort being exerted by the employees. All these companies have huge profit margins and owners can easily make their employees' lives more decent instead of treating them this way, like sociopaths they are.

</comment></comment></comment><comment><USER12> This is a sore subject for me because I think that there is a weird equivalence drawn between two fundamentally different groups: Companies led by sociopaths of the highest order and Companies with truly massive goals and a short timeline. There is certainly a venn diagram that would show good overlap between the two - but nobody is making the distinction. It's the second group that I think gets a bad reputation because of the first.

Pardon the military metaphor I am going to make, but it's the closest thing that I know of that can compare.

In the U.S. and most first world nations, you have two distinct groups within the military: The line military which makes up 95-99% of the force and special operations which is somewhere between 1-5%.

The line military generally works 7-4, has most weekends off and generally you can have a normal family life. There isn't much of a selection process for the troops and you only need to put in a huge effort for a small portion of the time. They mostly do low intensity training and in a really big conflict a subsection might see combat twice or three times.

Special Operations has a long selection and initial training process that can last years. This group is at the top of their game 90% of the time and is constantly in high intensity training. They see combat constantly regardless of if there is a big war going and a good portion have significant trouble maintaining a stable home or family life without significant effort from the spouse. If they start to fail, they get removed from the unit and sent to the line (fired). You've never seen "eat your own" like there is in Tier-1 Special Operations units - because you have to, to stay that good.

In my opinion, high tech, "disruptive" high growth companies by definition are the Special Operations groups of the business world. It's a tiny percent, but you have to be at the top of your game 90-99% of the time. That means dedicating your life to the job. That means hiring "killers" who are intensely driven.

The rest of the business world is the line and cannot, or do not want to be in that kind of "always on" 24/7 environment.

I think the disconnect is that there is a large group of people who aren't special operators, they are line soldiers, but they joined special operations companies. This is partly the fault of the company for not making it clear to them before hiring what they would be getting into, and partly their fault for not seeing the company for what it was.

This metaphor seems to apply to companies like SpaceX, Amazon, etc...

You can have a well run company with empathetic staff that crushes spirits constantly because they have such a high bar and a subset of their people were just not up to the task. "Your best isn't good enough" is a constant mantra during Special Operations selection and I don't see why such a bar would be any lower in an elite business organization.

This quote was great: "Amazon is where overachievers go to feel bad about themselves."

This same quote has been applied to Harvard, Stanford, the US Military Academies, Goldman Sachs, EXXON etc...so it's not unique to tech by any means.

Any relation to Objectivism may only apply to that first group I mentioned before, but is certainly not a prerequisite for a company which has insanely high standards.

<comment><USER13> I don't think being a "special ops" company is actually necessary, at all. For a high growth startup, you do not need everyone to work 12 hour days and fight each other to move up. It causes burnout, unethical decisions, and corporate resentment.

Hire smart people, treat them like human beings. If you're constantly working 12 hour days in a tech company, either you or your manager are doing something wrong.

<comment><USER12> _If you 're constantly working 12 hour days in a tech company, either you or your manager are doing something wrong._

Disagree. We have 4 people. We have the money for 4 people. We have the workload of probably 8 people. If I don't put those hours in, things don't get done that need to for us to stay alive.

I'm wasting my time right now responding to this because I think it's an important discussion. That just means I'll get 4 instead of 5 hours of sleep tonight.

I have two 15-20 slide decks to finish tonight: 1 for our board meeting tomorrow and another due tomorrow morning for an Angel pitch on Wednesday. I also just hung up from a technical call, cause we are trying to push a new release to the app store tonight. I still have to review some cards in leankit and respond to a handful of emails from today.

<comment><USER14> No one is talking about limiting high ranking employers/founders to 8 hours. They should work as much as they want because their efforts directly lead to financial gain for themselves (due to large stock holdings, etc).

The problem comes when the bottom of the barrel people are working those same hours. Working 80 hour weeks doesn't really help them unless they really like doing it. Some people legitimately like work. That's great. They shouldn't set the bar, though.

<comment><USER12> This is exactly the point though, don't hire "bottom of the barrel people" into such an organization.

They should be working someplace where they can comfortably be marginally talented.

<comment><USER15> > This is exactly the point though, don't hire "bottom of the barrel people" > into such an organization.

The corollary of this, in a startup, would be don't expect great employees to work 10+ hours a day for 0.5%.

If you expect people to put in that same level of effort, compensate them to the same degree.

Meanwhile those of us who've already been through that meat grinder will be enjoying our 9-5 and large salary, regardless your baseless slight about being "marginally talented."

<comment><USER12> _The corollary of this, in a startup, would be don 't expect great employees to work 10+ hours a day for 0.5%._

Can't agree with this enough - it's the same reason Special Operators get a shitload of incentive/hazard duty pay and signing bonuses.

 _Meanwhile those of us who 've already been through that meat grinder will be enjoying our 9-5 and large salary, regardless your baseless slight about being "marginally talented."_

I was simply responding to the OP's "Bottom of the barrel" comment.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> The article makes some interesting points, generalizes quite badly about things outside its ken, and overall is not really saying anything all that interesting. It's sort of like a fact-free fluff piece in favor of being deliberately less competitive. I guess it's a fine message, but in a very direct way it feels like a nonstarter. Good to have ideals though.

</comment><comment><USER17> I'm all for a push for ethical treatment at top companies.

If I were able, I'd like to work at one of them. But, since I'm not at the top of my game, I won't end up working at one of them. Instead, I'll continue working at my B.S. company. Here's what happens there: many of the leaders completely suck. They make decisions like the smart, driven, type-A leaders at <top tech company here>. The difference is that those decisions are often idiotic and/or ill informed. Their IT will buy software and hardware that is just wrong. Their architecture will be completely scattered and unthought. They will hire college grads, because no one with experience will work for them. Some days, I'd almost rather cry coming out of a meeting if I were challenged, at least. At least my career wouldn't be in the toilet. The grass is always greener on the other side, I guess.

<comment><USER18> This is why when you go home you work on your tiny SAAS aimed at a specific group of people who will shell out some money each month for it.

<comment><USER17> If someone is working for one of the top tech companies, they should appreciate that more. If you're doing the best you can and they are still challenging you, that might not be a bad place to be. If it almost doesn't matter what you do, the IT infrastructure and code is a mess, and when you try to change that, you can't do it significantly, then that is much worse for your career. However, it's where I personally need to be right now- safely making income for my family.

As for writing a SAAS app and making money- that's not some kind of magic recipe. I'm much older now with a family. It has been my dream of "making it big" for years and partially chasing dreams that has been holding me back from getting things done that I should be doing. It's fine to offer that as a possible solution, but too many believe it is the option for them and leave that as a lingering dream that only serves as a time trap. I've had dreams before that could have been made reality if I only had the people and resources to make it happen, but that opening didn't happen and I got distracted quickly. Sure, I didn't _make_ it happen, but I have limits. Sometimes the resources just aren't there, personally or externally. If you talk to a number of people that make it big, I think some chance opportunity will have come into play in their history 100% of the time. Sure, they gave it their all, or had a lot to give. But when trying to get started, sometimes the spark doesn't ignite a fire. I'm a rusty piece of steel and worn flint- the actual process of writing a SAAS app doesn't excite me anymore. I'm still sitting on top of a domain waiting to get inspired, but getting into iOS and Android development is really not as easy as it would seem I've found, and even just writing a web-based app to do what I want is a struggle since I suck at JS.

<comment><USER18> Perhaps dreaming smaller should be considered. In the end $1000 in recurring income can still make a difference, and is better than almost-billion-dollar- idea.</comment></comment></comment></comment>


New SPARQL engine powered by Hadoop - <AUTHOR> http://sparqlcity.com/product

<comment><USER2> I love the idea of SPARQL and making a machine-readable web, but, imho, it's just too complicated to ever become mainstream. The technologies that get widely adopted are usually embarrassingly simple to get started with: HTML, SQL, JSON, ...</comment>