Ferroelectric, pyroelectric, and piezoelectric properties of a photovoltaic oxide - <AUTHOR> http://aip.scitation.org/doi/full/10.1063/1.4974735 <comment><USER2> Related:

[https://news.ycombinator.com/item?id=13618392](https://news.ycombinator.com/item?id=13618392)</comment>


Why The Shortage Of UI Engineers Is Going To Get Worse - <AUTHOR> http://www.usabilitycounts.com/2010/10/20/six-reasons-why-the-shortage-of-ui-engineers-is-going-to-get-worse/ <comment><USER2> While I agree with the central theme ("UI/UX Designers and developers are going to get harder to find") I don't agree with a lot of his reasons.

He's obviously not an engineer and doesn't really understand AJAX, jQuery or HTML5.

Personally, I believe the shortage is going to get worse because there is going to be an increase in demand and there are VERY few paths through traditional educational outlets that will produce a good UI/UX designer/engineer. HCI at Stanford is the closest I've seen recently while interviewing candidates, and even then it has less to do with web technologies than interaction principles.

<comment><USER3> Just wondering, what really makes a good UX engineer? I'm studying HCI at a university other than stanford, and the course is total BS... I knew the principles we were supposed to learn were important, but having whole lectures spent on comparing command languages vs natural languages (i.e stuff like command languages are fast and good for experts and natural languages are good for novices... how to make prototypes out of postit notes... a low-fidelity prototype is a cheap prototype whilst a high fidelity prototype is expensive... Stuff I could tell you myself) makes it hard to spend my time focusing on the course. :(

The only useful thing I've learnt is that when you're doing a questionnaire with lickert scales, don't use mean because the distances between each point on the scale isn't equal...

<comment><USER4> It kind of depends on the scale you want to operate at. If you are Google level, UX tends to be VERY mathematically (statistically) driven. In which case I would focus on a degree in math.

Towards the lower end (read: almost everywhere else) of that scale, you're best bet is actually to look at a design degree, and learn the tech on your own.

Your best bet is experience in the field. Very few schools are teaching comp sci programs that are worth anything (Stanford, CMU a few others), and I have seen almost nothing of value in academia in terms of product/UX development (which is why I say focus on Math or pure Design). The web stuff you're really going to have to teach yourself, I think for at least another 3-4 years, maybe longer.

The author is somewhat right about the commoditization of the backend, but it's not to India, it's to frameworks like RoR. Larger companies that require serious, responsive scaling, sharding and cacheing are not, and will not be, outsourcing to these frameworks. But it's definitely not to India (data entry, sure).

<comment><USER5> Google is a combination of qualitative user research for the initial design backed by quantitative analysis upon release.

</comment></comment></comment></comment><comment><USER6> _"Who needs a backend engineer when you can do most of the work using REST and AJAX calls?"_

Yep, it's just that easy. Everyone knows that AJAX calls are outsourced to Chinese and Indian people on the back end. One per request.

<comment><USER7> Ignoring the gaping holes in the engineering side of the argument (because clearly there is you use the AJAX "language" to make calls to a magic blackbox that handles all of the data storage, notifications, business logic, etc), the UI/UX arguments aren't even fully baked.

For mobile applications he recommends developing a mobile website instead of a native app. The biggest downfall of mobile websites is that iphone and android have rather different vocabularies for user interactions (hardware buttons vs. all touch screen) and what makes sense on one platform often is confusing on the other. That's probably the biggest downfall I've seen to using a framework like PhoneGap. Yes, you can quickly mock things up to try (which is great for iterating on the UI/UX aspects) but there either end up being two versions (sharing some common code for logic/backend interface) with different UIs built on PhoneGap or you go ahead and build native apps to fully fit into the expected user experiences from both platforms.

Is it silly to ignore UI/UX, of course, but expecting the backend to just magically work (and work well) is just as unwise.

</comment></comment><comment><USER8> What hand-wavy nonsense. Yeah, programmers are just commodity and we will just offshore everything. This guy just states this flippantly without even acknowledging the several problems associated with that in Yet Another Case Of UX Worship. The idea that you don't need "backend guys" because you are just going to make "AJAX Calls" makes this guy just look silly.

I mean, yeah, Google wasted money on all those engineers. Why did they build on that infrastructure? All they needed was a few AJAX calls, duh! Finally, Google figured it out and through away all that massive indexing set of algorithms so they could just make AJAX calls.

I don't see what this guy's fascination is with AJAX and "making calls from Javascript" or how this means we don't need programmers anymore. Apparently, we can just appeal to some vague "throw it in the cloud and it will scale" mentality.

What makes it worse for me is that apparently this Usability guru was involved in this website:

<http://www.pickanexcuse.com/>

Really ... a usability guru?

<comment><USER9> I agree. Usability is important, and people who are good at it are valuable, but to turn that into "all this backend stuff is going to be outsourced" is ridiculous. Generally, building a great app requires highly-skilled people working in all areas of the application.

</comment></comment><comment><USER10> I like the general idea of putting more focus on good UX Engineers. But, this guy seems to think that making the back-end support a bunch of Ajax calls is easier than.

In fact, you start needing more work to make it efficient since you have to break down everything into exactly what needs to be in that call with appropriate permissions etc.

</comment><comment><USER11> Google is mostly about backend data processing. Google's front-end effort is nice, but dwarfs in comparison with the backend accomplishments. The same is true for most other businesses. So the demand for backend engineers would definitely be higher. And it's easier to outsource UI than outsource backend, because backend is typically more complex and harder to test and communicate about, so keeping developers closer to business users is even more important in for backend functionality, than for front-end functionality.

</comment><comment><USER12> (I'm the author of the article...)

As one comedian said, "I don't come down to McDonalds and make fun of you while you're working..."

You might not agree with all my reasons or some of my thoughts. However, the indifference of great UI by engineers kills companies. I have sat in environments time and time again where lack of attention of detail creates horrible user experiences, and this affect the bottom line in a very real way.

This should read as, "you are bitting the hand that feeds you."

The best engineers I have met ask questions, and have a genuine concern for the user. They want to create great experiences. The engineers that don't want to get involved in a positive way without being a destructive gatekeeper, I have no time for them.

We have now reached a tipping point where most companies are recognizing the need for great user experiences, because poor one's cost companies time and money, both in lost customers and lost productivity.

This indifference by engineers is the reason why a UI Engineer position was created by many companies; they realize that most back end engineers just want to code in a way that most people at car manufacturing plants want to put the bolt on.

That mentality didn't work out well for GM and other car companies. It's also not working out for most technology companies.

You can bitch all you want, but if you aren't contributing to the bottom line, you're just a cost center. And cost centers get outsourced, plain and simple.

<comment><USER8> You're missing the point and kind of making an ass out of yourself while doing it. You can't just say "You might not agree with all my reasons" and make some strawman. Most of the criticism has been around how you flippantly dismiss the skill of programming while serve up UX design on a pedestal but yet you make it out as if we are dismissing usability design. It's like a delusion of persecution. Also, I fail to see how a "UX designer" is any less a cost center than a programmer. Maybe you're not sure what your words mean?

Also, I have managed those outsourced situations and you are naive: "plain and simple." I mean seriously, either respond to the criticisms like an adult or don't play at all.

</comment></comment><comment><USER13> I am an experienced UI developer and while I agree with the main point of the article I would like to note that not all front-end positions are created equal.

I've worked at amazing shops where there was a clear role definition (made easier by the beauty that is Django) and the expected code quality was really high. There were project based code reviews to ensure this.

I've worked at places where a front-end developer meant you write Java in Eclipse (which took 15 mins to launch) that in turn generated JS that manipulated the DOM to create your markup. This is common practice in more places than we realize.

If you're a UI dev do your homework before accepting any position. The market is really dry for our field and people on the interview will lie to get you to come over. I've been to over a dozen interviews in just the past year (and I am thinking I should write more about this) and the industry expectations are simply rarely properly determined of what the role entails.

Apologies for the rant. Downvote if you sense my frustration :)

<comment><USER14> I'd love to hear more about interviews. As a front end dev with a BFA in design I've had surprisingly easy time finding jobs. I'm always interested in hearing about what other experiences people are having.

</comment></comment><comment><USER15> people who pride themselves on being "UX experts" are the most obnoxious people in our industry.

there, I said it.

</comment><comment><USER16> As an experienced Frontend Developer thats worked for a number of startups, this article is not well written. I prefer Developer rather than Engineer. It does seem like really good frontend developers are becoming more rare because it takes a unique blend of technical skill (mainly javascript), understanding Usability/UX and having aesthetic tendencies. More and more companies are looking for Frontend developers that can also write Framework template code (Rails, Django) which requires even more skill or exp. Its kinda a specialized niche. FYI, I am currently looking for a new job if anyone has an opening.....

</comment><comment><USER17> I agree with the sentiment that a UX/UI engineering need is going to be a necessary requirement not easily fulfilled in the near future. The good ones are fluent in back-end keyboard mashing, front-end spit shining, and everything in between. They need to have design skills, programming chops, user experience knowledge, a taste for the tasteful, a writers intellect, and a certain je ne sais quoi insight into what makes people want to use your product. They are not designers, nor are they programmers... They are both.

</comment><comment><USER18> I liked the article, but little bits don't add up. I'd say it's an entertaining read, but maybe not as solid on information as I'd expect for a frontpage HN article.

<comment><USER19> I found it so badly written it hurts.

"There has to be solid knowledge of how it should be used properly (as opposed to way most UI Engineers use it today), and how to create great user experiences that use jQuery right and not just to use it." Sigh. That's not just really badly written, it's almost almost 100% fluff.

</comment></comment><comment><USER20> usabilitycounts.com, your site is down. Seriously usability counts.

</comment><comment><USER12> I love creating a shit storm. :)

</comment><comment><USER21> lolwut, they aren't engineers. Maybe "engineers"

<comment><USER22> I don't know why you're being downvoted, but an engineer is a legal designation. Would you call Dr. Dre a doctor?

<comment><USER23> It seems to be a legal designation only in Israel: <http://en.wikipedia.org/wiki/Software_engineering#Profession> . Feel free to correct me if I'm wrong.

<comment><USER22> That's sort of the point. Engineer is a term thrown around, from QA to UI designers. While their positions are no less important to a company, calling them engineers is just wrong.

Also, you can get a true 'engineering' degree in software in Ontario, Canada (UWaterloo)</comment></comment></comment></comment>


Overkill - <AUTHOR> http://www.newyorker.com/magazine/2015/05/11/overkill-atul-gawande/

<comment><USER2> One thing this article touches, but does not give enough attention to (maybe because it is so counter-intuitive and hard to explain) is the false positive paradox.

To put in perspective, you should only do a test if you have good a priori chance of having a positive. Even a test with 1% false-positive rate you give you back much more false positives than real positives if the a priori chance is 0.1%.

This is counter intuitive, I'm not good at explaining stuff, so I'll give Doctorow the word, through this excerpt in Wikipedia: [http://en.wikipedia.org/wiki/False_positive_paradox](http://en.wikipedia.org/wiki/False_positive_paradox)

Probabilistics and statistics are hard, very important an counter-intuitive. Med and pre-med, and even high schools really should teach much more math than they do right now.

<comment><USER3> Counterpoint here, the false positive paradox won't surprise many physicians. It stems from concepts that are well-known among my colleagues and have been tested on each of my licensure exams. We apply them at least 10s of times each day.

You can read more about those concepts here- (Note: "a priori" === "pretest probability" in MD speak) [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2636062/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2636062/) [https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnosti...](https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnostic_testing)

<comment><USER4> I will add my anecdote to support this comment. Being an EE with probability- heavy background, I tried to give a "you guys don't use prior probabilities" sermon to a physician and his physician-in-training daughter. I was made to eat my words by their reply, which was essentially that it is something they are all taught and use all the time in assessing risks, interpreting results, etc.

The one time I had to pay $500 for some tests not covered by insurance, but egregiously wrong in the context of my ailment, the referral was from a nurse practitioner.

Two things: 1\. Perhaps doctors don't uniformly apply their training in interpreting test results and analyzing risks? (It is after all, quite common in all professions to forget some parts of one's training)

2\. In addition to doctors, there are also technicians and nurse practitioners, who some times may not have the rigorous training that one associates with med school.

</comment></comment></comment><comment><USER5> We tend to remember the one wasteful case and forget the next 25 which were all reasonable - the old man with the worsening shortness of breath, the middle aged guy with gross hematuria, the drunk who crashed into a parked car riding his moped, the kid with the early onset puberty for a bone age study.

Dr. Gawande is a smart person, a contemporary of Ezra Klein and Ezekiel Emanuel, and a lifelong politically involved leftist. His essays on waste in medicine over the past 10 years have shaped progressive thinking on the subject, with citations from everyone from President Obama (during the sale of Obamacare) and many others. His political alignment is why Dr. Gawande cannot speak the obvious heresy that the United States' insane malpractice climate has anything to do with overtesting (although he has considered that in a prior book). He has made his career promoting the idea of waste and accordingly sees it behind every bush. I don't mention this to say that there is no merit to some of his observations; just to provide context of his background, alignment, and world view.

</comment><comment><USER6> duplicate of: [https://news.ycombinator.com/item?id=9485389](https://news.ycombinator.com/item?id=9485389)

</comment><comment><USER7> When a third-party is paying all procedures are recommended.

<comment><USER8> That's demonstrably false. See, for example, the English NHS where there is clear guidance about what tests are appropriate.

<comment><USER9> NHS is firs party when NHS is making recommendations.

<comment><USER8> NHS is not making the recommendations, NICE make the recommendations.</comment></comment></comment></comment>


2017 Barkley Marathon features one finisher, heartbreak for Gary Robbins - <AUTHOR> http://runningmagazine.ca/2017-barkley-marathons-recap/ <comment><USER2> The 6 second thing is a little bit of a mistake that sensationalizes the story. He actually ran off course for 2 miles and missed some of the checkpoints - full text from the race director below [1].

Obviously this race is pretty hardcore and the people that participate are freaking awesome! The race should be sensational based on that alone, but alas journalists need to add a little more spice...

        A message from the RD of The Barkley Marathons #bm100:

        i wish i had never said 6 seconds...

        gary had just come in after having run off course
        and missing the last 2 miles of the barkley.
        that is, of course, not a finish.

        i do, however, always record when runners come in,
        whether they are finishing a loop, or not.
        so, i had looked at the watch,
        even tho there was no possibility that he would be counted as a finisher.

        so, when someone asked if he had gotten in before the limit;
        i foolishly answered.

        i never expected the story to somehow become that he had missed the time limit by 6 seconds.
        he failed to complete the course by 2 miles.
        the time, in that situation, is meaningless.

        i hate it, because this tale perpetuates the myth that the barkley does not have a course.
        the barkley is a footrace.
        it is not an orienteering contest,
        nor a scavenger hunt.
        the books are nothing more than unmanned checkpoints.

        the boston marathon has checkpoints.
        and you have to show up at all of them or you can be disqualified...

        that does not mean you are allowed to follow any route you choose between checkpoints.

        now, the class with which gary handled this terrible disappoinment
        at the end of a truly magnificent performance...
        that was exceptional,
        and is, in and of itself, a remarkable achievement.

        but he did not miss the time limit by 6 seconds.
        he failed to complete the barkley by 2 miles.

        laz

[1]:[https://www.facebook.com/trasie.phan/posts/10212547374898805](https://www.facebook.com/trasie.phan/posts/10212547374898805)

<comment><USER3> Read this from Gary:

[http://garyrobbinsrun.com/blog/2017/4/close-but-no- cigarette](http://garyrobbinsrun.com/blog/2017/4/close-but-no-cigarette)

Amazing.

<comment><USER4> > The river was maybe fifteen feet wide and absolutely raging from all the > rain we were experiencing. I took one step off the river bank and was > already chest deep. I would never have made the decision to attempt to swim > such waters under anything other than a highly sleep deprived and stressed > state of mind.

It really seems like just a matter of time until someone dies on the course.

<comment><USER3> That's true of many trail ultras with single track. A wrong step in the wrong place you're going down a cliff.

</comment></comment></comment></comment><comment><USER5> I first knew about the Barkley Marathons when I stumbled across the documentary on a streaming platform. I recommend to watch it to anyone who wants a sneak peak on what this absolutely insane/crazy race is about. [http://barkleymovie.com](http://barkleymovie.com)

<comment><USER6> I saw it on Netflix and loved it. I also highly recommend it for anyone that wants to watch a unique documentary.

</comment></comment><comment><USER3> I ran the Umstead 100 this weekend. It was my second hundred (my first was techically a 24h race where I got to 105 and called it quits). Umstead took me 25 hours and kicked my ass. And as far as hundreds go, Umstead is an easy one.

I can't even imagine the sufferfest that is the Barkley.

I don't know why we push ourselves this way. Maybe I'll figure it out one of these races.

Respect to Gary. If you didn't watch the video, he thanks Laz (the R.D.) and gives him a hug. What a fine and proud showing for the ultra community.

<comment><USER7> Congrats, that's damn hard effort. I've run 50K and I could see maybe pushing it out to 50 miles. But going on to 100 miles seems just so so so much farther. Don't know how you do it!

<comment><USER8> Running 100 miles doesn't begin to describe the Barkley.

It's more like running 100 miles...on a section of the Appalachian Trail...that hasn't been maintained in 20 years.

<comment><USER9> Running, power hiking, and navigating 100 miles :)

</comment></comment></comment></comment><comment><USER10> [https://en.wikipedia.org/wiki/Barkley_Marathons](https://en.wikipedia.org/wiki/Barkley_Marathons)

</comment><comment><USER9> This race is a beast.

I've done a bunch of a ultras and know quite a few better runners than myself and seeing them tap out on this race just tells you how gnarly it really is.

</comment><comment><USER11> A strong contender would have to be the Marathon des Sables, around 250 kilometers (156 miles) worth of racing through the Sahara desert in southern Morocco.

Contestants get access to rationed water and a tent. All food has to be carried by the runners. Along the way, they get to deal with dehydration, blistering heat, sand storms, and the very real risk of getting lost. One stage is typically about two marathons in length, and has a time limit.

It's one of very few events that require insurance for corpse repatriation.

Up to 2008, two deaths occurred, one of whom died of a heart attack in his sleep after completing the long stage in 45th place (out of about 750). To gain entry, you have to have a medical certificate anda resting ECG report from at most 30 days before the start of the race, so that's quite extraordinary.

</comment><comment><USER12> Interestingly, the 100mi road record is 11:46:37. I doubt that was done with ~10mi of vertical climb though...

<comment><USER13> But this isn't 100 miles of a course (in fact it's likely more). It's also not a course. You need to figure out how to get to the checkpoints over obstacles, through brush, night and day. You also need to do it alternating clockwise/counter clockwise, except the last lap... it's a far cry from a flat, supported 100 miler.

<comment><USER12> Yea, it's more of an orienteering ultramarathon

</comment></comment></comment><comment><USER14> With a title like this, I expected a true tragedy had occurred. It is brutal for Gary but such is the nature of the Barkley Marathons.

Knowing that humans push themselves to these limits for the sheer fact of having done it is something that gives me a lot of motivation. I have been following the Barkley Marathons for a while and I dream that someday I may subject myself.

</comment><comment><USER15> For those unfamiliar with this race, the Outside magazine article on it is a great read: [https://www.outsideonline.com/1924491/60-hours-hell-story- ba...](https://www.outsideonline.com/1924491/60-hours-hell-story-barkley- marathons)

</comment><comment><USER16> This is mind blowing. I do a bit of running, some half marathons etc. but nowhere close to Ultras and this just amazes me and also excites me. I would want to push myself to do this at least once...see how my body responds.

<comment><USER17> Barkley is beyond an ultramarathon. It is utter sadism. Only 15 people have ever finished it. Most of the Barkley is actually climbing and walking with a little bit of running thrown in here and there.

</comment></comment><comment><USER18> Thanks for posting this - I'm excited to see the rest of the information as it comes in!

</comment><comment><USER19> I don't get how the first person to complete 4 loops had the "choice" to decide which way to run the last one. Was he choosing for everyone? This doesn't make sense, because my understanding is runners can go ANY direction they want. If they have to reverse for the 5th loop, they have to reverse relative to their OWN race direction.

<comment><USER20> For the first two loops, everyone runs clockwise. For the next two, counter clockwise. For the last lap, the leader picks a direction and anyone left alternates, so odds follow the leader, evens go the opposite way.

In this case, John picked CW, so Gary had to go CCW. Although per twitter, they had discussed it during the 4th lap and decided it mutually.

<comment><USER19> Thank you. I must have mis-remembered or mis-understood the documentary. Or perhaps the rules changed.

</comment></comment></comment><comment><USER21> Who gives a damn about their official cutoff?

Data point/fact: Garry Robbins finished in 60:00:06.

Coming to the finish from the wrong direction is a problem; maybe they could figure it out somehow and translate that to a penalty of some number of seconds.

<comment><USER20> So, first off, I don't think anyone, least of all in the running community wants to take anything away from Gary's accomplishments. He ran an amazing race, no question about it.

That being said, this race has its allure and mythical status specifically because it has a variety of rules/challenges that must be precisely followed. Not meeting any of them means you are not a finisher of the Barkley Marathon. If we start bending the rules because someone was close or from the emotions of things we start to devalue the accomplishment itself.

Ok, so if we say 6 seconds, that's really close enough. What about 6 minutes? What about 60?

What if he ran the right distance but took a wrong turn? Should we give him time credit for the extra couple miles?

What if he had the wrong page from a book? He still made the checkpoint... What if he had 3 wrong pages?

Eventually, being a finisher becomes a very confused and confusing accomplishment to even understand. I think it's far better to be very clear as to the rules, and then follow them, even if heartbreak ensues.

<comment><USER21> _> Not meeting any of them means you are not a finisher of the Barkley Marathon._

Article is scant about anything other than the >60hr time. It doesn't say he would not have qualified had he come in under 60. (So the fellow who finished under 60 hours met all those other requirements to the letter? Not a single wrong turn, etc?)

> _Ok, so if we say 6 seconds, that 's really close enough._

Close enough to what? Nobody is saying change 60:06 to 60:00.

> _What about 6 minutes? What about 60?_

It's very simple. Any time should count as a time, if anyone official is there at all to witness and record it. At the cutoff time they prepare to go home, and then leave. Nobody owes you a recorded time after the cutoff.

When the last timer packs up leaves, any subsequent finishers can simply be left not recorded. The status is something like "they are not officially noted as having finished". That's a big difference from saying "did not finish". If one person claims "did not finish" and another "finished", one of them is lying or mistaken. But there is no contradiction between "finished" and "not recorded".

They were there; that's why they (and thus all of us) know he was 6 seconds over. It's effectively recorded.

If Gary is not going to be on record as a finisher, then shut the fuck up about the six seconds. Just say "Not recorded! I saw someone that might or might not have been Gary at some time after the 60th hour; I had reset my stopwatch by then in any case ..."

The six second precision with which we know that Gary supposedly _didn 't_ finish is the laughable contradiction here.

<comment><USER22> There's traditions around this race, and it's okay if you don't agree with them but not being part of the community that embodies these traditions means you cannot come in and redefine them.

Also, no offence, but you clearly don't know enough about the race to really be talking about how it should be operated.

This race is full of obscure rules and insanely harsh conditions, it's just how it goes.

<comment><USER23> Ahh but the parent poster is part of the community that doesn't believe that the Barkley community is fair, so you cannot come in and define _that_ community because you clearly don't know enough about it to really be talking about how it should be operated.

You can defend the race without telling someone else that he has no right to his own opinion.

<comment><USER24> That's not what's being said. What's being said is that 'kazinator's opinion does not _count_ here, which is a different and much more reasonable assertion. Telling other people how to be is never valuable and always rude.</comment></comment></comment></comment></comment></comment>


Scaling the peaks of Hacker News - <AUTHOR> https://www.divio.com/en/blog/scaling-the-peaks-of-hacker-news/ <comment><USER2> Ironically, their demo link is now down [0]

[0] [https://demo.django-cms.org/](https://demo.django-cms.org/)

<comment><AUTHOR> Can you try again? It is working, but we've had a temporary glitch on DNSimple.

If it doesn't work right away, try [http://demo.django- cms.org](http://demo.django-cms.org).

<comment><USER2> Thanks for the reply. https link is still down for me. I can confirm http alternative is working.

<comment><AUTHOR> We're waiting for a change to propagate.

Worse, it turns out that the glitch was... my mistake.

Thanks for pointing it out!

<comment><USER2> No problem and glad to help. Happy Friday!

<comment><AUTHOR> ... and now it should be working. Have fun with the demo. Happy Friday to you too!</comment></comment></comment></comment></comment></comment>


Head-On Crash Test of 2009 and 1959 Cars (video) - <AUTHOR> http://video.consumerreports.org/services/player/bcpid1078549944?bctid=41311737001 <comment><USER3> Maybe this will finally dispel the myth about older cars being safer in these sorts of collisions because they were built like "tanks".

<comment><USER4> Yep, for more info:

[http://www.autoblog.com/2009/09/26/pics-aplenty-iihs- reveals...](http://www.autoblog.com/2009/09/26/pics-aplenty-iihs-reveals- before-and-after-of-malibu-bel-air-cr/)

The after pictures clearly show why the 1959 car kills you, while the 2009 let's you live.

PS: _"The dummy's head struck the steering wheel rim and hub and then the roof and unpadded metal instrument panel to the left of the steering wheel.... The windshield was completely dislodged from the car and the driver door opened during the crash, both presenting a risk of ejection. In addition, the front bench seat was torn away from the floor on the driver side."

Ouch.... an instant death according to the IIHS. And the '09 Malibu? "A high acceleration was recorded on the left foot, indicating that foot injuries would be possible._

[http://www.autoblog.com/2009/09/17/video-iihs- celebrates-50t...](http://www.autoblog.com/2009/09/17/video-iihs- celebrates-50th-anniversary-by-crashing-modern-malib/)

<comment><USER5> Youtube: "This video is no longer available due to a copyright claim by Insurance Institute For Highway Safety." :(

<comment><USER6> Umm someone needs to explain that you're supposed to let the viral video spread where it will, not send take down notices when it does start to spread.

</comment></comment></comment></comment><comment><USER7> It would be interesting to see a video of two 1959 cars crashing compared to a video of two 2009 cars crashing. It's nice to have a tank, but if everyone has tanks, maybe I'm not better off.

<comment><USER8> You're still better off in a tank if you hit a telephone pole or a tree, for instance.

<comment><USER9> Do you mean a tank, literally, as in an M1 Abrams? ...otherwise, did you see the video? Crumple zones, passenger cabin intact, airbags, seat belt, etc?

<comment><USER8> Huh? I took conanite to mean newer cars are "tanks," only in the sense they survive crashes better than old cars. He was saying that it might not be a relative advantage if the other car hitting you is equally impervious. Irresistible force, etc. etc.

My only point was that you are much better off in a modern car striking a telephone pole than you would be in a 1959 car, even if it is true that you are somehow worse off when two modern cars strike each other (although I doubt that also). Nothing about literal tanks intended whatsoever.

Your comment seems to take the exact opposite meaning from what I intended to say, so I must have expressed myself poorly.

<comment><USER9> I see what you mean now, but there are several threads in this discussion that use "tank" to refer to old cars (i.e. made-out-of-steel rust buckets.)

</comment></comment></comment></comment></comment><comment><USER10> I think it's unfair to compare cars from 50 years ago against cars today without comparing the behavior of the drivers. Although the cars themselves may be safer, more people may be on the roads and may be driving more recklessly which would lead to a higher number of accidents as well as having the damage be equal.

I remember reading something about a study being done that showed people that cars drove much more dangerously around bikers with helmets than bikers without; if you feel that other cars are safer you may drive more dangerously.

Edit: Here's the study: <http://www.bath.ac.uk/news/2006/9/11/overtaking110906.html>

<comment><USER11> This is a very valid point and raised very well. It does not deserve these downmods.

<comment><USER12> why? this feels like a red herring fallacy.

we're talking about the technology and quality of the cars across time as it relates to safety.

you pull in people as drivers and it fundamentally changes what we're talking about. more examples:

"I think it's unfair to compare cars from 50 years ago against cars today without comparing the state of modern traffic laws and speed limits"

"I think it's unfair to compare cars from 50 years ago against cars today without comparing the quality and ubiquity of roads, highways and interstates"

etc.

<comment><USER10> My general point was that although cars are safer than before, we still need to look at the system they are a part of which affects safety. I don't want people to make the conclusion that it's safer driving now than before - <USER2> may or may not be the case.

<comment><USER13> Well... it is:

[http://www.roadandtravel.com/safetyandsecurity/highwayfatali...](http://www.roadandtravel.com/safetyandsecurity/highwayfatalityrates.htm)

Note that nationwide statistics are not available that far back, but consider this report from Washington State:

[http://www.wsdot.wa.gov/planning/wtp/datalibrary/Safety/MVFa...](http://www.wsdot.wa.gov/planning/wtp/datalibrary/Safety/MVFatalities.htm)

<comment><USER10> But that hasn't necessarily come from car safety although I'm sure that's a large contributor.

It can also be attributed to better education, more cracking down on drunk drivers, etc.

<comment><USER12> I think it's unfair to compare cars from 50 years ago against cars today without comparing the behavior of the drivers, improvement of safety education, improved laws, better police enforcement, etc..

see what i did there? ;)

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I'd like to see some comparisons of active safety (the ability to avoid crashes and retain control in adverse conditions). I suspect the improvements in handling and braking in the average car since 1959 aren't nearly so dramatic. Aside from the SUV craze, the average front-wheel-drive[0] family sedan built today feels like it wants to fall off the road every time it encounters a corner.

I have to wonder why the average car buyer doesn't seem to notice or care. I suspect most adults have, at some point in their lives driven a car with decent handling and braking.

[0] Front-wheel-drive alone is not to blame for this; I've driven FWD cars that did not feel like that.

<comment><USER15> Don't agree. I don't know if you have actually driven any of the boats of the 60-80s but they handle very very badly even if they are rwd. I am talking about large old fashioned american cars, like ford crown victorias chevy caprices, etc. They have very bad steering response shake like crazy on the slightest turn tires squeel with slightest provocation etc. Those cars were built to drive in a straight line.

I think any modern fwd sedan will do better. For example I used to drive a 90s crown victoria and then switched to a later model ford contour. The contour handled sooo much better even if it was fwd.

<comment><USER14> I have driven some of those old boats, and they do handle very poorly, but not really worse than a lot of truck-based SUVs today. Looking outside of the US where large boats never caught on, the improvement in active safety appears even smaller.

I came to the opposite conclusion regarding '90s Crown Victorias vs. Contours, however every Crown Victoria I've driven was a sport or police model with stiffer suspension than normal. Comparing to my grandmother's Buick Roadmaster, I'll agree with you.

Again, I'm not saying modern cars aren't better. I'm saying the difference in active safety is not as big as the difference in passive safety, and I'm wondering why that is. Today's best handling cars are much better than the best cars of decades ago, but the average isn't nearly so improved.

</comment></comment></comment><comment><USER16> What isn't clear to me is, do safer cars translate into fewer traffic injuries and deaths?

I recall reading an economics paper that indicated that safer cars encouraged more reckless driving and the two competing factors canceled each other out; as cars got safer, the number of accidents increased until the number of deaths and injuries returned to their prior levels.

I wonder if we all still drove 1959 cars would we be using cell phones while driving?

<comment><USER17> There is an interesting table at [http://articles.moneycentral.msn.com/Insurance/InsureYourCar...](http://articles.moneycentral.msn.com/Insurance/InsureYourCar/crash- test-1959-chevy-vs-2009-chevy.aspx)

"Highway safety then and now"

Registered vehicles: 1959 - 71.5 million 2008 - 255 million

Miles driven: 1959 - 700 billion 2008 - 2.9 trillion

Fatalities: 1959 - 37,910 2008 - 37,261

<comment><USER18> A 75% reduction in fatalities per million-miles-driven in 50 years. That's very good to see.

Thank you for bringing in the numbers.

</comment></comment></comment><comment><USER19> I wonder how much improvement there has been between more recent models of cars. I doubt that there are many people driving the 1959 Chevy Bel Air on today's highways, but there are probably plenty of people driving cars that were built in 1999. I'd be interested to see how much of a difference in safety there is between similar car models from '99 and '09.

<comment><USER20> I think the major passive safety features of this decade are better crumple zones and side-impact airbags. IIRC Euro NCAP made their crash tests much tougher recently to adapt to the improved safety across the board.

<comment><USER21> Actually, I suspect the bigger effect is driver and passenger airbags becoming standard.

<comment><USER20> I think that happened in the 90s. :)

</comment></comment></comment></comment><comment><USER22> I like the rust cloud from the 59. Makes you wonder what kind of shape the car was in before they prettied it up to crash.

I drive 2 old aircooled volkswagens in the summer and am just about ready to give up the hobby because of this. I don't like the extra amount of risk the hobby puts on my life.

<comment><USER23> According to the senior VP of the crash institute, the cloud was dirt from the 50 year old car, not rust.

[http://wheels.blogs.nytimes.com/2009/09/18/more-details- abou...](http://wheels.blogs.nytimes.com/2009/09/18/more-details- about-1959-bel-air-crash-test/)

</comment></comment><comment><USER24> The choice of camera angles seems biased. Most of the shots showed the driver's side of the old car where the impact is and only the passenger side of the new car.

I'm not disputing their overall conclusion, but it'd be fairer to start with overhead angles and show equal time on both sides.

<comment><USER25> They show the same camera angles for both, and an overhead shot of the impact. Both cars impact on the same (driver's) side. The most important two shots in this are the interior shots, and the left-hand door. On the Bel Air, you can see the passenger compartment get crushed -- the steering wheel crushes the occupant against the seat, the door is more-or-less ripped off, and the roof crumples inwards.

On the modern car, it does what it's designed to do -- the steering wheel stays put, and the passenger compartment forms a solid cage around the occupants.

</comment></comment><comment><USER26> one of the comments links on another site about this crash test links to this page:

[http://www.carcraft.com/junkyardcrawl/ccrp_0910_x_framed_che...](http://www.carcraft.com/junkyardcrawl/ccrp_0910_x_framed_chevys/photo_01.html)

There's a bit of a hint why this particular model came of so bad.

more info on the car:

[http://www.thetruthaboutcars.com/ask-the-best-and- brightest-...](http://www.thetruthaboutcars.com/ask-the-best-and-brightest-is- this-the-bel-air-the-iihs-destroyed/)

</comment><comment><USER27> You'd think that after 40 years of automobile safety innovation we could do better than this. In reality there shouldn't even be a scratch on the Malibu let alone even a remote possibility of injury to the passengers.

<comment><USER28> If the car didn't have a scratch (ie. the energy wasn't dissipated by the crumple zones), a lot more of the impact would be felt by the passengers.

What's the point of having a car without a scratch of the driver is turned to jelly...

<comment><USER29> This is true even with roll cages in race cars. You'd think you want the strongest metal available, but that's not right. They actually use a softer steel so the cage deforms a on impact.

The guy who welded mine said the shock from an impact to a titanium cage would effectively shatter your body. He was far from a doctor, but it made sense.

</comment></comment></comment><comment><USER30> Up next, a 1950s computer vs an iPhone, a TV from 1950 vs a modern day TV, what else can we compare? Everything! Technology has progressed, how about that!

Besides the omg look at the crash shock factor, what does this tell us? The only thing I get out of it, is to be reminded that we should rely on the insurance industry's crash tests, not the government's. The insurance industry is much tougher on cars.

<comment><USER31> It's a common belief that old "solid" cars are safer than their light, flimsy modern counterparts. This test dispels that belief fairly dramatically.

<comment><USER30> Is it really that common?

<comment><USER32> Mostly among the 40 and up crowd, I think. I'm not sure I know anyone younger than me who believed that (I'm 36).

<comment><USER3> I'm 28 and my primary inclination was to think that newer vehicles crumple while older cars do not, leading to more internal injuries for passengers of older vehicles and more vehicle damage of newer vehicles. However it appears that the older vehicles lose on both fronts.</comment></comment></comment></comment></comment>


Eventbrite Adds Reserved Seating - <AUTHOR> http://techcrunch.com/2014/03/05/eventbrite-reserved-seating/

<comment><USER2> Related to this is Social Tables ([https://www.socialtables.com/](https://www.socialtables.com/)), a DC-based startup making some really cool event-planning software.

</comment><comment><USER3> I built a similar (albeit not drag and drop) designer for an intranet seat- planner application eight years ago. Really like what Eventbrite are doing though. They're killing the ticketing space.

</comment><comment><USER4> AMA: I worked on the frontend for eventbrite's seat designer tool.

<comment><USER5> Why are the FEES we pay for tickets so inflated? Is there actually justified reasons as to why it takes so much to have this process happen (that isn't made visible) or is it pure profits for the man up top?

<comment><USER6> Full Disclosure: I'm an engineer at Eventbrite. But commenting independently. My views and opinions do not reflect Eventbrite's official position(s), and "oh god please don't come after me with pitchforks".

Take a look at the Eventbite fees here - [http://www.eventbrite.com/fees/](http://www.eventbrite.com/fees/)

I think they're totally reasonable. I'm not going to get into specifics that would expose our finances, but I'll say that we have a lot of people behind the scenes taking care of building and maintaining the product on both the organizer and attendee facing side. On top of that we have some _excellent_ customer service people who take care of hiccups and guide users in their time(s) of need. All those fantastic people need to be cared for (ie. paid).

If you're asking about the general perception of ticketing, I'll refer you to an article in the LA Weekly. [http://bit.ly/1kAT1K0](http://bit.ly/1kAT1K0) (tldr; There are existing patterns in particular industries to deflate the perceived ticket prices, only to inflate them at checkout.)

There's a brief clip explaining Eventbrite fees here: [http://bit.ly/1f28hw6](http://bit.ly/1f28hw6)

Hope this clears things up a bit.

</comment></comment></comment><comment><USER7> Another really good raphael use-case.</comment>


Using A-Frame for WebVR and AR - <AUTHOR> http://elevr.com/using-a-frame-for-webvr-and-ar/ <comment><USER3> I built a Kubernetes Pod visualizer using A-Frame back in October of last year - [https://medium.com/@iamnayr/building-the-kubernetes- virtual-...](https://medium.com/@iamnayr/building-the-kubernetes-virtual- reality-experience-b681464f0c98)

Fantastic framework, I haven't had my hands on it in a number of months but I know there have been some big improvements just in that time frame alone.

</comment><comment><USER4> Austin McDaniel just gave an interesting talk at ng-conf where he demoed a visualization of an Angular4 app's architecture and component parts as a series of gardens populated with trees using A-Frame.

While not perhaps a very useful visualization, seeing A-Frame in use was great and now I need a VR capable rig!

</comment><comment><USER5> I just got an Oculus Rift and am dying to try my hand at making something of my own to experience with it.

Is A-Frame a tool I could use for this purpose?

<comment><USER6> Yup, A-Frame supports Rift/Touch and Vive with controllers and room scale (Firefox Nightly)! Watch me code A-Frame room scale VR with motion capture recording [https://www.twitch.tv/videos/133862568](https://www.twitch.tv/videos/133862568)

</comment></comment><comment><USER7> This is a really amazing use of Web Components.

</comment><comment><USER8> Can anyone comment on how this compares to Unity? I want to believe.

<comment><USER9> Unity is currently far more mature than A-Frame and WebVR. Products vs very small tech demos.

A-Frame integrates with webdev tooling - <USER2> [https://github.com/aframevr/aframe-react](https://github.com/aframevr/aframe- react) . Think HTML5 vs Flash.

A-Frame is pioneering, making architectural choices which may or may not pan out. Think Angular 1.

Neither WebVR nor UnityVR are currently usable on Linux. Think "I got my simple test scene to not core dump!". MacOS has almost nothing. Android has some support from both, and iOS less, but I'm unfamiliar with them. There's a lot of Windows-centrism in the current VR community - if someone says something "works", they likely mean on Windows, with android a distant second. Once upon a time, Microsoft would write MS-only non-standard web extensions... now Google and Mozilla are doing the writing for them.

So, I generally recommend: Unity on Windows for VR beginners and exploration; Unity or [https://www.unrealengine.com/](https://www.unrealengine.com/) (better Linux support) for more serious work. But there are other options. And for WebVR, Windows (or Android?), A-Frame for beginners, direct three.js for more serious work, and perhaps aframe-react for exploratory VR webdev.

Aside: And for a hypothetical never-met person who wishes to run a Vive with WebVR on Linux, there's also my [https://github.com/mncharity/node-webvr-alt- stack](https://github.com/mncharity/node-webvr-alt-stack) ... but I didn't finish WebVR 1.1 and A-Frame support.

</comment></comment><comment><USER10> Is this just a modern VRML?

<comment><USER6> So VRML's a 3D file format, and A-Frame's an entity-component JavaScript framework. Check out [https://github.com/aframevr/aframe](https://github.com/aframevr/aframe) and you should be able to discern.

<comment><USER11> VRML both had node definitions and a scripting part, it's not just a 3D file format in the sense that it's not just describing 3d meshes but also how to interact with it.

<comment><USER6> True, but the interaction pieces were still part of the format, and probably limited in power. Though I'm a bit too young to have been experienced that era, so I don't fully understand the comparisons.

But I know today A-Frame has full access to JavaScript, DOM APIs, three.js, and Web so it's looking more towards building full stateful applications.</comment></comment></comment></comment>


How normal people will use Windows 8 - <AUTHOR> http://www.youtube.com/watch?v=v4boTbv9_nU&feature=relmfu

<comment><USER2> I visited the new Microsoft store in Boston recently and felt that the experience was on par with my expectations.

On three separate occasions the Windows 8 tablet and the POS systems froze. The employees were as confused as I was when demonstrating the device. Everything was made of metal colored plastic. The Windows Phone challenge was very rude and condescending.

I'm a full time Ubuntu user and use Thinkpads so I reasonably expected a solid build quality. I was left with a sour taste after I left the store. I felt like Microsoft was fine with cloning the Apple store and then putting down users who did not use latest and greatest Windows toaster.

Again, all on par with my expectations.</comment>


Zstandard: Fast real-time compression algorithm from Facebook Opensource - <AUTHOR> http://facebook.github.io/zstd/ <comment><USER2> see Compression Benchmark at ([https://github.com/powturbo/TurboBench](https://github.com/powturbo/TurboBench)) reply

</comment><comment><AUTHOR> tl;dr:

Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression/speed trade-off ratios, while being backed by a very fast decoder. It also offers a special mode for small data, called dictionary compression, and can create dictionaries from any sample set.

Zstd is currently deployed within Facebook. It is used continuously to compress large amounts of data in multiple formats and use cases. Zstd is considered safe for production environments.</comment>


Asynchronous Database Access in Ruby - <AUTHOR> http://www.igvita.com/2008/09/05/asynchronous-database-access-in-ruby/

<comment><USER2> There's a strong urge to come up with a Rails stack that's end-to-end async, that looks and tastes like Rails but automagically performs 10x faster.

I'm not saying that's irrational, but it's not necessarily the best way to meet speed targets. Database queries are not immune to the 80/20 rule. You find the queries that are killing you, you factor them into handlers that run straight SQL async handlers, and you keep standard ActiveRecord for the vast majority of your code that isn't performance sensitive.</comment>


PurseIO » Spend Bitcoin on Amazon, Buy Bitcoin using a Credit Card - <AUTHOR> https://purse.io/ <comment><USER2> My maths must be way out here, or the intarwebs is throwing up some very weird exchange rates. If not, this a collossal rip-off.

Most favourable exchange on there for Amazon UK at the mo:

* Buy someone's £46,85 wishlist and receive 89,66mBTC

* 1 mBTC = 13,33 US Cents

* 89,66mBTC = $11,95 = £7,11

So, spend £46,85 to buy £7,11 worth of Bitcoin. WTF??!! Even VirWox is better value than that!</comment>


(2007) Is C++ hurting Mozilla development? - <AUTHOR> http://benjamin.smedbergs.us/blog/2007-11-05/what-if/ <comment><AUTHOR> Found this recently. If I understand correctly, it interestingly lines up with the very early stages of Rust.</comment>


How Did Software Get So Reliable Without Proof? (1996) [pdf] - <AUTHOR> http://www.gwern.net/docs/1996-hoare.pdf <comment><USER2> It turns out software was only reliable in the dimensions we we carefully looking at. I'm not sure 'proof' would have helped any - we might well have proved that, say, Flash could successfully display any valid animation you could throw at it without even thinking about whether it was also pwning your machine.

<comment><USER3> Most of the security vulnerabilities involve C undefined behaviour. Any proven-correct program necessarily won't invoke undefined behaviour.

(We could also have used memory-safe languages)

<comment><USER2> Many, but I'm not sure most. However they nearly all involve someone (be it the language designer, compiler writer or application writer) not thinking about all the malicious possibilities of the code in question. Because that's not the problem they're trying to solve right now, and they're only human. And I only use security vulnerabilities as an example - the next thing we're all not thinking about might be power consumption, or creating AI paradoxes, or whatever. But we're definitely not thinking about it, and no-ones going to be proving it.

<comment><USER3> > And I only use security vulnerabilities as an example - the next thing we're > all not thinking about might be power consumption, or creating AI paradoxes, > or whatever. But we're definitely not thinking about it, and no-ones going > to be proving it.

I don't think you have to be thinking about it. Every time some new class of vulnerabilities has arisen, e.g. SQL injection, it just hasn't affected well- formed programs written with good tools, and certainly not verified programs. Correct programs just don't have these problems, as a side effect of being correct.

</comment></comment></comment></comment><comment><USER4> I haven't gotten a chance to read the entire paper however the reference to the Therac-25 in the beginning was slightly tilted. Yes - it was a programmatic error that dispersed a deadly amount of radiation. But, if I remember correctly, the Therac-25 is often used in Software/Computer Engineering ethics classes as an instance in which a hardware safety mechanism would be appropriate. So while the article is right that a software malfunction did disperse a deadly amount of radiation, the fix for this isn't necessarily a software solution

</comment><comment><USER5> How did living things get so reliable without proof?

<comment><USER6> Living things aren't very reliable. They don't last very long, in general, and their replication mechanism is highly error-prone.

<comment><USER7> That's an extremely narrow view of life. A broader view might be that life on Earth began a few billion years ago, and has managed to hang on ever since. Do you realize that you are, most likely, the direct descendant of one of those early lifeforms? Every single one of your ancestors, for billions of years, has managed to produce viable offspring in an unimaginably long chain of living organisms. How can you call that unreliable? Sure, bits of your DNA are being corrupted every second. Sure, tens of billions of your human cells die and are cast away each day. And, sure, each agglomeration of cells we call a human only lasts a few tens of years. But life has enough redundancies, contingency plans, and diversity that it will in all likelihood keep going for a long, long time.

Having a single instance of a particular organism last for a long time just isn't what life is about. If you look at the full context of what life is doing, it is way goddamn more reliable than anything humans have designed, proofs or not.

<comment><USER6> I agree with your argument as far as it goes, but if you're responding to me, you're making a straw man argument. I was saying that _living things_ ( _i.e._ individual organisms) aren't very reliable; you're saying that _life_ is very resilient. That's a true statement, but utterly non-responsive to the point I was making.

<comment><USER8> Individual organisms are incredibly reliable when you consider the range of functions they perform. Compare, say, a quadcopter drone (or any other small air vehicle) with a parrot.

Lifespan without service or repair? Drone, weeks. Parrot, decades.

Autonomy before recharging/refueling? Drone, 20 minutes. Parrot, hours to days.

Cognitive faculties? Drone, slim to none. Parrot, long term autonomous in complex environments.

Reproduction? We can't build things that do that _at all_ yet, let alone pack one into a 1kg autonomous aircraft.

Yes, we can build simple machines to replicate one function of a living creature, and those machines will be more reliable over short durations, but only because they are orders of magnitude simpler. You might as well call an LCD screen "unreliable" because it won't outlast a photograph.

</comment></comment></comment></comment></comment><comment><USER8> "Beware of bugs in the above code; I have only proved it correct, not tried it." \- Donald Knuth

<comment><USER9> Surely code you haven't proved correct is going to have way more bugs in it than code you tried to prove is correct? Maybe I'm misunderstanding but this quote is always posted as if it's some kind of rebuttal against formal methods.

<comment><USER4> I always thought that correctness (in computer science) only asserted that an algorithm acted as specified. For example, say the specification was: The input accepts an element in the set of real numbers, and the corresponding output is that number added to the next element in the set of natural numbers.

If this algorithm is used to dispense some type of medication, and I enter 3.1 and it murders you with 10 grams of alprazolam... well.... the algorithm is still correct.

<comment><USER9> > I always thought that correctness (in computer science) only asserted that > an algorithm acted as specified. For example, say the specification was: The > input accepts an element in the set of real numbers, and the corresponding > output is that number added to the next element in the set of natural > numbers.

Nobody is saying the formal proof approach will find all bugs, just that it helps find more bugs compared to not using it. You could make the same argument with acceptance testing: unless you specifically tried 3.1 as the input during testing, it might not give you the output you expect. Formal proof can be complimentary to traditional automated tests and QA.

</comment></comment></comment></comment><comment><USER10> Systems for processing highly classified governmental/military confidential information often are required (depending on the country laws) to provide formal proof of their entire work. Other than that (and because it is too expensive to run such an analysis) people just need to look at the code to check whether it's doing what it's supposed to do.

</comment><comment><USER11> > How Did Software Get So Reliable

Lol!

</comment><comment><USER12> > _Specialisation involves a deep commitment to a narrow selection of > presentation, reasoning methods, paradigm, language and application area, or > even a particular application. The whole point of the specialisation in > formal methods is to restrict the notational framework as far as necessary > to achieve some formal goal, but nevertheless to show that the restrictions > do not prevent successful application to a surprisingly wide range of > problems. This is the reason why specialist research into formal methods can > run the risk of being very divisive. An individual researcher, or even a > whole community of researchers, becomes wholly committed to a particular > selection of specialisations along each of the axes: say an operational or > an algebraic presentation of semantics, bisimulation or term rewriting as a > proof method, … The attraction of such a choice can be well illustrated in > certain applications, such as the analysis of the alternating bit protocol > or the definition of the stack as an abstract data type. The perfectly > proper challenge of the research is to push outwards as far as possible the > frontiers of the convenient application of the particular chosen formalism. > But that is also the danger: the rush to colonise as much of the available > territory can lead to imperialist claims that deny to other specialisms > their right to existence. Any suggestion of variation of standard dogma is > treated as akin to treason. This tendency can be reinforced by the short- > sightedness of funding agencies which encourage exaggerated claims to the > universal superiority of a single notation and technique._

> _The consequences of the fragmentation of research into rival schools is > inevitable: the theorists become more and more isolated, both from each > other and from the world of practice, where one thing is absolutely certain: > that there is no single cure for all diseases. There is no single theory for > all stages of the development of the software, or for all components even of > a single application program. Ideas, concepts, methods, and calculations > will have to be drawn from a wide range of theories, and they are going to > have to work together consistently, with no risk of misunderstanding, > inconsistency or error creeping in at the interfaces. One effective way to > break formal barriers is for the best theorists to migrate regularly between > the research schools, in the hope that results obtained in one research > specialisation can be made useful in a manner acceptable by the other. The > interworking of theories and paradigms can also be explored from the > practical end by means of the case study, chosen as a simplified version of > some typical application. In my view, a case study that constructs a link > between two or more theories, used for different purposes at different > levels of abstraction, will be more valuable than one which merely presents > a single formalisation, in the hope that its merits, compared with rival > formalisations, will be obvious. They usually are, but unfortunately only to > the author._

<comment><USER3> > There is no single theory for all stages of the development of the software, > or for all components even of a single application program. Ideas, concepts, > methods, and calculations will have to be drawn from a wide range of > theories, and they are going to have to work together consistently, with no > risk of misunderstanding, inconsistency or error creeping in at the > interfaces.

I'm not convinced. I remain hopeful that one or other technique will show itself adequate for complete coverage of real-world programs. Compare programming languages - naïvely one would think different languages have different specializations and it's important for programmers to be fluent in a wide variety of them and good at interfacing between different languages. But I'm increasingly coming to believe that good general-purpose languages are possible and that better systems are built in a single language, by programmers specialized to that language.

<comment><USER12> Ah, but there's a difference between formal verification and programming languages. The problem of general formal verification (known in general as the model-checking problem[1], regardless of the approach used which could be model-theoretic or proof-theoretic) is literally the hardest problem in all of computer science. Any computational problem can be reduced in polynomial time to program verification (i.e. model checking). In fact, all NP-complete and PSPACE-complete problems are trivially reduced. Finding a general tractable solution (even if it includes human intelligence) is therefore a mathematical impossibility. The best we hope for is a solution that is applicable to programs people write. The question is then whether there is one well defined "class" of programs humans write to which an efficient algorithm (again -- even one involving humans in the loop) applies. So far results from decades of verification research show that this is not the case, and there are at least several such classes, but maybe the classes aren't general enough. If humans are part of the algorithm (e.g. interactive proofs, manual dependent types), the class might be wide, but our experience has shown that verifying programs in this way takes tremendous effort, even for small programs, and even when the programs are kept especially simple for the sake of verification. Even if there is a solution that is applicable to most (I'll settle for most) programs, it is so far nowhere in sight. Most efforts are directed at finding partial solutions for specific domains.

[1]: Called that because it is the problem of _checking_ whether a given program is a satisfying theory -- i.e. a _model_ \-- for the given logical property.

<comment><USER3> That it's a hard problem doesn't necessarily mean we're approaching it wrong. Indeed the fact that it's mathematically proven to be hard offers some confidence that we might not be.

Are there examples of cases where a system was very hard to verify with any one general technique, but much more practical to verify by using several domain-specific techniques and some kind of bridge between them (i.e. not simply coming up with a general approach for which the specialized techniques were, well, specializations - since as far as I can tell your quote is advocating for something different)?

<comment><USER12> Sure. In fact, I believe this is the norm rather than the exception. Software systems often use model-checkers, abstract interpretation and manual deduction to be verified. Each have their strengths and weaknesses. In fact, you can see all three methods used in one of the common representations of logic (which I've picked because I know you're a Scala guy): types. Simple types (like H-M types) perform crude abstract interpretation, but can only prove properties of finite state machines. Liquid types use model-checking algorithms to infer and check some kinds of dependent types. These, however, still don't cover all properties, so in practice, people need to resort to manually-inferred/checked dependent types. Using those alone would be extremely cost prohibitive.

In the safety-critical software world, you see projects using model checkers to verify individual modules and deductive proofs (say, in Isabelle) to verify the interaction among modules or vice-versa. In the less-safety-critical software world, you see people commonly use static analyzers (employing abstract interpretation), even though those can only verify certain properties, because both interactive deductive proof and automatic model checking are too cost prohibitive.

But the problem in general is this: we are not at a point where any single approach nor a combination of several lets us fully verify a real-world program even of moderate side at any acceptable cost. I don't believe any moderately-sized program has ever been fully verified using any technique.

<comment><USER3> > But the problem in general is this: we are not at a point where any single > approach nor a combination of several lets us fully verify a real-world > program even of moderate side at any acceptable cost. I don't believe any > moderately-sized program has ever been fully verified using any technique.

Sure. Which means I think we simply don't know enough to say one way or the other what kind of approach we will need for effective proofs. Maybe we will need to combine multiple techniques and have bridges between them. But maybe we really will be able to extend one particular technique to cover all the important cases, or find a single general approach that recovers all the useful techniques we've found so far as special cases.

<comment><USER12> Right. But it is important to understand that this is the hardest problem in computer science (human-level AI is much easier), and so general-enough solutions must somehow be aligned with how humans tend to construct programs (because a completely general solution is impossible, and solutions that are not aligned with how people program can only hope to solve human-produced code by accident). For example, we do know that we cannot create a general-purpose language that will make verification easy for all programs written in it; that is a mathematical impossibility. Whatever approach may work one day, it will have to be based on empirical observations of programs people tend to write.

<comment><USER3> > For example, we do know that we cannot create a general-purpose language > that will make verification easy for all programs written in it; that is a > mathematical impossibility.

Not true. We know that we cannot create a Turing-complete language with that property. We haven't proven that we'll never be able to create a general- purpose non-Turing-complete language. I might argue that Idris is already an example of such a thing.

<comment><USER12> > We haven't proven that we'll never be able to create a general-purpose non- > Turing-complete language.

Well, if you're referring to total languages, then they are Turing complete for all intents and purposes.

Total languages are not immune to the halting problem (or, rather, to its finite generalizations known as "bounded halting"). The classical formulation of the halting problem simply tells you that verifying a program in general is undecidable. But this is simply because a Turing machine is a finite structure that can describe an infinite one. So in some way, the halting problem is undecidable in the same way that finding the maximum or sorting an infinite list doesn't terminate. Yet once the list is finite (or the Turing machine is known to terminate), you then need to ask yourself how hard is it to find the maximum element in a finite list, how hard is it to sort it, and how hard is it to verify a program that is known to terminate? The answers to all these questions are well known, and quite different from one another. Finding the maximum takes Ω(n) (worst-case), sorting is Ω(n lg n), and verification is Ω(|S|), namely at least linear in the number of states the program takes until it terminates (this directly follows from a trivial generalization of the halting problem called "bounded halting"). So sure, total programs are known to terminate, and all we have left to do now to prove them correct is to overcome a super-exponential lower complexity bound... In practice that is just as undecidable as for programs that are not proven to terminate.

And, BTW, the lower complexity bounds for verification of finite state machines written in expressive languages is already PSPACE-complete. There cannot be a useful _general_ programming language with verification bounds that are less than exponential (unless, that is, PSPACE=P); but that is a good thing: verification of total-functional programs is super-exponential. So my point is that even completely general verification of freakin' finite state machines is out of our reach in large-enough programs. It's doubtful you can find a useful computational model that's simpler.

Also, notice that any program could trivially be made total without changing its semantics. Give me a program you want verified, I'll add a decreasing counter starting with 10100 to all its loops/recursions, and now your program is total and you'll never know the difference. Does that make it any easier to verify?

The total-language people know this. They don't claim that this makes practical program verification any easier. In fact, the motivation for totality has nothing to do with program verification but with using C-H to prove general mathematical theorems. Non-termination, as you know, is the same as inconsistencies in those proofs (although not in proofs about programs; non-termination introduces no inconsistency to those). However, some people who are only familiar with the classical formulation of the halting theorem have concluded that totality => no halting problem => verification is possible! This is false, and was abundantly clear even in the 50s. Kurt Gödel, in a letter to von Neumann in 1956, mentions in passing this finite (and obvious) generalization of the halting theorem before raising (for the first time!) the question of P vs. NP.

<comment><USER3> > unless, that is, PSPACE=P

Which of course remains an open question.

> even completely general verification of freakin' finite state machines is > out of our reach in large-enough programs. It's doubtful you can find a > useful computational model that's simpler.

Eh, maybe. I find it very hard to reason about finite state machines in programs as a human, so it doesn't surprise me that proving correctness properties about them would be hard. I'm not at all convinced that a general- purpose programming language necessarily includes support for fully general finite state machines.

> Also, notice that any program could trivially be made total without changing > its semantics. Give me a program you want verified, I'll add a decreasing > counter starting with 10100 to all its loops/recursions, and now your > program is total and you'll never know the difference. Does that make it any > easier to verify?

Yes - it eliminates the class of errors where one has "implemented" something by an infinite loop. It helps for concrete programs in the same way that it helps for theorems, no?

<comment><USER12> > Which of course remains an open question.

Technically yes, but as Scott Aaronson says, a universe where P=NP would probably look very different from ours (and PSPACE is harder than NP). In any event, verifying effectively Turing-complete languages (even total ones) is much harder than EXPTIME (and we know that EXPTIME > P).

> I find it very hard to reason about finite state machines in programs as a > human

I didn't mean it like that (programming with explicit states). Just a model of computation where the number of states is bounded. Any program with finite memory is a finite state machine. Every language must support at least that.

> it eliminates the class of errors where one has "implemented" something by > an infinite loop. It helps for concrete programs in the same way that it > helps for theorems, no?

No, because when you prove a _program_ property A you know that you've really proven "A or no-termination". So being able to prove any property is not an inconsistency. This is not the case for mathematical theorems that don't have running time, and where "no termination" is meaningless. If running time is 0, then A just means A, and if A is a contradiction then you have a real problem.

Now, what about eliminating non-termination from programs? Well, you're not really doing that either, because as far as we're concerned, a program that takes 10^100 steps doesn't really terminate at all. One could claim that people are more likely to accidentally write a function that loops forever than one that loops 2^64 times. This may be true, but this again relies on empirical facts rather than general mathematical properties.

In any case, what makes you think that languages like Idris may make verification easy? The few cases of programs verified in this mostly-manual way took _years_ to prove, even for very small and intentionally "dumbed down" code.

<comment><USER3> > I didn't mean it like that (programming with explicit states). Just a model > of computation where the number of states is bounded. Any program with > finite memory is a finite state machine. Every language must support at > least that.

Any program is (or can be modeled as) a finite state machine, sure. But useful programs are a small subset of the space of possible finite state machines. It's not beyond hope that we could find a subset that allowed us to write most useful programs but was easier to reason about.

> One could claim that people are more likely to accidentally write a function > that loops forever than one that loops 2^64 times. This may be true, but > this again relies on empirical facts rather than general mathematical > properties.

I am worried about not-terminating-in-reasonable-time (though much less worried than I am about looping forever), but can't we just make that one of the properties we keep track of and prove with the same tools?

> In any case, what makes you think that languages like Idris may make > verification easy? The few cases of programs verified in this mostly-manual > way took years to prove, even for very small and intentionally "dumbed down" > code.

I guess my experience that keeping track of more properties in the type system doesn't make coding any harder. Compare Rust - writing Rust programs essentially means proving that things have valid lifecycles and are not used outside of them. This is a complex property that mainstream programmers were not previously proving at all - that many had despaired of ever being able to prove. And yet in practice people who write Rust find that not only can they do it, but the additional proof effort quickly becomes barely noticeable. Likewise with Idris you're proving termination in finite (though possibly unreasonable) time and once you get used to doing that it doesn't actually seem noticeably harder than not doing so. So I'm kind of hopeful that we can just keep improving the type systems used by mainstream programming languages until writing a program naturally includes proving the important properties of that program without even noticing.

<comment><USER12> > It's not beyond hope that we could find a subset that allowed us to write > most useful programs but was easier to reason about.

It is beyond hope that we find a subset that's _generally_ easier to reason about. Verifying one line programs of this sort:

        if (a && ~b || c || ~d && e ...)
          crash()

is already NP-complete (SAT). Adding some loops that are of depth 2 makes it PSPACE-complete (TQBF, aka QSAT). I can't think of a useful programming language that doesn't have even small-constant-depth loops (which makes it finite state) and simple boolean algebra. Software verification is about verifying some subsets of programs that humans happen to write. It is categorically impossible to write a general-purpose language where _all_ programs are easier to verify than PSPACE.

> but can't we just make that one of the properties we keep track of and prove > with the same tools?

Sure, with super-exponential complexity :) But again, this is in the worst case. We hope that humans _usually_ write programs that are simpler.

> I guess my experience that keeping track of more properties in the type > system doesn't make coding any harder.

If you mean H-M types as in Haskell, then the properties those types enforce are either trivial (i.e. true for all programs, e.g. function returns Int), or a result of a finite state machine (that's the proving strength of H-M types). You cannot prove even a single bit of a program's output that's a result of non-constant-depth recursion (there's a special case for recursion which is of depth that's a constant multiple of the input length, like `map`). If we take Haskell, you can replace each of your functions with constant-depth recursion (0 or 1 would suffice in virtually all circumstances) and the program will still typecheck yet will be completely wrong.

As for Rust's linear types, that is precisely what I'm talking about. A certain class of bugs (memory errors) that humans are known to make and that verifying with abstract interpretation is easy. _That 's_ what software verification aims to do. We cannot hope of proving general programs generally correct, regardless of the language they're written in.

BTW, thinking of type systems muddles understanding in this case, because type systems are just notation. The verification taking place in most sound type systems is abstract interpretation. Yes, we can find more important properties to verify with abstract interpretation, but it has been proven that some properties will forever elude abstraction.

The way abstract interpretation works is that it translates the program into an "abstract" program which is just a finite state machine, with the number of states equal to the number of concrete types, and running an exhaustive model checker on that; this means that the complexity of inferring this kind of types is Ω(|S|), namely, linear in the number of concrete types. As abstract types can multiply the number of concrete types, the inference can become exponential in the worst-case, but it usually doesn't. Once you introduce dependent types, the algorithm is still Ω(|S|) -- linear in the number of state, i.e. types -- but now the number of types is unbounded, and so the problem becomes undecidable.

> until writing a program naturally includes proving the important properties > of that program without even noticing.

You're right, but this entirely depends on which properties are found to be important (and has little to do with type systems specifically; again, this is mostly notation[1]), and that, in turn, is rather empirical.

[1]: Not entirely, though, but mostly.

<comment><USER3> > I can't think of a useful programming language that doesn't have even small- > constant-depth loops (which makes it finite state) and simple boolean > algebra. Software verification is about verifying some subsets of programs > that humans happen to write. It is categorically impossible to write a > general-purpose language where all programs are easier to verify than > PSPACE.

Eh. Again, I don't think we necessarily need arbitrary boolean algebra - something that's hard enough to reason about as a human - and may be able to capture the valuable use cases in a simpler model. Fundamentally if we have any hope of doing verification at all, surely we can characterize the constraints we need for verification and then push them back to the programming language level.

> If we take Haskell, you can replace each of your functions with constant- > depth recursion (0 or 1 would suffice in virtually all circumstances) and > the program will still typecheck yet will be completely wrong.

Could you expand on this / give an example? I don't think I follow. And I take it from what you're saying that this doesn't apply if we permit dependent types?

> BTW, thinking of type systems muddles understanding in this case, because > type systems are just notation. The verification taking place in most sound > type systems is abstract interpretation. Yes, we can find more important > properties to verify with abstract interpretation, but it has been proven > that some properties will forever elude abstraction.

Properties that are important for real-world programs? Abstract interpretation sounds very similar to how we reason about programs as humans. It just seems like to the extent that we understand how to do something at all we should be able to formalize that understanding.

If we're talking about using programs to address open questions - a best- effort at the travelling salesman problem, say - then I can see how you might end up with a program that had a pile of heuristics that seemed to work but with no concrete understanding of how or if it worked - except that even then you'd probably use a well-known, formally validated heuristic rather than using the program as a medium for mathematical research. I just think (empirically) it would be very rare as a business requirement to need to do something completely unproven, and therefore it might be possible to have a useful general-purpose programming language that didn't allow doing that.

<comment><USER12> > Fundamentally if we have any hope of doing verification at all, surely we > can characterize the constraints we need for verification and then push them > back to the programming language level.

That is not what the verification community is trying to do at all, because trying to find a language that is not even NP-complete to verify and is still _generally_ is absolutely impossible (explanation later). What the verification community is trying to do is to come up with approaches that make it possible/easy to verify _some_ code, which is hopefully the majority of code. As far as languages go, the hope is that a language can make some verification of some code easier than in other languages. A general-purpose language that is at least useful in some cases, and in which _all_ programs are easily verifiable can only exist if P=NP. The reason is this: at best, you can hope that the proof of correctness is checkable in polynomial time. This puts it in NP. If _constructing_ a proof of correctness is possible in polynomial time, so is coming up with the solution in the first place. Namely, such a language wouldn't even need to be programmed. It will be a constraint solver (that somehow is in P). SQL minus boolean expressions might be the best you could do. Possibly useful, but certainly not general-purpose.

> Properties that are important for real-world programs?

Yes, and they are quite common in distributed algorithms.

> It just seems like to the extent that we understand how to do something at > all we should be able to formalize that understanding.

Yet we can't because when we reason about something we assume a certain regularity of the state-space, that is often wrong in practice. That link I showed you where there are visualizations of state spaces, shows the one for the CAN bus protocol. It is the most complicated example on that page, and it is the very real protocol used by different components in your car that need to communicate.

> I just think (empirically) it would be very rare as a business requirement > to need to do something completely unproven

That depends a lot on what you do. If you design schedulers for OS kernels, concurrent data structures, or distributed algorithms you see that these difficult cases are the rule rather than the exception.

I work on concurrent and distributed algorithms, and I do formal verification (TLA+ is my current tool of choice) most day every day. TLA+ offers both a theorem prover (a la Isabelle/Coq/Agda) as well as a model checker. Manual deductive proofs are virtually never worth the trouble (that would take me years), and even with a model checker -- where you just press a button and if there's a bug, any bug, you get a full trace of the execution that led to the bug -- it's still a lot of work, because distributed algorithms are just essentially complex.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> The paper is talking about rigorous formal mathematical proofs of correctness like in the sense of of the seL4 kernel ([https://sel4.systems/Info/FAQ/](https://sel4.systems/Info/FAQ/)) where they have confirmed the kernel is completely free of bugs such as buffer overflows, null pointer exceptions, use-after-free, etc.

<comment><USER13> Of course, there might be bugs in the specification (the proof the made was that the kernel matches the specification)

<comment><USER9> > Of course, there might be bugs in the specification (the proof the made was > that the kernel matches the specification)

Nothing is perfect, but this approach catches more bugs than any other technique. Also, some specifications are simple to specify e.g. it's not hard to specify that an array index must be within the bounds of the array and if you did get this wrong you'd notice quickly as you'd be unable to prove obvious facts are correct.

<comment><USER13> _but this approach catches more bugs than any other technique_

Have you seen the specification for sel4? It's fairly complex.

</comment></comment></comment></comment><comment><USER14> All programming is proof. You just don't often know what exactly it is you're proving.

Anyway, the point is moot: software is not that reliable. It's good at the common case, and is very brittle at the edge cases. That's my experience anyway.

<comment><USER15> I was a math major and while trying to get a job in programming I got asked how I felt my skills in mathematics would transition to programming-

The language (spec) is my set of axioms, the standard library existing theorems, and my program the proof.

It's kinda drab now that I think about it, but for a 'firing from the hip' answer I still stand by it.

<comment><USER16> Not so drab -- that sort of thinking can lead to a pretty big deal, namely, the Curry-Howard Isomorphism[0]. From the linked WP page:

 _In other words, the Curry–Howard correspondence is the observation that two families of formalisms that had seemed unrelated—namely, the proof systems on one hand, and the models of computation on the other—were, in the two examples considered by Curry and Howard, in fact structurally the same kind of objects.

If one now abstracts on the peculiarities of this or that formalism, the immediate generalization is the following claim:_ a proof is a program, the formula it proves is a type for the program. _More informally, this can be seen as an analogy that states that the return type of a function (i.e., the type of values returned by a function) is analogous to a logical theorem ... and that the program to compute that function is analogous to a proof of that theorem. This sets a form of logic programming on a rigorous foundation: proofs can be represented as programs, and especially as lambda terms, or proofs can be run._

(Of course, in the context of those job interviews you mentioned, this sort of thing would probably be overkill, unless the job involved functional- programming...in which case, though, they might well not have asked you how your "skills in mathematics would transition to programming" in the first place :-) ).

[0] [https://en.wikipedia.org/wiki/Curry–Howard_correspondence](https://en.wikipedia.org/wiki/Curry–Howard_correspondence)</comment></comment></comment>


Changes to Linkedin API Developer Program - <AUTHOR> https://developer.linkedin.com/blog/posts/2015/developer-program-changes

<comment><USER2> This can only accelerate linkedin becoming (more) irrelevant which is not a bad thing in my opinion. I keep a linkedin (basic) profile because it's handy to stay in touch with work acquaintances and keep doors open. But for the love of God, no, I will not grant you access to my address book. No, Mr. Headhunter, you may not have my resume before showing me an actual opportunity.

Linkedin is now eerily similar to Facebook (which I shun) with work anniversaries, timelines, "You may know" and "you may be interested in" lists and all this other nonsense. It's nothing more than a trolling site for headhunters and large corps to pick off low-hanging fruit.

No big loss.

</comment><comment><USER3> We feel like we could be making more money. We also don't like that occasionally bits of user generated content make it out of our captive portal. We're very sorry about having to blah blah _yawn_ ... Sorry, just got really bored in the middle of lying to you. Anyway there nothing you can do so deal with it.

</comment><comment><USER4> Opinion from ApiEvangelist: "Ultimately, the move by LInkedIn is no surprise to me, and the platform is purely a distribution channel for me, and has been for some time.. Meaning I only syndicate content there, and you will never find me actually engaging very deep on the platform, building relationships there, because along with other platforms like Quora I do not have any ownership over any of the exhaust I generate. As a professional this is unacceptable to me, as I have a valuable brand that I carefully maintain. As other professionals realize this, they too will mostly abandon the business social network, leaving it to be a spammy corner of the Internet where HR professional prey upon the semi-professional, aspiring employee types." [http://apievangelist.com/2015/02/12/changes-to-linkedin- deve...](http://apievangelist.com/2015/02/12/changes-to-linkedin-developer- program-are-no-surprise/)

</comment><comment><USER5> Definitely could see this coming. While it's a shame that linkedin is closing down their API even more, they are just shooting themselves in the head which is great news :)

Anyway, the future of professional social networks is destined to be rich ones for each profession/vertical, not the "crappy/high noise for everyone" linkedin model.

I know I'm focusing more on crunchbase[0] and angellist[1] APIs for my startup.

[0][https://developer.crunchbase.com/](https://developer.crunchbase.com/)

[1][https://angel.co/api](https://angel.co/api)

<comment><USER6> Don't be so sure about Crunchbase API. It's owned by AOL, which haven't played nicely in the past, too. [https://news.ycombinator.com/item?id=6680040](https://news.ycombinator.com/item?id=6680040)

<comment><USER5> Sure, the "halflife of openness" of crunchbase is probably smaller than angellist.

However, at this point crunchbase is pretty open (to the point of giving data dumps) so I'll give them the benefit of doubt for the next 2 years.

Anyway, startups seem to tend towards closing off access to their data as they become the dominant player so it's a question of when not whether.

</comment></comment></comment><comment><USER7> Does anyone know of any existing (scrubbed for anonymity is fine) datasets of linked-in data? Doesn't have to be particularly recent.

<comment><USER8> For free or for money?

I believe there are web scraping companies that sell monthly dumps of LinkedIn data, but I don't know off-hand how much something like that would cost.

It would also likely depend on whether you wanted US-only data or worldwide.

<comment><USER7> US-only is fine. For money is potentially fine (I've no idea how much something like that would cost), if I knew it was legal - though if there was a legal way to get that data through scraping I'd probably just do that myself :)

<comment><USER8> As I mentioned in a sibling reply, legality depends on your jurisdiction but in the US AFAIK scraping is no different from connecting to LinkedIn 15,000,000 times with a copy of Chrome.

To further illustrate this point, LinkedIn provides a helpful directory at the bottom of their homepage, allowing one to browse every user[0] and every company[1] in their graph. Hard pressed to argue that it's illegal to navigate to every single profile if you serve a directory of them.

Also, I believe there are companies[2] and shops[3] that specialize in doing that kind of thing. I would say it would be worth reaching out to them, if nothing else just to get a sense for what something like that would cost.

0 = [https://www.linkedin.com/directory/people-1-5-1/](https://www.linkedin.com/directory/people-1-5-1/) 1 = [https://www.linkedin.com/directory/companies](https://www.linkedin.com/directory/companies) 2 = [http://scrapinghub.com/](http://scrapinghub.com/) (makers of the outstanding Scrapy framework!) 3 = [http://webscrapinghelp.com](http://webscrapinghelp.com)

</comment></comment></comment></comment><comment><AUTHOR> It looks like they also wiped out their developer forum which was home to a wide range of questions and answers for specific API uses. It's too bad this is gone, it had some great content.

[http://developer.linkedin.com/forum/](http://developer.linkedin.com/forum/)

</comment><comment><USER9> This is one of the two reasons that I refuse to sign up for LinkedIn. The other one is spam.

</comment><comment><USER10> [https://developer.linkedin.com/support/developer-program- tra...](https://developer.linkedin.com/support/developer-program-transition)

They're basically restricting access to 90% of their API.

</comment><comment><USER11> Forgive me for the possibly silly question, but does this mean developers won't be able to make LinkedIn a login option for their apps? I'm not sure if and how OAUth and APIs are related.

<comment><USER12> You will still be able to login with linkedin: [https://developer.linkedin.com/docs/signin-with- linkedin](https://developer.linkedin.com/docs/signin-with-linkedin)

They just rehashed the website @ developer.linkedin.com

</comment></comment><comment><USER13> The classic pattern. Have an open API, grow until you think you're large enough that people are locked in, then lock it down.

</comment><comment><USER14> This is a non constructive comment... they suck ...</comment>