Wind turbines scalability limited by roads and underpasses - <AUTHOR> http://www.nytimes.com/2009/07/23/business/energy-environment/23turbine.html?_r=1

<comment><USER2> The technical problems themselves would be interesting to work on, and I'm glad I know about them now.

But sadly, the article itself is exactly what you'd expect from an undereducated journalist writing on a subject that sits at the intersection of two disciplines (engineering and economics) that lay audiences are all but incapable of understanding separately. You can tell, for example, that even for the author the notion of "opportunity cost" just didn't penetrate, if he was exposed to it at all. One of many bizarre moments of hilarity: the knee- jerk impulse to inform us that the death toll from transporting turbine parts has risen as high as one.

There is no discussion whatsoever of the real costs of large-scale deployment of wind power systems. At best (i.e., if the rollout of wind generation continues to accelerate), we _might_ be able to get 4-5% of our power from wind after another 20 years of effort, with the help of generous federal subsidies and at the cost of greatly increased instability in the power grid.

EDIT: One more thing. I'm skeptical that this new information will prompt environmental groups to stop arguing that we should ignore nuclear power options because the build-out phase would put too much stress on existing infrastructure.

</comment><comment><USER3> This seems like it would be decent application for the modern cargo dirigibles that have had so many false starts recently.

e.g. [http://www.engadget.com/2008/07/11/boeings-skyhook- jhl-40-bl...](http://www.engadget.com/2008/07/11/boeings-skyhook-jhl-40-blimp- gives-us-hope-for-yesterdays-futu/)

<comment><USER4> Or making them onsite...

</comment></comment><comment><USER5> Agree with Grandmaster Birt -

That NYT writer has become a real renewable energy Concern troll: always whining about some so-called unsolvable problem for renewable energy at the NYT Green Inc blog - it should be called Brown Inc.

</comment><comment><USER6> It's kind of impressive how the NYT is regarded as The Enemy's Propaganda Organ for people of so many disparate political alignments.

Really, people - noting a genuine technical problem with effects on peoples' lives is some attempt at sabotaging renewable energy? That attitude's right up there with the warbloggers from a few years back who'd squeal in outrage if anyone in the media dared question the president.

</comment><comment><USER7> Airship?

</comment><comment><USER8> This is one of the worst piece of journalism I've seen.

The "omfg" stories were horrible. They are claiming that wind power sucks because we have bad drivers on the road slamming into overpasses.

<comment><AUTHOR> I think that the logistics involved with the deployment of wind turbines is really a limiting factor. As such, this is an excellent opportunity for hackers to work on. The question is, given the constraints that current wind power generation faces, what could we do to overcome them?</comment></comment>


Shadowless Skyscrapers - <AUTHOR> http://nymag.com/daily/intelligencer/2015/11/welcome-to-the-age-of-shadowless-skyscrapers.html <comment><USER2> So basically, the architects have actively designed a courtyard between the buildings where the sun is guaranteed to be in my eyes 100% of the time if I turn my head the wrong way.

I don't know why that seems so annoying to me. Fundamentally, it's just line- of-site to the sun, the same as just being "in sunlight". But for some weird reason, the subjective experience of having the sun glint off a glass surface and hit me in the face is really unpleasant.

The idea and implementation are really cool, though.

<comment><USER3> You may find sunlight annoying, but many people actually love it. Especially in some places where (and during some times when) people do not get that much of it.

For instance, a Norwegian town sitting at the bottom of a narrow valley set up a large mirror to get a little more time of sunlight: [http://edition.cnn.com/2013/11/01/travel/rjukan-sun- mirror/](http://edition.cnn.com/2013/11/01/travel/rjukan-sun-mirror/)</comment></comment>


The "Design by Committee" Straw Man - <AUTHOR> http://www.attackofdesign.com/the-design-by-committee-straw-man/

<comment><USER2> The classic counterpoint to the author is a fun video from Microsoft about Microsoft:

[http://www.youtube.com/watch?v=aeXAcwriid0&feature=relat...](http://www.youtube.com/watch?v=aeXAcwriid0&feature=related)

The author is speaking to a particular definition of "design by committee." The article doesn't address the extrinsic constraints placed on a particular project by people with a place at the table but who don't have any responsibility for the success of the project.

<comment><AUTHOR> Wait, I'm confused, are you speaking about my article, which is itself a counterpoint, or the original article?

Whatever the case, I completely agree that a real discussion of design by committee should include managing all the different types of shareholders, and not just the "dumb client's wife" type.</comment></comment>


Smalltalk on the JVM - <AUTHOR> http://redline.st/

<comment><AUTHOR> Unfortunately the Guides and Books links aren't much use. Instead, have a look at the "small talks" series on his blog <http://blog.redline.st/category/series/>

<comment><USER2> Yeah we haven't really done anything with guides and links yet ( more pressing, mundane details ). I'd just skip them entirely and read what James has so far in the Small talks series.

We also did a presentation at Smalltalk Solutions a couple months ago.. <http://vimeo.com/22084832> which has some good background info from a 'what are they trying to accomplish angle'. Its geared very much towards a smalltalk audience but others can get stuff out of it.

There is also the mailing list where we'd be happy to answer any questions.

</comment></comment><comment> mister-m I'm not sure I understand what this is. This is smalltalk on the JVM, right?</comment>


Big Data's Big Problem: Little Talent - <AUTHOR> http://online.wsj.com/article/SB10001424052702304723304577365700368073674.html <comment><USER2> I'm not sure that the kinds of employees that this article describes will ever be a large number. There could be more of them in the future, but someone who is top-notch at all of statistics, programming, and data-presentation has long been less common than someone who's good at one or two of those. Companies might consider looking at better ways to build teams that combine talent that exists, instead of pining for more superstars.

I'm reminded indirectly of an acquaintance of mine who works on repairing industrial machinery, where companies complain of a big skills shortage. They either fail to realize or are in denial about what that means in the 21st century, though. It might've been a one-person job in the 1950s, a skilled- labor type of repairman job. But today they want to find one person who can do the physical work (welding, etc.), EE type work, embedded-systems programming (and possibly reverse engineering), application-level programming to hook things up to their network, etc. Some of these people exist, but it's more common to find boutique consulting firms with 3-person teams of EE/CE/machinist or some such permutation. But companies balk at paying consulting fees equivalent to three professional salaries for something they think "should" be doable by one person with a magical combination of skills, who will work for maybe $80k. So they complain that there is a shortage of people who can repair truck scales (for example).

<comment><USER3> I completely take most of your points, but I think that pretty much all quantitative PhD's are going to be close to "data scientists". Given that stats and explaining your research are requirements, all that's left is to train them to program, which a lot of people are already doing. As a matter of fact, since I heard about this big data stuff I've been honing my skills in this area, in case the hype actually manifests.

<comment><USER4> _delirium's post acknowledges your point but is looking for an even rarer person:

 _"There could be more of them in the future, but someone who is top-notch at all of statistics, programming, and data-presentation has long been less common than someone who's good at one or two of those"_.

Someone that can program, understands statistics and can present the data in an appealing manner without losing significant fidelity. Many people underestimate the difficulty and skill required in presenting data in a way that makes sense and also actually says something.

There is a significant gap between presenting data that is satisfactory to a research advisor and something that a business person with barely enough time to think can grasp without misconception.

<comment><USER3> Again, I completely see the difference (and am actually in the process of moving full time to the private sector from academia, so will probably understand a lot more in six months) but visualising data well is not that hard. Step 1: learn R Step 2: Learn PCA Step 3: Learn ggplot2 Step 4: play with the different geoms until you understand them (seriously though, everyone's eyes are optimised to find patterns, and if you can apply significance testing to these then you should be good) Step 5: profit!? Note that I am being somewhat facetious here, but I suspect that the mathematical knowledge and ability to apply this to business problems will be the real limiting factors, as good practices in data analysis, programming and visualisation can be learned. Granted that will take a long time to learn, and there will be individual differences, but its doable.

Whether or not it will be done at all though is another matter.

Again, delirium's point is trivially true if one requires these people to know _all of_ statistics, programming and data presentation as I don't think there's anyone who knows all of any _one_ of these subjects.

I suppose it somewhat depends on what the skill levels for each of these areas need to be, and that varies from person to person as well as from application to application.

<comment><USER5> Allow a short vignette from a former academic and now management consultant.

We spent six months at a major pharmaceuticals client examining their reimbursement data. Poring over many millions of rows of transaction data and thousands of payment codes (which, of course, were unique across sales geographies), we determined the ten regions at highest risk of reimbursement collapse. R was used, maps were created, beers all around.

But almost none of it was used for the executive presentation. In fact, the only part that was included was that we had ten regions that needed fixing, and our suggestions on how to fix it. You see, the CEO was dyslexic, the chairman of the board was colorblind, and the COO was a white-boarding kind of gal, so given this audience the nuts and bolts of our advanced statistical analysis were simply irrelevant.

This is hardly surprising. If we are having so much trouble hiring people who are fluent in Big Data, how can we expect business leaders to be even conversant? With only slight exaggeration, the way you do your analysis and the visualizations that you create are not important.

Companies are demanding Big Data scientists because they suddenly have lots of data and see the term Data Scientist in the news. But what they really want is not Data Scientists, it's business insights and implications from Big Data. The customer needs 1/4" holes, but we're all arguing over which brand of laser powered diamond drill they should buy.

<comment><USER6> Nailed it.

</comment></comment></comment></comment></comment></comment><comment><USER7> "claims of severe talent shortage in Big Data [http://online.wsj.com/article/SB1000142405270230472330457736...](http://online.wsj.com/article/SB10001424052702304723304577365700368073674.html) Ok... where are the high salaries (500k$ a year)? No? No real shortage."

<https://twitter.com/#!/lemire/status/196245665951649793>

Business has a shortage of "big data" folks in much the same way I have a "huge sailboat" shortage. Neither of us want to pay for it. We want it, but not for the going rate. Only one of us has a media platform, though.

<comment><USER8> The salaries are already moving north of $200k even outside of Silicon Valley and New York City and getting more expensive by the month. How high do they have to be before we have a "shortage"? The problem is not lack of money, it is that demand has greatly outstripped a finite supply.

Very high wages do not automagically create new people with the requisite skills and this is the real bottleneck. It takes significant aptitude and years of training/experience to become useful as a "data scientist". It is not as easy as I think people are imagining. We train people with excellent raw skills where I work, usually strong applied mathematics backgrounds with natural programming skills. It is much easier than trying to find someone outside with these skills, though we do attempt outside recruitment. It still takes years to develop the people we train into a good, basic data scientist.

<comment><USER9> Look, this job title is at most 2 years old. How can someone have years of experience in this? OTOH, there are plenty of people with strong applied math and good programming skills.

<comment><USER8> The set of skills existed before it had a trendy job title so you can have the experience even if it was called something else. This is true of most of the people currently working as data scientists. In a similar vein, I was designing big data systems years before "big data" became a term or trendy. For any particular odd skill mix you can come up with, there are people with that skill mix who are already doing a similar job. But usually people do not intentionally build that skill mix until it becomes an official job title and career path in the eyes of the public so it is a very small pool of people.

In the case of modern data scientists, having strong applied mathematics and programming skills is about halfway to where you need to be and a good starting point. The demand has temporarily grown much faster than the convertible talent pool can develop the additional set of skills required.

<comment><USER9> I am of opinion that if demand is high enough, companies will start hiring "halfway there" people. But this will happen only if the market grows big enough. Right now it is still a niche market where companies are cherry- picking right candidates, it seems. At least this is the impression I get from reading this thread.

The question of the size of the market is crucial. Small labor markets are very inefficient. This means that the number of qualified people is small enough, but the number of companies they can choose from is also small. It is hard to find a job when the number of companies hiring is probably less than 100.

</comment></comment></comment></comment></comment><comment><USER10> As an engineer who's investing in developing "deep expertise in statistics and machine learning" I can only stand to benefit from it, but something about the current wave of Big Data hype makes me instinctively a bit wary.

Does this skills shortage really exist to the extent claimed? are there really enough people out there who would know what to do with a 'data scientist' if they were able to hire one? I see more talk than action, I see vendors circling around looking to flog freshly-buzzword-compliant BI tools, prognosticators trying to push nervous businesses into engaging in an arms race over data.

Of course there's real value there too, for some at least. I hope my concerns prove unfounded, but worth retaining a healthy skepticism I feel :-)

<comment><USER11> As someone in the big data field on the ground (VP of Engineering). Let me give you my thoughts on it.

Your impression about the hype is correct. There are a lot of vendors offering BIG solutions, if you pay them BIGGER money. Where I used to translate the word enterprise to $$, now I translate Big Data to $$$$$$$.

When I'm hiring, I don't go looking for Big Data people, because generally they don't exist. Statistics is a really great general addition to a programmers toolkit. Machine Learning is valuable as well, although in my experience the application is more limited. What this article doesn't mention is a whole host of other skills required.

Modeling, and not just a formal mathematical model, but applying any type of model to your data to get insight. Check out the model-thinking class on coursera.

Exploratory Data Analysis, much different skill than confirmatory statistics.

Design of Experiments, specialized subfield within statistics.

Logistics, how to setup, maintain, and maximally utilize an efficient distributed cluster and build a pipeline getting your data to the cluster, cleaning it, building it into a model, and then extracting insight and delivering that end value.

Those are a couple of the skills at a high level. At a more nuts and bolts level, Hadoop is the defacto standard for Big Data. Learning how to build a data pipeline out of the Linux tool chain is very common in the data science world.

The overall value stream for Big Data is deep and wide. Most companies don't have expertise in much of these, and so at the current time you have to learn them yourself or find a company focused on building a team around it.

If you are just learning this yourself, you'll probably get an academic knowledge. If you want to make yourself valuable in the marketplace, you'll really want to get hands on experience. Knowing a z-score is one thing, building a process to gather data and compute a model against it is a whole different ball game. As the article mentions, if you have nice clean data it's easy to apply a model. If you have messy ugly data from 20 different vendors and 200 clients with various failures, anomalies, and you have to figure out what type of model is helpful, oh and you have a deadline because for 500th time someone promised something impossible to the client, then you have something closer to what Big Data is today.

* grammar edits

<comment><USER12> _deadline because for 500th time someone promised something impossible to the client_

This is a killer in machine learning applications. The toolsets rarely cover the entire extent of what needs to be done, so at least some custom code needs to be written. But results aren't deterministic - you don't really know if it's going to work until you run it. Several iterations are often needed to get to the first useable results. It has all the problems of building any piece of software, plus another layer of risk that the accuracy just won't be there with the first thing(s) you try.

My point is... actually agreeing to be the machine learning guy on a project totally sucks because time estimates are almost meaningless, and the modern business culture is to label anything late as a failure.

<comment><USER13> The company I used to work for had a performance based product. They only got payed if they actually showed improved accuracy against a given evaluation set. Then they got a fraction of the cost savings (say 1 year's worth).

This seems like it could be a good model for machine learning consulting, and one that I would certainly be willing to explore.

It would work something like this :

      1) You show me your problem and your data.

      2) We  come to an agreement on how accuracy would   translate

into financial results and on a fair split of the savings or earnings.

      3) I develop a model.

      4) You evaluate it based on 2.

      5) I get payed based on 2.

If my model doesn't meet minimum performance criteria I don't get payed. If it does very well, and assuming the problem was economically interesting in the first place, you save a lot of money and I get a fair sized chunk of it.

Feel free to explain why this business model wouldn't work.

Edited for formatting.

<comment><USER11> Most business people aren't interested in model accuracy as a term. They want something that provides benefit, e.g. cost savings, increased revenue, increased profits, etc.

The sales process of convincing someone they need an accurate model is tough, especially because robust models are time consuming and expensive to build.

If you can come up with a model that shows good results, and people know they need those results, then you can start a company selling either a service or product to get those results. If people don't know they need your results - then you have to educate them, in which case it's a much more difficult business to start.

I don't know many business people with the temperament, understanding, or the pocket book to deal with general research type problems.

</comment></comment></comment></comment></comment><comment><USER14> Managers frequently wail about skill shortages, but very often it's pure hypocrisy. The real problem is the reluctance to do any training (and I don't mean formal training) combined with the desire to get _proven experts_ in whatever field. Proven experts must have years of experience in applying their expertise. If _no one_ lets people with less experience to work in that field, where the hell would those experts appear from? Another dimension?

Can I be a 80% developer and 20% "data scientist" in your company to try the new role out? The bigger your company is, the less likely the answer to be a "yes". Since Big Data implies a big company, the resulting "shortage" is not surprising. It's self-made.

<comment><USER15> True statement. It seems the trend now days is "get a grad degree, foot the bill and time yourself". Many companies I see and work with tend towards that mentality, as opposed to building a base of highly skilled workers on from the inside. It's easier and cheaper for a company to ask if you have a piece of paper, than for them to train you and get you up to speed.

</comment></comment><comment><USER16> How media sees Big Data:

BIG database => BIG machine learning algo => BIG MODELS => PREDICTIONs, Insights => $$$

How it is actually done:

awk -F"\|" '{print $1}' SCRAPED_file_pipe.txt | sort | uniq -c | head -n 10 => $$$

<comment><USER17> Ah, you've used Ab Initio then.

</comment></comment><comment><USER18> This article is nonsense.

Talented developers are talented developers. At Quantcast we used Hadoop in production before it was even called Hadoop and now we process 10PB a day. We forbid our sourcers from using Hadoop as a resume search term because it meant absolutely nothing.

Statisticians who can code are scarce, but companies that know how to use them are scarcer.

</comment><comment><USER19> Actually that is silly -- McKensey should now that there is and will never be a talent shortage. There will only be shortage of talent at a particular wage rate.

If the companies paid newly graduated 'data-scientists' (what other kind of scientists are there? The tea-leaf reading kind?) 200k/year then they would have a lot more. It is pretty simple economics.

<comment><USER20> Companies already do pay close to $200k/year for entry level data scientists.

 _(what other kind of scientists are there? The tea-leaf reading kind?)_

"Data scientist" refers to the guy who can set up a hadoop cluster, do statistics on TBs worth of data, derive useful conclusions and speed it up by tweaking the low level data formats or microoptimizing the calculation.

The issue is rarely paying these guys an extra $20k, it's simply finding them.

Setting up some lasers and a photonic crystal, imaging the output, making a graph in excel or matlab and drawing conclusions is a different skillset. Someone who can do the latter is a scientist who uses data, but he is not a data scientist.

<comment><USER21> How hard can it be, though? Like taking a normal CS person and making them versatile with hadoop and so on? Could it be done for 20K$?

<comment><USER22> _How hard can it be?_

Very hard. You run into all types of candidates who just aren't there yet: people working on research that's irrelevant to real world applications, people who have done data analysis/BI work that brand themselves as "data scientists," those who have the pedigree but cannot process and explore real- world data, those who have good analytical chops but not the distributed or advanced modeling experience, etc.

I've witnessed it first-hand, and it's tough to find the right person.

<comment><USER9> If it is that hard the bar is probably set too high. Most of the skills are learned on the job after all. Most smart PhDs who can program well and have sound knowledge of statistics can learn to do this stuff.

<comment><USER22> Given enough time, anyone smart enough to finish a PhD can acquire a set of skills. :)

But it's more than just solid statistics. We're talking about having enough mathematical fluency to develop models _rigorously_ (not just "oh, we'll minimize MSE!!"), test those models, then implement those models--possibly using a distributed algorithm.

From what I hear, these skills take years to develop. Choosing to groom the wrong person is an extremely costly mistake, so making the choice is difficult.

<comment><USER9> All mathematics consists of rigorous models. But choosing and tweaking a model is more of an art. Most data scientists apply existing models to new data, they do not develop new ones.

I am sure it takes much less than "years" for any smart PhD in applied mathematics to learn most of data analysis tricks. It is not theoretical physics after all.

<comment><USER22> _Most data scientists apply existing models to new data, they do not develop new ones._

I meant "develop" in the software sense. Data scientists use off-the-shelf libraries during initial research, but those libraries usually lack an important feature preventing them from going into production (typically, no support for concurrency).

 _I am sure it takes much less than "years" ... to learn most of data analysis tricks._

I used to be cynical about "data science," too. After four months of working on a data science team, though, I'm a believer.

A data scientist is really a "full-stack data developer." He or she needs the ability to work with advanced models, use them to analyze large amounts of data, and modify those models to work concurrently or in a distributed system if desired (and its often desired). It's more than just "analysis tricks."

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER23> Until the pay is comparable to finance, good luck?

I'd love to work on (arguably) cooler problems, but the combination of lower pay and the constant need to use the "hot new thing" to solve problems doesn't make transitioning look remotely attractive.

Really, the second is the HUGE obstacle: \- You don't know anything about aNNs? Sorry, no job. \- Nobody uses aNNs anymore, SVMs are all that matters. Sorry, come back after you catch up. \- SVMs? Man, we need someone who's got expertise in optimizing RFs and Bayesian Trees. We don't want "black box" machine learning. We need to "understand" the results. Sorry no job. \- Decision trees? GTFO man. We're doing rNNs now. \- I'm pretty impressed with your data mining knowledge, but we're looking for someone with a background in DLMs and GPs. Sorry, no job. \- repeat until vomit/suicide

I kind of wonder about the need for "badass" math skills; I'm not terribly convinced that math wizards are extraordinarily high value relative to people with other types of data analysis skills.

</comment><comment><USER24> It seems the problem is that some companies are looking for person who is expert in setting up scalable systems (Hadoop cluster, storage, high availability, etc.) and that she/he also knows statistics and efficient ways of processing and understanding the data. Good luck with that.

My observation is that requirements like this come from people who did mainly web programing (and actually that was making a lot of money so with money they become influential): assuming that this equivalent of writing both ruby code and javascript code.

Building team is hard and in order to solve "big data" problem you need to build a balanced team.

</comment><comment><USER25> Basically a solution looking for a problem.

They are right, the complexity that big data caters for requires expertise at both technical and business level that would be costly (though may not be at infrastructure level). In the current economy, it looks even more difficult where businesses want to squeeze the maximum out of dollar investment.

IMO, its too early stage for big data solution adoption. However stage could be set for startups who can come up innovative solution that brings the cost level down together with simple and useful easy to grasp solutions.

</comment><comment><USER26> I've done this kind of thing most of my career, including doing it for NASA and Unilever Research. You can't really train an average graduate to do this. You need someone with a pretty highly developed integration between 1:intuitive/creative abilities, 2:mathematical/analytical skills, and 3:engineering/ability to make things happen. Add to that 4:work experience in the real world, and 5:ability to easily understand how things work in a field you delve into for the first time... And there's very few people in the world who can do this. At my previous work place we tried for a whole year to hire someone who would at have at least some of these skills and seems promising to develop the rest on the job. We couldn't find anyone although we interviewed about 30 different people (from about 500 resumes most of them with a PhD in ML from a good university). And this was in central London, UK.

<comment><USER9> But you never tried training them. Sure, no one can do it right off the bat, without prior experience.

</comment></comment><comment> why-el So I was wondering if any fellow HNer is on a quest to be at least comfortable around these problems. Can you share your plans? Currently I am starting with some linear algebra and I have plans to move to statistics then pick up a book on machine learning. I would really use some advice.

<comment><USER27> Andrew Ng's machine learning class on coursera is a very nice, easy introduction to the subject.

<comment> why-el Yes, and I started that before realizing that I need more background knowledge, hence starting with some math. :)

</comment></comment></comment><comment><USER28> Not entirely sure this is true, to be honest. Most of "data science" lies in the work of collecting and cleaning the data to get it into a usable state. A recent story on the"fallacy of the data scientist shortage"[1] goes into more detail, but in reality what we want in this quantity are better data __analysts __. I love the idea of data science as, essentially, viewing statistical analysis from a computer science perspective, but the breathless predictions of a huge shortage seem a little overblown.

[1]: [http://smartdatacollective.com/nraden/48952/fallacy-data- sci...](http://smartdatacollective.com/nraden/48952/fallacy-data-scientist- shortage)

</comment><comment><USER29> Can anyone explain what talent refers to in this context? Is it someone who has learned this stuff, someone who is capable of learning it, or someone who was born with an innate understanding?

</comment><comment><USER30> Looks like the prelude to yet another H1-B buildup.

<comment><USER31> The only way we can really measure "shortage" is via compensation. If that's so, we need more hedge fund managers and surgeons, not grunt data crunchers. There are many problems with guest workers. The richest people in the world get special access to indentured labor. It targets specific industries thus amounting to a subsidy. It helps big business crush small business. The H1-B in particular is a tool to increase outsourcing and keep wages down (wages which are typically earned in the highest cost of living areas in the country at 60 hours a week). H1-B ? No, thank you! On the job training and good wages? Yes, please!

</comment></comment><comment><USER32> I just listened to a lecture on this at BU. Emerging Internet Technologies at IBM or something of that nature. He was basically trying to sell us his product that crawled the internet (mainly a firehose at Twitter) and gathered statistics for advertisers and presented it in pretty graphics.

The main issue they had was developing language recognition. Deciding if a user 'liked', 'loved', 'hated' or was 'neutral' about a product. Another issue that stood out to me had to do with their reliance on the internet. Just because 200 users tweeted that 'this movie is going to suck' does not really represent the overall opinion.

To reiterate, the whole buzz of the lecture was the biggest turn off. He wasn't explaining about how to expand on his product or where to go from here. Just that they had developed a product and we could use it instead of attempting to develop one ourselves.

<comment><USER33> Maybe 200 is too small a sample size, but you can glean and predict stuff with that sort of data. My master's thesis was about predicting box office sales based off twitter data. (coincidentally these guys published a couple months before me: [http://www.fastcompany.com/1604125/twitter-predicts-box- offi...](http://www.fastcompany.com/1604125/twitter-predicts-box-office-sales- better-than-anything-else)) but we had incredibly similar results. It is pretty interesting to see what you can do with that data.

If it makes you feel any better, you can build it yourself, I certainly did. I didn't even use any libraries like NLTK to build my sentiment analysis. Read some research papers and built it from scratch (code wise at least, the ideas used were fairly common). It's a fun challenge. I still work with that code every day and use it in my startup now :)

</comment></comment><comment><USER34> Surprised Ben Rooney did not mention IBM acquisition of Vivisimo the day before (Apri. 25) this article (Apr.26) <http://www-03.ibm.com/press/us/en/pressrelease/37491.wss> "IBM Advances Big Data Analytics with Acquisition of Vivisimo"

</comment><comment><USER35> This is good news.

</comment><comment><USER36> Why don't the people who hire doctors, dentists, and lawyers suffer from the same talent shortage that the people who hire 'big data' computer scientists feel?

Because it's better across the board to start your own startup than work your ass off for a 4% raise at a place which recognizes you as top talent. I'm on the verge of starting a startup myself, removing myself from the people in this list. There is a shortage of talent in computer science, but never in the other disciplines, it may take another 30 years for the suits to have the ability to understand why.

<comment><USER37> I agree with your overall line of thinking.

In your last sentence you wrote: "There is a shortage of talent in computer science, but never in the other disciplines, it may take another 30 years for the suits to have the ability to understand why."

Could you elaborate on this point -- do you feel that there is not a shortage in disciplines such as dentistry and law because many people _are_ willing to work very hard for only 4% raises?

Thank you!

<comment><USER19> You can actually make good money by going into the law or medicine. You have to work and and be skilled of course, but lets be honest that is also required for a start-up.

I can't help but think that ability is because you can't run a law firm without being a lawyer so the boss has some idea of what it means to be a good lawyer and how to treat them.

<comment><USER38> Most lawyers at the top-income end hate their bosses and jobs. Law firm partnership track at large firms is a dog-eat-dog 80-hour week hell. People are only happy when they are the few who claw to the top, or the many who drop out. The ones in the middle are suffering as bad as any stereotypical bank programmer.</comment></comment></comment></comment>


Ask HN: Recommended stories (real or fiction) to motivate programming newbies? - <AUTHOR> <comment><USER2> [https://youtu.be/02h74L1PmaU](https://youtu.be/02h74L1PmaU)

<comment><AUTHOR> My mother had one of these knitting machines, I think was a Brother one, as a child it felt like a space ship dashboard :D Thanks for the link!

</comment></comment><comment><USER3> Check out nocsdegree.com</comment>


Apple, It's Time To Block iOS 5 Beta Users From Reviewing Apps - <AUTHOR> http://techcrunch.com/2011/08/17/apple-ios-5-block-app-store-reviews/ <comment><USER3> They should also ban reviews from jailbroken devices, where users may have installed custom fonts or other "features" for which you can't possibly test and account.

And yes -- any test Apple does for this _could_ be defeated, but I expect most jailbreak developers would side with their app writing comrades and let it stand.

<comment><USER4> The reason that Cydia doesn't have reviews (or even ratings) is that we haven't found a way to make a working review system either, so we don't even try. I don't think that any of us would argue or work against something like that, but I also seriously doubt that Apple will do it.

(I'm a contributor to the Cydia open source project, but I don't speak for saurik or Cydia, and that's probably not the official stance.)

<comment><USER5> So, I /am/ saurik, and I probably /would/ argue with and work around something like that, as I think I should be allowed to review things in the App Store.

Really, the "official stance" I have is that developers who believe that having their application fail horribly when the platform is even slightly different is not their fault are deluded: Apple is going to change the platform over time, and when your app fails for Apple they tend to get pissed and just block your app from the store and may even de-prioritize your updates.

Seriously: if your app fails because the font is different, there is something epically wrong with your coding methodology; and yes: Apple actually did change the font slightly for the iPhone 4 (on retina screens they switched to Helvetica Neue from Helvetica).

Meanwhile, in desktop-land, everyone from large companies to small independent developers deal with this supposedly horrible problem of random fonts, different operating system versions, upgraded libraries, client-side firewalls, or any other number of things that would make developers like kogir freak out, and somehow the world doesn't end, and almost all of the software I download works without issue (with the exception of Chrome on Tablet editions of Windows, and that's squarely Google's fault). (Hell: even almost all of the software on the App Store works under these conditions.

What should /really/ happen is that Apple should internally have a different "test version" of iOS that developers are not allowed to see; one that has a newer version of SQLite (which I state in specific as developers make all kinds of incorrect assumptions about the SQLite ABI) and has every core library compiled to the opposite instruction set (Thumb if normally ARM, and vice versa), that has every font replaced with Comic Sans, and uses a different pixel density, on a screen with a slightly different aspect ratio, using a CPU that runs slightly faster than the shipping devices.

The reason this needs to be done is that an app that fails to run correctly in that (hidden) configuration is pretty much guaranteed to be an app that is going to stop working when any of the following occur: iOS 6, iPhone Nano, iPad+Retina, or another "game changing" device (such as the iPad when it came out). Rather than let these apps in the store just to fail later, they should be weeded out before they even get in.

(And yes: Cydia itself strives not to make any assumptions about any of the things listed, and in fact worked almost entirely correctly when loaded for the first time on an iPad with the flag flipped for "don't show me in a 1x/2x box"; certainly well enough that the reviewer in this scenario would have given it a pass were they testing it in that environment. The one place it tends to get burnt is with what it does to WebKit; and the current implementation, which works fine on everything from 2.0 through 5.0, is likely stable enough to "stand the test of time".)

(FWIW: I do believe that people using beta versions of iOS should not be able to review applications. These people are using a version of the firmware that is under NDA... they should not be tempted to be providing feedback that might violate that agreement to applications that fail to operate correctly, possibly due to no fault of the app. I have a good example of this related to Cydia on 5.x, but I can't tell you what it is, for this very reason ;P.)

<comment><USER6> Um.

One of the main reasons beta versions of an OS are made available is to find bugs _in the OS_. Apps can crash because the OS isn't doing what it's supposed to. Which is why we're up to iOS Beta 5. [edit: didn't see your FWIW - <USER2> that an edit?]

I'm afraid this reads like a hacker rant who is angry that lesser mortals get to write code too. I'm a bit disappointed this comes from someone as influential as yourself.

[edit] It is a great luxury to have a standardised platform to write for and it cuts development costs. Any Android devs want to comment on their experience of writing code for diverse hardware and setups vs iOS? Or indeed Android users wanting to comment on the quality of software being written?

<comment><USER5> A) Yes, my FWIW was an edit. B) I do not see how anything you have said here responded to any of my arguments. (edit: in specific, I don't even know what your one comment about finding bugs in the OS has anything to do with this conversation) C) My friends who are developers on Android claim that supporting multiple devices is not actually a serious issue.

[edit] D) Further, I believe that a response like mine is the only serious response that can be given to someone (kogir) who actually just suggested, "with a straight face", that a full 10% of the people using the App Store should be /banned from leaving reviews/ because he doesn't want to deal with /fonts being changed/. Are you telling me here that you actually agree with him?

<comment><USER7> _"Further, I believe that a response like mine is the only serious response that can be given to someone (kogir) who actually just suggested, "with a straight face", that a full 10% of the people using the App Store should be /banned from leaving reviews/ because he doesn't want to deal with /fonts being changed/. Are you telling me here that you actually agree with him?"_

Considering that that "full 10%" of the people you speak of have broken their terms of service and are running a non-sanctioned OS, why should Apple be obligated to publish their reviews? I understand the anger at Apple bricking jail-broken phones, but I really don't see how they should be obligated to accommodate them.

</comment></comment></comment></comment></comment></comment><comment><USER8> Apple should remove the reviews section from the App Store for people running pre-release versions of iOS and replace it with a form to send a bug report to the app developer (and possibly even a place for a list of known issues that the app has on iOS 5).

<comment><USER9> Apple should just add a bug reports section to the app store period.

<comment><USER10> Potentially, but if so, it should be something which you'd have to opt into or open only for fellow developers.

The last thing I want is to get spammed with "bug reports" from lay users with trivial or non-existent issues.

If they implemented this but made it take even one extra step like flipping some switch in the settings, it would filter out most lay users and garner some more useful feedback.

</comment></comment></comment><comment><USER11> This reminds me of how PyPI ratings got disabled after a long discussion with the community, started by jacobian:

<http://code.activestate.com/lists/python-catalog-sig/3482/>

I strongly agree that the PyPI ratings were mostly useless and that removing the feature was a good decision.

iPhone app ratings can be much more useful for popular apps, but for the long tail of apps, they suffer the same problems as PyPI's rating and review system did.

Amazon and Yelp ratings, on the other hand, are quite useful. I think it's partly because users have profiles and are better able to express themselves.

<comment><USER12> I think the app ratings for the popular apps are all useless (they're well know apps, other people rave about them, so no need to read all the reviews). I agree on the long tail being useless as well.

Instead, I think it's most useful for the "middle tail": those apps that do get a fair bit of usage, but are not considered popular: the developer may not have build up a reputation yet, but the reviews help confirming the application does what it is supposed to do (or not of course).

</comment></comment><comment><USER13> An alternative fix would be to have the iOS version used when submitting a review. It doesn't limit "free speech", but it gives context on what the problem actually was.

<comment><USER14> I think this is a great idea. Apple could allow filtering to see _IF_ I really want to upgrade to iOS 5 right away or wait until iOS 5 reviewers are happy.

</comment></comment><comment><USER15> I pointed this out the last time this came up, but an additional measure would be to also crack down on sites that sell access to iOS betas, if they aren't doing that already. That would (likely) drastically reduce the number of non- developers running beta OSes since isn't it $100/year for a developer account, but you can install betas on many devices? There are sites selling access for less than $10. If it's not against the TOS of the developer accounts, it definitely should be.

<comment><USER16> Apparently Apple has noticed this issue and they have taken down a few sites. ([http://appadvice.com/appnn/2011/08/apple-takes-steps- against...](http://appadvice.com/appnn/2011/08/apple-takes-steps-against- developers-selling-ios-5-access-what-you-need-to-know)) In fact Apple can try to increase the technical barrier to install iOS beta. (Must upgrade in Xcode or must own at least one app in App Store to be eligible for beta access.) This way can block the vast majority of non-developers while not affecting real iOS developers.

<comment><USER17> There was speculation that that might have just been betas expiring, and not developers selling access getting their access revoked ([http://www.macrumors.com/2011/08/05/apple-disabling- ios-5-be...](http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-beta- installs-and-developer-accounts-over-sales-of-device-slots/)).

> must own at least one app in App Store to be eligible for beta access

That doesn't work for many reasons. App Review, as if they didn't have enough of a hard time with a bad reputation, would just be flooded by trash from first time developers or contract developers/companies that just created an account. Apple could make it mandatory to install using Xcode, but in the past, Xcode upgrades have bricked devices where iTunes doesn't. So I have a feeling that's not really a priority and more of a burden. Plus, it's braindead simple to install and use Xcode to install betas, and anyone desperate enough to pay $10 but not $100 should easily be able to get around that.

I really think all Apple needs to do is to block reviews from iOS 5 devices. Developers running betas won't be reviewing apps in the store (that's incredibly hypocritical and cruel if so) so nobody's going to be missing that. This doesn't harm beta testers and devs that want to install iOS betas.

<comment><USER18> I'd also block reviews from beta versions of iTunes, since many reviews are entered via iTunes.

</comment></comment></comment></comment><comment><USER19> Maybe Apple doesn't block beta reviews because it incentivizes developers to fix their apps before the release? Most of those problems should be trivial to at least work around with version-dependent logic to minimizes regressions on versions < 5.x.

Bad reviews don't exactly hurt Apple, and it's not like iOS developers are scarce these days, so it might just be Apple having their priorities set straight (users > devs) and not a coding blunder..or maybe it's just me overthinking it.

<comment> X-Istence Except that developers are not even given the chance to fix issues before Johnny Public is allowed to start rating apps. The whole point of the beta is to ALLOW the developer to test their apps and find the bugs, as well as to iron out the bugs in the core OS. What if it is an issue with a core API that is now returning something that it shouldn't and it wasn't caught by Apple in testing, then the developer is getting blamed for something they can't even fix.

You have to be a part of the developer program ($99) to get pre-release software, and as part of that you go with the understanding that it isn't meant for everyday use.

Johnny Public shouldn't be running the beta in the first place. If by the time iOS 5 is out the developer still hasn't fixed the bugs in their app, THEN it is perfectly fine to start leaving bad reviews as the developer has had plenty of time to fix their app, but during the beta period, no.

<comment><USER20> It is additionally time consuming building work arounds for issues that might be altered by the time the OS makes it to gold master. Beta is beta, it might mean the OS does not work with your app not your app does not work in the OS.

</comment></comment></comment><comment><USER21> This is just obvious. Isn't pointing out a bug thats due to an unreleased OS breaking your NDA anyway?

<comment><USER22> Yeah, but non-developers who manage to install iOS 5 probably haven't actually signed the NDA.

The funny thing is as a developer you're not even allowed to mention in your description that something in your app is broken on iOS 5, since _that_ would be breaking the NDA.

<comment><USER23> Apple also doesn't like it when you fix an issue, release an update and mention that you fixed a bug which affected iOS 5 compatibility. I needed to change it say "compatibility fixes for future iOS versions".

</comment></comment></comment><comment><USER24> There are reasons to block the iOS5 users comment or review the applications.

1\. iOS5 is beta , so chances are there that inclusion of new APIs , pre release iOS version can adversely affect the already approved application. But as the problem is not with the Application but with the Operating System , the review can be biased or completely irrelevant.

2\. As a Developer I will welcome the step by Apple that, this will allow users to comment and review apps from the users who run most stable Operating system, so any bug on my application is because of me, not because of the iOS5.

</comment><comment><USER25> All the effort developers are putting into complaining about people on iOS 5 Beta leaving negative reviews could better be spent testing and fixing their app for iOS 5.

Are you going to complain when iOS 5 goes live and your app still isn't ready because you stayed up until 2am writing that blog post rant instead of ironing out that last bug?

Even if Apple blocks iOS 5 devices from leaving reviews, you can still leave a review through iTunes.</comment>


China: The Rapid Rise of a Research Nation - <AUTHOR> http://www.nature.com/nature/journal/v528/n7582_supp_ni/full/528S170a.html <comment><USER2> I believe the Chinese culture is the antithesis of innovation, they are masters at following procedures, not at breaking the mold.

I had to train Chinese people myself in China, and I don't know if you can innovate in the sciences without first innovating in the political arena. You have to change the mind of most Chinese and this takes generations.

I mean, if someone is intelligent enough to solve very hard problems, it is intelligent enough to know they first have to remove the chumps in power who makes bad decisions. There are lots of chumps in power in China.

Chinese educated in Canada, Europe or USA could improve the research done in China, but this people is equally dangerous to people in power.

When I was there, it was the beginning of the Chinese bubble, everybody was happy because everything, while bad, was improving. When times get tough, we will see.

<comment><USER3> There are always some free thinkers that make it through any repressive system, if just by accident. China has a lot of talent, so we just need a bit of chance that someone creative can get through.

You will find pockets of creativity everywhere in China.

Chinese culture does emphasize group innovation more than individual innovation/invention. This leads to very different results, taking the edge/risk off many ideas (as any consensus driving process would). But it isn't really limited by that, you can always get away with more as long as no one is watching closely.

<comment><USER4> > you can always get away with more as long as no one is watching closely

In China everyone's being watched and listened to. Foreigners like me are being watched closely, and locals are watching each other far more than occurs in Western countries (like Australia). Even just the general awareness that people are likely to be watching tends to modify behavior.

And of course it's easy for people to watch and listen nowadays. Modern phone/internet technology is being actively used. Bugs don't even need to be inside someone's apartment nowadays for others to listen in. Cameras can be so small it's virtually impossible to find them.

But some people in the West are just as involved in surveillance as occurs in China, particular people in business who protect their cash flows and power structures.

</comment></comment></comment><comment><USER5> We had visiting scholars from some of China's best institutions, they didn't know what a Fourier transform was. They couldn't write MATLAB because they always had somebody else do it for them.

I would like to see these claims motivated by statistics from the Nature Publishing Group's journals.

<comment><USER6> This makes no sense.

I have an American degree and I've heard the phrase "Fourier transform" before, but I couldn't have possibly told you anything about it until I googled it just now. I wouldn't have expected any random Chinese academic to know either, no matter how good their institution was. I write software in a variety of languages every day at my full-time job, and while I've seen MATLAB code and could certainly teach myself to write it, I wouldn't bother if I could just have somebody else do it. Why would you expect some arbitrary group of Chinese scholars to share very specific skills and knowledge? If you gathered up a gaggle of researchers from anywhere else, you'd have to get pretty lucky to land a group where everybody was familiar with both one specific area of mathematics and one specific programming language.

<comment><USER7> Maybe take the charitable view that these visiting academics were in a field where this is surprising and expected knowledge. I don't get the feeling these were English majors.

</comment></comment></comment><comment><USER8> Good news for humanity as having more of China's resources devoted to research can only increase the pace of innovation worldwide.

<comment><USER9> Yeah, this "us-vs-them" tribal nonsense is seriously pointless in the realm of scientific research. I want to have stuff made of futuristic new materials due to innovation in chemistry and live to 120 due to innovation in biology and medicine. I couldn't give two shits whether the papers leading to these advancements were published by Chinese or US scientists.

<comment><USER10> Then we probably need to find a better method than trademarks and copyrights to allow companies to make money so the entire world can actually enjoy the new discoveries.

</comment></comment></comment><comment><USER11> The Chinese paper mills take a shotgun approach. Publish as many papers as possible, with little care for quality.

I saw one Chinese professor who published more papers in a year than the number of days. When you spend that much time submitting papers, nothing is getting a second look.</comment>


Newly discovered Mayan text says 2012 is the end of the calendar not the world - <AUTHOR> http://www.slashgear.com/newly-discovered-mayan-text-says-2012-is-the-end-of-the-calendar-not-the-world-29236443/ <comment><USER2> This isn't new, this has been known for a very long time.</comment>


'The Joe Rogan Experience' launches exclusive partnership with Spotify - <AUTHOR> https://newsroom.spotify.com/2020-05-19/the-joe-rogan-experience-launches-exclusive-partnership-with-spotify/ <comment><USER2> Comments moved to [https://news.ycombinator.com/item?id=23239304](https://news.ycombinator.com/item?id=23239304), which is currently on the front page.</comment>


Journal of Universal Rejection (JofUR) - <AUTHOR> http://www.universalrejection.org/

<comment><USER2> JofUR reminds me of this scene from Big Fish. What would you do if you knew what the outcome was...

Senior Ed Bloom: People needn't worry so much. It's not my time yet. This is not how I go.

Will Bloom: Really?

Senior Ed Bloom: Truly. I saw it in the eye.

Will Bloom: The old lady by the swamp?

Senior Ed Bloom: She was a _witch_.

Will Bloom: No, she was old and probably senile.

Senior Ed Bloom: I saw my death in that eye, and this isn't how it happens.

Will Bloom: So how does it happen?

Senior Ed Bloom: Surprise ending. Wouldn't want to ruin it for you.

-From Big Fish, the best movie about fathers and sons you might never have seen

[http://www.imdb.com/title/tt0319061/quotes](http://www.imdb.com/title/tt0319061/quotes)

</comment><comment><USER3> The /dev/null of academia?</comment>


Ask HN: What do you wish someone made so you could buy it? - <AUTHOR> And how much would you pay for it? <comment><USER2> A PlayStation 1/2 Mini, with HDMI, SD Card support and Wireless controllers. I'd pay anything for it. (No emulators, just a hardware clone in a small size with those features.)

<comment><USER3> I'd pay money for something that let me play PS1/PS2 games during the commute.

</comment></comment><comment><USER4> A usable jet pack would be pretty cool. Or a flying car ala The Jetson's.

</comment><comment><USER5> Pebble Time Steel 2</comment>


Why Men Work So Many Hours - <AUTHOR> https://hbr.org/2013/05/why-men-work-so-many-hours <comment><USER2> One reason for long work hours may be the proliferation of bullshit work. If you are working in bullshit position (stategic coordinator for the east coast and such) you can't be judged based on your results. By definition, there are no results. Thus, you signal your usefuleness/status by working long hours.</comment>


Ask HN: What do you guys think of SAP? - <AUTHOR>

Inspite of being one of the most widely deployed platforms out there and a lot of enterprise software being written for it , you hardly get to see anything about it on HN. <comment><USER2> It's a private technology stack. People here want to be able to see the source code, if not then what're we going to talk about? No sales guys here. Best Regards,

</comment><comment><USER3> I guess there must be a lot of opportunities in the SAP ecosystem (edit: or replacing said ecosystem), but I haven't seen it myself and it's VERY closed, there isn't even good reference material out there (and official training is quite expensive).

Though I just Googled and it doesn't seem to be that bade anymore, I found they lauched openSAP

[https://open.sap.com/](https://open.sap.com/)

though it doesn't seem to have any useful courses for beginners (I guess they don't want to kill their partners' cash cow)

and an Udemy course

[https://www.udemy.com/learn-sap/](https://www.udemy.com/learn-sap/)

In my country learning SAP / ABAP is seen as a fast track to a good salary (if you can find one of the limited job opportunities). I've been passed over for a good opportunity for not knowing SAP basics.

</comment><comment><USER4> Old and stale. A system build to trap big companies with deep pockets. It's a super efficient business model and the system works but the vision behind the platform is too much 1985, even today.

The way companies around the globe are making and managing business has changed. Even big companies are (slowly) changing.

I believe time is ready for something new.

</comment><comment><USER5> There are some new things like HANA implemented and developed, but when I worked with BOBJ it was package of several different apps that was acquired.</comment>


Verizon fined $1.3M for supercookie header injection [pdf] - <AUTHOR> https://transition.fcc.gov/Daily_Releases/Daily_Business/2016/db0307/DA-16-242A1.pdf <comment><USER2> 131b revenue in 2015. So, at 1.3m, a 0.001% fine, aka _half a minute of revenue._

Interestingly it doesn't _directly_ fix the problem either (although it wrecks the current free-profit model, yay!), _" To settle this matter, Verizon Wireless will pay a fine of $1,350,000 and implement a compliance plan that requires it to obtain customer opt-in consent prior to sharing a customer's UIDH with a third party to deliver targeted advertising"_

But lest anyone think this is a UIDH prohibition, the next line goes on to say customers must at least have the ability to opt-out from _internal_ Verizon usage, meaning the UIDH will be there (unless the customer opts out) and that a persistent, unique identifier that follows the user wherever they go is permitted. This ruling is primarily about Verizon _sharing_ the targeting information: Verizon is still permitted a persistent attack on their users, but they are now only permitted to sell customer data on an opt-in basis. Ad- networks will have to do their own tracking themselves for everyone else.

Hopefully Verizon's profits from this schtick are shorn from this shift, to a degree where they give up this _disgraceful corpoate panopticon_ they've been going to the bank on.

<comment><USER3> I'm not interested in defending Verizon here, but your math is wrong. Comparing fines (or taxes, the other popular variation of this error) to revenue is meaningless. You need to compare them to profit. The amount of money that they handled but didn't benefit from is irrelevant.

Verizon's profits for 2015 were $4.22bn. That makes it a 0.03% fine.

But that's still not close enough, because this infraction was in "Verizon Wireless", not the entire company. So to really get a sense of its relevance, you need to figure out what the profit of that arm of the business was.

I can't find that number anywhere. Anybody got a hint?

<comment><USER4> I think it's time we tie criminal/civil fines/fees to income. I'll even accept net.

All I know is when I'm fined, there's a possibility that it might be the last straw. The last straw to homelessness.

When a rich man, or remotely successful corporation is fined it's just someting to talk about.

It seems to violate one of those admendments that people seem to forget about --the 8th? It doesn't matter because nothing will change.

<comment><USER3> That's not unreasonable in principle, but then it gets more complicated: do you propose to hold the entire company accountable for the actions of a small part of it? If not, then it devolves into the kind of accounting complexity that results in random internet outrage because it's beyond the understanding of most people.

If you do propose to hold the entire company accountable for any action taken in its name, then consider what you are enforcing: this would mean companies would be immediately obliged to disempower their entire staff from making decisions at any level, and require review and approval for all actions, to make sure that nobody ever makes a mistake that could be punitively expensive.

Neither of these is going to turn out to be a simple solution to a complex problem.

Worse, you can't even use a simple rule here, because what do you do about companies that aren't making a profit? Do they effectively have carte blanche to violate the law in order to improve their situation? That's probably not what you want, so you'd end up with some complicated mix of both systems.

<comment><USER5> _That 's not unreasonable in principle, but then it gets more complicated: do you propose to hold the entire company accountable for the actions of a small part of it?_

Of course.

 _this would mean companies would be immediately obliged to disempower their entire staff_

No.

Their entire staff is _already_ "disempowered" to make decisions that could put the company in legal trouble. Also this is intended, not merely reckless. Do you really believe this was some nobody's idea? Come on!

Please, someone with real legal knowledge could you explain why this is not like Volkswagen.

I suspect that privacy violations are not quantified or else a "class action" would dry any and all the profits.

<comment><USER3> There's a big difference between "not authorised" (the current reality) and "disempowered". You are not authorised to send emails that place the company in legal jeopardy. You are disempowered from doing so if every email that you send has to be reviewed by a company officer first. The norm today is that you are trusted to not exceed the limits of your authority.

How about every line of code that you write being reviewed by legal to make sure it was within the bounds of the law?

There's plenty of scope here for a far more defensive position on ensuring compliance. That is what you would expect and desire from any attempt to massively increase the liability of errors, no?

<comment><USER5> So were you talking about sending emails?

OK then.

I don't know why I thought it was about a deal of hundred of thousands or maybe millions of dollars affecting customers' privacy :->

<comment><USER6> What if the deal was over email?

To be serious, I think you're making an arbitrary destinction.

<comment><USER5> Please check your sarcasm dectector, it might be malfunctioning.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> VerizonWireless does all sorts of _ahem_ questionable things to the network traffic passing through it, particularly unencrypted traffic like plain HTTP. If you're concerned about image quality, one of their more insidious but unnoticed intrusions is their on-the-fly recompression and/or resizing of images.

Always using a VPN (or SSH tunnel) solves most of the problems.

      $ ssh email@example.com -4ND 127.0.0.1:1080

But you'll need to make sure ppp(8) ignores the HLDC errors they inject into long standing sessions. It will work if your settings and chat script are correct.

Lastly, check your contract; you might be one of the lucky ones who have the clause stating VPN traffic is not counted towards your bandwidth cap and/or rate limit.

<comment><USER8> T-Mobile got called out pretty hard when it was discovered they were downscaling videos [0] under the guise of their "Binge On" program. But hidden tracking cookies are far more disturbing that (noticeably) downscaled images/videos. Ultimately, the people are the consumers of images/videos. Who knows who the consumers of that tracking data might be... Obviously advertisers, but who else? The NSA?

[0] [https://www.eff.org/deeplinks/2016/01/eff-confirms-t- mobiles...](https://www.eff.org/deeplinks/2016/01/eff-confirms-t-mobiles- bingeon-optimization-just-throttling-applies)

<comment><USER9> my tmo network service is often a crude joke. 10gb/mo of basically trash service. at least i get 4 bars!

<comment><USER10> Are you in an area where T-Mobile offers band 12 LTE, and does your device's radio support it?

[http://www.spectrumgateway.com/t-mobile-700a-spectrum](http://www.spectrumgateway.com/t-mobile-700a-spectrum)

Band 12 is rolling out pretty quickly and is spectrum that allegedly helps with a lot of issues caused by buildings / dense urban environments / long range / etc. I can't confirm personally since I don't own a device that supports LTE band 12 (purchased before TMo bought this spectrum....) but reports seem to be very positive.

</comment></comment></comment></comment><comment><USER11> This is a big victory:

> 16\. Termination of Investigation. In express reliance on the covenants and > representations in this Consent Decree and to avoid further expenditure of > public resources, the Bureau agrees to terminate the Investigation. In > consideration for the termination of the Investigation, Verizon Wireless > agrees to the terms, conditions, and procedures contained herein.

Verizon has agreed to pay $1.35M and will likely notify the FTC by mail if it makes a change. It has agreed to abide by the law. If you put this in perspective, this is way more than a slap on the wrist. If we assume a gb costs ~$10 and an average user uses ~6gb then:

($1,350,000 fine / $10/gb) / (6gb/user * 12months) = 1875

This is almost very nearly 1900 people! A huge number. Obviously this is back of a napkin, and the actual size of headers is pretty negligible so there isn't any sense in backing that out of the calculation, because the users already paid for the bandwith.

Plus, verizon is _literally_ the only company out of hundreds of providers doing this. Surely between the weight of this fine and the competition the company will go bankrupt soon.

Big win! Say what you want about the FTC but they closed down the investigation saving an untold number to the US tax payer, Verizon is forced to break the bank, and the response time was rapid, 4 years open shut.

The FTC has been super sharp on policing the industry, by allowing the Governement to subsidize huge swathes of infrastructure costs and selling a finite amount of bandwith, they have been able to keep companies on their toes, not allowing any one company to own telephone, wireless, and internet capabilities.

I hope they can keep this up because Verizon is the only bad actor in the entire space, so it is pretty much all taken care of now.

<comment><USER12> There is a thing in the world called "petty injustice". It's when you get screwed a little bit. Entities like Verizon make a great deal of money on _aggregate petty injustice_. There is no legal recourse: things like this FTC investigation are (as you intimate) like the buzzing of a fly. A minor cost center (mostly legal fees). And even those minor bites are used to attack regulation as "bad for business". Class-action lawsuits are forbidden by virtually all contracts. The only real recourse is to switch carriers; however even that is not real because they all play by the same rules.

Computers are really the thing that make _aggregate petty injustice_ a workable business model, because doing any computation millions of times with humans would cost far too much. This is one reason why dealing with the problem is actually a hacker/programmer moral imperative.

The last piece of the puzzle is why the FTC, SEC, etc. are so ineffective. These are the police of big business. Why are the police of individuals so harsh and powerful, but the police of business so weak and ineffective? I think it has to do with the politics of ignorance. There is no political pressure on the FTC to do it's job; it's too far removed from any elected official. No-one is going to pick the next president based on who they appoint as FTC chairman. One of the reasons is that the country is divided on regulation itself, which means that a large fraction of people, even the victims of petty injustice, would prefer that Verizon simply get away with it. These are the same people who would interpret a harsher penalty as an "anti- business" Obama/Democrat move, instead of simple enforcement of the law.

It's the 21st century and I think it's time that we enumerated some new rights in the face of unprecedented assaults on our freedoms. There needs to be the equivalent of a "fiduciary responsibility" for communications companies. People should not be allowed to give away their legal rights (the right to file class-action lawsuits). The justice system needs to be reformed, with technology and simplifying policies, to make it much faster and much much cheaper. (Not quite related to this case, but our personal devices that represent a very real extension of our minds should be absolutely protected from intrusion.)

<comment><USER13> > There is a thing in the world called "petty injustice". It's when you get > screwed a little bit. Entities like Verizon make a great deal of money on > aggregate petty injustice.

That's a very succinct and clear way of putting it. I wonder if there is an essay or other origin for the particular term you are using?

<comment><USER14> "Wartime: Understanding and Behavior in the Second World War" (1989) links "petty injustice" to "chickenshit behavior" \- "the petty harassment of the weak by the strong; open scrimmage for power and authority and prestige; sadism thinly disguised as necessary discipline; a constant 'paying off of old scores'; and insistence on the letter rather than the spirit of ordinances."

</comment></comment></comment></comment><comment><USER15> It took about 10 days for the opt out to work for me.

"Overall, Verizon reported a profit of $4.22 billion" reported by forbes for the 2015 operating year. That is profit, not revenue.

So, 1,300,000 / 4,220,000,000 = .000308 ouch..

<comment><USER16> I just found opt-out instructions[1]. How did you validate that the opt-out worked?

[1] [http://www.clarkhoward.com/how-opt-out-verizons-super- cookie...](http://www.clarkhoward.com/how-opt-out-verizons-super-cookie- tracking)

</comment></comment><comment><USER17> Verizon was fined for _not disclosing_ supercookie injection. They can still do it, but have to allow for opt-out.

Amusingly, they don't do it for "government or enterprise" accounts.

<comment><USER9> I'd like to benefit from collective bargaining with mobile carriers, too.

</comment></comment><comment><USER18> Here is a discussion from a year or two ago that explains the header: [https://news.ycombinator.com/item?id=8500131](https://news.ycombinator.com/item?id=8500131)

</comment><comment><USER19> The punishment should fit the crime. If penalties were tied to some sort of assessment of economic gains from the violations, it might start to make companies weigh their actions a bit more...

</comment><comment><USER20> $1.3m? They won't even notice.

<comment><USER21> They probably paid their lobbyists more to fight against the FCC over this.

</comment></comment><comment><USER22> A lot of threads here are focused on the miniscule fine, but the larger impact is that this is another case study that can be used in the next net neutrality debate. Verizon is giving evidence to the argument that they can't be trusted with network communications, evidence that will surely come back to bite them in the future.

</comment><comment><USER23> Even more reason to use HTTPS for everything... We can't even trust the providers we do pay not to sell data on us.

</comment><comment><USER24> $1.3M is less than their lawyer fees.

</comment><comment><USER25> And there is now going to be yet another misc 8 cent charge on my bill. Where is my 50 cent rebate for going paperless and saving them the stamp?

<comment><USER26> To be fair, most the the surcharges on your bill are government taxes of various kinds.

</comment></comment><comment><USER27> 0.001%.

Is there any reason why Verizon would even bother to comply??

Just wait and see what happens if you get another suit that penalize you 0.01%, then comply.

I'm being sarcastic of course.

</comment><comment><USER28> ELI5 what is/was the super cookie ? What is/was the purpose, and how does/did it work ?

<comment><USER29> When you did an request for a web page on their network, Verizon would silently add an header to your request with an ID tied to your data plan contract. The point was to enable them to sell analytics services that tracked users from their network across the sites of their clients.

<comment><USER30> so even if the service isn't free you're still the product.

<comment><USER31> If they can sell you and get paid by you for the privilege, why wouldn't they? Capitalism, as a good and effective market philosophy, is built on all parties being aware of all aspects of the transaction and ideally of all transactions... a perfect market. If such a thing doesn't exist, which it does not, then it's selective fuckery. However, it tends to exhibit less fuckery than most other systems... except when it doesn't.

<comment><USER30> sounds like "If you're not paying for the product, you are the product. needs to be updated to just "you are the product"

<comment><USER31> well, it's more like, "You may be the product even if you're paying. Who knows because nobody will tell you because they're not required to tell you." But yeah, 'you are the product' is probably the most accurate.

</comment></comment></comment></comment></comment></comment><comment><USER32> With such a dent in their profits I'm sure Verizon will learn their lesson.

</comment><comment><USER33> Despite all the outrage against Verizon, a small part of me feels sorry for them. Having consulted with large corporations in the past, I know that most (all) of them don't generally have expertise in this kind of thing, and usually outsource it to a variety of digital agencies. My guess is that they will be having a very hard conversation with one or more of their vendors.

<comment><USER34> I can not believe such a huge company did not consult lawyers and that the guy who was forced to write the code didn't speak to their managers about potential implications (whether it went up the chain is another matter).

Besides that, ignorantia iuris nocet. Governments don't have problems with fining individuals unaware of arcane and complex laws. Why would we feel sorry for unscrupulous corporations?

<comment><USER33> Agreed on all points, however -

 _the guy who was forced to write the code didn 't speak to their managers about potential implications_

That isn't really the way things work in an agency/Fortune 500 relationship. What _really_ happens is something like:

-> the development manager attempts to translate the implications of said software to an account manager who is directly responsible for client communication -> the account manager has little idea of what the development manager is talking about, which does not stop them from attempting to translate in turn to the client's marketing team -> the one tech representative from the client company listens to the translation in abject horror, knowing that any questions they ask will never be adequately answered -> everyone goes out for drinks on the agency's tab

This isn't a raging critique of the process, it's just what normally happens.

That said, Verizon got fined for a good reason.

<comment><USER35> [http://ogun.stanford.edu/~bnayfeh/plan.html](http://ogun.stanford.edu/~bnayfeh/plan.html)</comment></comment></comment></comment>


Why Amazon, Facebook, Google and Apple Are Bad for America - <AUTHOR> https://www.politico.com/news/agenda/2020/07/28/agenda-amazon-facebook-google-apple-hearing-383612 <comment><USER2> So many unsubstantiated claims...</comment>


Educational Augmented Reality for kids - <AUTHOR> https://www.indiegogo.com/projects/educational-augmented-reality-for-kids#/ <comment><USER2> Hmm, I think the app could just play a video of real world instead of 3D models. And about those fairy tales books, it seem the only needed part is the book cover? :/</comment>


The Invention of Sliced Bread - <AUTHOR> http://priceonomics.com/the-invention-of-sliced-bread/ <comment><USER2> Just to add some perspective from the outside world (I.e. Not America). In Italy sliced bread never really took over common bread. Yes we make toasts, and yes, we have sliced bread, but that's usually some industrial, over- processed bread loaf that you'll never find fresh at a bakery.

Here in Germany, were I live now, it's a mix. Common bread everywhere, but you can slice your own loaf at the supermarket with a terrific automated spring- loaded circular saw machine that's very cool to operate.

I just want to point out that "the best thing since sliced bread" is a sentence culturally linked to the U.S. conception of bread. I point that out because I had conversation with American friends in the past who were surprised by the non-universality of this sentence (as often happens with other cultural references).

<comment><USER3> French here, sliced bread is like "the worst thing ever".And dont get me started on industrial bread.If your bread isnt like less than 30 min fresh,hot from the oven, you shouldnt eat it.And by the way,we call sliced bread "english bread".

<comment><USER4> Dutch-near-German-border here. I think your opinion stems from the quality of industrial bread in France. If you look in the Netherlands you'll find a wide variety of very tasty pre-sliced industrial bread, always fresh baked in the morning.

That being said, I always buy unsliced artisan German bread from my local baker who imports it from across the border.

And for fun, let's fan the flames a bit. You can't compare the simple french baguette to the rich flavour of artisan German bread, simply no contest ;)

<comment><USER5> As a Frenchman who has lived 7 years in France, I can't agree with you.

Is the Dutch industrial sliced-bread better than the French sliced-bread ? Yeah, mostly because it's "freshly" baked every morning.

But it is still industrial bread, which can't be beaten by bread fresh from the bakery.

As for german bread, they do have a wide variety, I give you that, but once again, it's more and more industrial.

My parents in law, who are germans, complain that more and more bakery only bake pre-made bread and don't do their own anymore.

So yeah, if you have the chance to have a "real" baker nearby, it can be quite gorgeous, but in the general case, not so much.

<comment><USER6> Dutch industrial bread is nice in its own way, but it's not really bread like a baguette is. White Dutch bread has a cake-y character to it, which goes very well together with more 'delicate' toppings like butter & hagelslag (chocolate sprinkles), jam, a young dutch cheese or boiled ham, which are common in the Netherlands. Whole wheat sourdough bread, no matter how great, just doesn't mesh with delicate toppings like strawberry jam. That said, a baguette or artisan bread is a far superior base for a good old cheese, Serrano ham or other 'strong' toppings. This is why brown dutch bread is never worth buying IMO -- it's kind of a weird thing like cake made with whole wheat flour that reminds me of antroposophic foods: it's slightly less unhealthy but it tastes like crap. If you're gonna eat something unhealthy it better taste good.

</comment></comment></comment></comment></comment><comment><USER7> First, that is one of the sloppiest colorized photos I've ever seen.

Now on to the article. Though I had heard bits and pieces of lore behind the invention, this is the first complete history. I'm surprise that he went about building the large-scale automated machine first, rather than evolving it from simpler devices. That the development was delayed and almost lost for good serves as a warning more than his success is an inspiration, I think. But he was persistent which I suppose is the single most important quality an innovator can have.

Not to mention a sense of humor. "Mac-Roh Sales & Manufacturing". And then he ended up selling his business to "Micro-Westco".

I once asked my grand-father "What was the best thing _before_ sliced bread?" Without hesitation he said, "Indoor plumbing."

<comment><USER8> About the photo: I think it looks fairly representative of a colorized photo of that time. Note that this is not a photoshop job - it was colorized in 1920, probably by the photographer.

<comment><USER9> I agree, my grandmother has boxes full of photos that look like this.

Here's another example [https://www.pinterest.com/pin/465911523921633003/](https://www.pinterest.com/pin/465911523921633003/)

</comment></comment></comment><comment><USER10> As someone being born in Spain, form me sliced bread means: "The worse thing ever". An industrialized type of food that also requires to use the kind of ugly plastic that ends in the stomachs of turtles, dolphins and whales.

Now the crustless bread is even worse as they need lost of additives for it to stay fresh.

In China and Japan good bread was in the top list of things I missed, so I bought my own bread machine. But getting good flour was not easy.

In the US I could get it but it was expensive.

People from China, Japan or the US loved to come over my house to eat. Most of them have not tasted good bread ever, but they believe they do because of sliced bread.

</comment><comment><USER11> "Trusting the great potential in his hypothetical invention, he immediately set out to conduct some market research. He put together a brief questionnaire "for the purpose of determining a thickness of slice which would be most nearly universal in acceptance," and placed it as an ad in several large newspapers. Over the course of a few months, he managed to cull responses from 30,000 housewives.

Armed with this knowledge, he embarked on a long, painful journey to bring his invention to life."

Talk about validating your idea!

I found it amazing this guy was doing what we recently discovered, back in early 1900's

</comment><comment><USER12> Here's a parallel read, also interesting

[http://99percentinvisible.org/episode/good- bread/](http://99percentinvisible.org/episode/good-bread/)

</comment><comment><USER13> I've become a convert to home baking with a bread machine, and the loss of easy slicing is one if the biggest downsides. The purchase of a slicing guide and electric knife has offset this somewhat, but it's still easy to get wrong. What I'd like is a compact home bread slicing machine.

<comment><USER14> You might consider just getting a nice long, sharp bread knife. There's something fun about a great knife.

<comment><USER15> Absolutely agreed on the great knife bit. As for the bread... I think ditching the expectation of perfectly even slices is a good start. Cut a piece and cut it in half; there, half a sandwich (assuming your bread maker spits out loaves with a square cross section). Want another half? It may not be the same thickness as the first one, but as long as you're a little bit careful, your pieces should at least be consistent with _themselves_ , so there you go.

</comment></comment></comment><comment><USER9> I'm old enough to remember buying your basic sandwich bread at the grocery store and taking it to the bread slicing machine. I guess the idea was that it would keep the bread better until slicing it, after which it would quickly be turned into all the lunch sandwiches for 3 or 4 kids for the week. I think the presliced bread cost a little more since it had shorter shelf-life or something.

These days, even my local grocery sells fantastic artisinal breads. Which you can either keep in the loaf and cut at home, or take it to a similar machine and have it pre-sliced for you. You can of course get the old basic sandwich bread in cheap loaves...$2-$3. But even the "fancy" breads are usually under $4 a loaf.

</comment><comment><USER16> Interesting to see the early wrapper for Wonder Bread, where it's quite clearly labelled as "Wonder-Cut Bread". Obviously the "-Cut" became superfluous and was dropped over time.

</comment><comment><USER17> related to me by a drunk baker some evening many moons ago -

'sliced bread has more surface area, so goes stale faster; companies loved it because their bean counters found out that people ended up buying more bread for the perceived convenience. By the time they started crazy additives for longer life, people were convinced that sliced bread is better. Throughout, people were eating less and less bread ... as a slice with your meal so its usage has changed quite a bit as well.' As the evening wore on, he probably had a point, but I missed it.

Now I do not know the veracity of the above ... what is clear to me is that (like many things in the western world) bread is no longer bread (that strange sliced white loaf is probably full of sugar, etc) ... having lived in countries for 20 years where I slice my bread I've not noticed the incredibly time consuming act of slicing bread.</comment>


This shouldn't exist - johnx123-up http://googleinternetsecurity.blogspot.com/

<comment><AUTHOR> Anyone can create a fake replica of a google blogger template and host it up on blogger. ^ Scary!</comment>


Viewing Issues on BitBucket Requires Logging In - <AUTHOR> https://bitbucket.org/pypa/setuptools/issues/409/unable-to-build-wheel-from-unittest2 <comment><USER2> I suspect this repository has the issue tracker mode set to "private". If the owner changes it to "public" issues will be anonymously accessible: [http://i.imgur.com/JqCvXNU.png](http://i.imgur.com/JqCvXNU.png)</comment>


Vim key mapping is turing complete - <AUTHOR> https://github.com/divVerent/vi-turing <comment><AUTHOR> Today I woke up and thought to myself, Vim key mappings are basically string search and replace, can I build a Turing Machine out of that?

Then I did a search. Alas, it has already been done.</comment>


We're 10 today - <AUTHOR> https://blog.openstreetmap.org/2014/08/09/10th-birthday/

<comment><USER2> A fun side-by-side comparison of OpenStreetMap in 2007 vs today:

[https://mvexel.github.io/thenandnow/#10/52.2644/5.2899](https://mvexel.github.io/thenandnow/#10/52.2644/5.2899)

<comment><USER3> Besides a few city names, USA was completely missing in 2007?

<comment><USER4> Much of the progress came from importing the TIGER roads data.

I don't know it first hand, but the story is that other maps of the US were in pretty good shape and data was relatively available (either the TIGER data, or from states and counties), so interest wasn't very high.

</comment></comment></comment><comment><USER5> I find OpenStreetMap to be the perfect example that it is very feasible to beat Goggle at one of its core competences.

<comment><USER6> Here's hoping for OpenStreetView and OpenAerials ;)

<comment><USER7> Mapillary [1] is a crowd-sourced Street View-like with a cc-by-sa license, but it's far from being usable for now (not enough pictures).

[1]: [http://www.mapillary.com/](http://www.mapillary.com/)

</comment></comment></comment><comment><USER8> Congrats, I am building a project around OSM, that's a bowl of fresh air compared to other platforms!

Weird, no party in SF or Austin? Finally, the "hypest" tech cities are the ones which are the less involved in OSM, that's a surprise!

<comment><USER9> Suprisingly OSM does not have a lot of users in USA for it's size. All data (incl. map data) from the US government is public domain, so the USA has access to a lot of free map data. But in Europe (specifically the UK), the government map data is heavily closed. Hence OSM was started in the UK.

<comment><AUTHOR> But is that public domain data any good? Mappers in the US are still using MapRoulette to clean up Tiger imports etc.

I think it's also that many Europeans, especially Germans, are very sceptical of google while in the US google gets a lot of love, especially these days with fiber.

<comment><USER9> > _But is that public domain data any good?_

No not really. Some people think the import into OSM of TIGER data has hindered the creation of a US OSM community, because the map looks "good enough"/"looks done", when it in reality it's not.

</comment></comment></comment></comment><comment><USER10> Thanks! This reminded me to look up the area I'm visiting today and see what I could do. Turns out there was a big old lake missing, so signed up and added it. Easy and fun!

</comment><comment><USER11> I think the maps are very good but all Android maps I have tried that use OSM have terrible search interface.

In Google I can type in "IKEA Burbank" and get the right result. I can't do that in any of the OSM apps.

I also need to know in what city a certain street is. I can't do "1234 Foothill, Los Angeles" like in Google but I need to know that this number is in Glendale.

If they fixed this I think adoption would go up a lot.

</comment><comment><USER12> I would like to know about OSM process for separating good contributions from bad ones. Similar non-map projects could benefit from learning how they did and how they do.

<comment><USER13> There isn't really a process at all. The barrier to entry is so high that those dedicated enough to map usually make it through with good intentions.

Mapping is really hard, and even with all the work we (OSM) have done to make it easier, we still get far less than 1 percent of our users to actively contribute.

</comment></comment><comment><USER14> It's not as widespread as Wikipedia yet, but OSM is one of the few community built resources to be used by everyone using computers.

</comment><comment><USER15> I would like to see, some mobile OS bundle OpenSteetMap or a app which becomes a hit on top of OSM. The usablity of any map will be its data, and I feel OSM lacks compared to GMap or Bing map.

<comment><USER16> There are a few power users of OSM. Like, MapQuest and FourSquare. However, either of those have neither the detailed information of Google/Bing nor their popularity, so I see your point.

<comment><USER4> Foursquare doesn't use OSM for places:

[https://developer.foursquare.com/overview/venues](https://developer.foursquare.com/overview/venues)

I guess Google is doing their own thing, Bing is listed as using Factual:

[http://factual.com/](http://factual.com/)

(Along with Yelp, Trulia, GroupOn, others).

Facebook is also in the business of having a landing page for every semi interesting place (I think this is one of the things that drove Google to push Google+ so hard).

</comment></comment></comment><comment><USER17> Congrats and thanks to the developers of openstreetmap!

How do we make openseamaps.org better? OpenRov? Robots solar powered?

</comment><comment><USER18> 10 years! Congrats and thanks to all who made openstreetmaps possible.

</comment><comment><USER19> Congrats!</comment>


Showdown Angels, Arrington to Go Mano a Mano - <AUTHOR> http://www.wired.com/epicenter/2010/09/angels-fight/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29&utm_content=My+Yahoo

<comment><USER2> Of McClure...

"his blog uses font colors like it's 1998 Geocities all over again. He Twitters without an internal editor. And he laces his stage performances with expletives as if he were a pirate."

Should be quite amusing on Monday.</comment>


"They Wanted Me Gone" – Edward Snowden New Book Interview - <AUTHOR> https://www.theguardian.com/us-news/ng-interactive/2019/sep/13/edward-snowden-interview-whistleblowing-russia-ai-permanent-record <comment><USER2> The Guardian and the media all benefited from his actions, something he likely took into account when crafting a plan to establish his celebrity.

Profiting from unprosecuted crimes in exile is despicable in my view.</comment>


Ask HN: How to Measure Software Quality? - <AUTHOR> I cannot find a way to find a good metric for software quality...<p>Number of bugs found by the customers / per month? Any other ideas? What are the quality KPIs in the companies you're working for? <comment><USER2> It's helpful to measure design quality and build quality - not just build quality. Software developers are mostly concerned about build quality. Software development should always proceed from complete and accurate specifications and the specification process is a key design process - not a build process.

Poor design quality increases the chance of poor build quality. That's why it's important to measure both.

As an aside, I've always felt that 'Defect' is a more useful term than 'Bug'. 'Bug' seems to be too open to interpretation (i.e. one persons bug is another persons feature). 'Defect' however, has a useful definition:

'An operation is defective if it doesn't conform to the specification.'

This provides a solid basis for identifying Defects. Once you can formally and accurately establish what a Defect (i.e. Bug) is, you can use the following two simple formulas as KPIs.

1\. Design Quality Change Requests/Requirement Specification (DQ = CR/RS) 2\. Build Quality Defect Reports/Requirement Specification (BQ = DR/RS)

The ideal is zero. In other words, zero Change Requests per Requirement Specification and zero Defect Reports per Requirement Specification. However, I suspect that no project in history has ever achieved that. Nevertheless, it's the quality target to aim for. Once you start treating these values as KPIs you can start to monitor the two numbers over time and steadily work to reduce them, thus improving quality in a measurable way. For instance, you may find that the project you're working on has a DQ of 2.5 and a BQ of 3.6. Your mission (if you choose to accept it) is to steadily increase quality so that those two numbers reduce over time.

By viewing the results across a date rand you can start to see quality trends.

Design Quality is an important metric to monitor because that's often where the weakness lies in software projects. Managers love to offload design to developers in an ad-hoc way, largely because design is hard and time consuming (and no-one likes to write specs).

If you're concerned about build quality, first check the design quality. If you're building a product from poor, incomplete, inaccurate specifications, you're going to find it much harder to achieve decent build quality.

</comment><comment><USER3> Your infrequent reminder:

 _Goodhart's curse: Every metric that can be tracked gets incentivized and ultimately gamed._

Beware what you choose to measure.</comment>