The End of Food? - <AUTHOR> http://www.newyorker.com/reporting/2014/05/12/140512fa_fact_widdicombe <comment><USER3> > One of Silicon Valley's cultural exports in the past ten years has been the > concept of "lifehacking": devising tricks to streamline the obligations of > daily life, thereby freeing yourself up for whatever you'd rather be doing

Well, to be frank, I'd rather be eating. What's the point of your life if you don't have time to even eat, let alone enjoy yourself.

<comment><USER4> I suppose the argument is that you derive some amount of enjoyment from eating due to hunger.

If you were never hungry, would you eat for enjoyment?

Soylent's premise is that you have time to do other fun or profitable things instead of making time for a meal.

<comment><USER5> _If you were never hungry, would you eat for enjoyment?_

Absolutely, I would. The range of sensations available through the consumption food is second only to sex in its breadth and power and sensuality. Anyone who would forego that because it's "inconvenient" utterly baffles me.

<comment><USER6> I think a helpful parallel for this is alcohol consumption. I'm a huge cocktail nerd: I love tinkering with them and admiring them, finding complexity and nuance in slight variances in recipes and ingredients, admiring the deep levels of possibility in preparing a great drink. If anything, the drunkenness is a distraction from the mixology.

And at the same time, there are people who drink only when they want to get drunk -- or who abstain altogether, not liking the bite of alcohol or the consequence it conjures. And that's totally normal.

I think it's more likely than not that at some point in the future, eating food will be the same way -- there will be a pronounced spectrum of interest once it's decoupled from the need for subsistence.

</comment></comment></comment></comment><comment><USER7> Somehow that title, contrasted with the chart ([http://www.nytimes.com/2014/05/01/business/economy/changed-l...](http://www.nytimes.com/2014/05/01/business/economy/changed- life-of-the-poor-squeak-by-and-buy-a-lot.html)) showing how the price of food, especially has gone up in recent years while tech prices have fallen, makes me worry about a future reality where real food is so expensive it's only for the elite, and everyone else subsists on supplements like Soylent.

<comment><USER8> This is part of the plot of "Soylent Green".

[https://en.wikipedia.org/wiki/Soylent_green](https://en.wikipedia.org/wiki/Soylent_green)

<comment><USER7> Right. Disconcerting to see its namesake being a foreshadow of said plot becoming quasi-reality.

</comment></comment></comment><comment><USER9> I find it annoying that whenever Soylent is discussed, people always take such an extreme view of it. Why do so many people frame the discussion with "you will replace all meals with it".

It's similar to stories about electric cars. Someone will always comment about how electric cars won't work for people who have to drive 500 km non stop on a daily basis, therefore they are doomed to failure.

<comment><USER10> Agreed. The stated purpose of Soylent is not to replace all meals. It is to be a convenient replacement for inconvenient meals. But, that idea doesn't grab enough attention to sell ad space. Therefore, pretty much every article I've read about Soylent tries to manufacture controversy around the idea of giving up on food entirely in favor of brown paste.

That said, when my month's supply arrives, I'm looking forward to giving up food for a month entirely in favor of brown paste. I see it as a fun opportunity for an interesting cleanse --not a permanent lifestyle change.

Every report I've read from people trying out a 100% Soylent diet has been consistent: You feel fine. Your jaw gets stiff from underuse (chewing gum is recommended). You poop less and less often. The biggest problem is really social awkwardness. Eating with people is important. It's awkward being the guy at the table with a bottle of brown paste.

<comment><USER11> Do, please, keep us updated on that experiment. I did not even realise that Soylent was an actual product (presumably, not made of people) until I read your comment.

</comment></comment></comment><comment><USER12> I would personally be happy if I had the option to have a convenient liquid meal replacement, but I'm also skeptical Soylent is going to be the solution to it.

There's still too much we don't know about how we get nutrients from food to be able to fully replace fresh unpackaged products with powders. I suspect we're still at least a few decades away from being able to do that safely on the long term.

Nutrition research has been terribly inconclusive for the past century and there's no indication that this is changing anytime soon. It's not the researchers' fault: the domain is incredibly complex when you take in consideration various timespans, all the various human generic permutations etc.

<comment><USER13> The way I see it is that so many people get by on nutrient-low, unbalanced diets that it's impossible to know exactly what the optimal combinations of food to eat really is. Humans are pretty resilient.

I doubt Soylent will be better or worse for you than the diet of a typical HN user if taken in the long term. Time will also tell: if after 5 years no adverse health effects are reported after a sufficient number of heavy users begin drinking it, then that's probably as best a "green light" as one could ever get.

</comment></comment><comment><USER9> _Rhinehart is not a fan of farms, which he refers to as "very inefficient factories." He believes that farming should become more industrialized, not less. "It's really the labor that gets me"_

This seems a little ignorant to me. Farms (at least in Australia) are amazingly industrialized. I visited my cousin's dairy farm recently and one person was working at milking dozens of cows at a time.

I walked away with the image that modern farmers are technical early adopters, looking at ways technology can improve their return on investment.

<comment><USER14> This is definitely true, look at the rise of John Deere's GreenStar - <USER2> controlled tractors and implements which dramatically improve yields. Farmers are like hackers in many ways.

<comment><USER15> I guy I knew years ago used to work for a software company that sold software to farmers, and he said he was always surprised by how incredibly high tech farming and farmers where.

</comment></comment></comment><comment><USER16> I've been making a homemade version of Soylent ([http://www.cookingfor20.com/2013/06/18/hacker-school- soylent...](http://www.cookingfor20.com/2013/06/18/hacker-school-soylent- recipe/)) since September. It makes up one or two of my meals each day. I've got to say, I really really like it. I think it has a good chance of catching on. I still love eating out socially, but it has replaced all my my mediocre meals. I can imagine it replacing those aisles and aisles of junk, easy food (cereal, canned soup, mediocre pasta).

That said, I quickly had to come up with a better name than Soylent. I call mine Science (for human consumption). It makes it distinct from food. "Did you get lunch?" "No, I just had Science today."

</comment><comment><USER17> I hope people do understand that this is short term solution only. Replacing your all food indefinitely with this is most certainly going to damage you in a ways you can't know or imagine. Just by looking at the content I can see that the mix is seriously lacking essential fats, is using very low doses of some vitamins, particularly C and probably contains substitutes to others like retinol (beta carotene).

The idea is great in specific context. People have been living on single food item in multiple occasions so far. CBB to find references for all now, its easy if you need them but there are known cases of

1\. Potato only for 6 months ([http://goo.gl/XqxMx](http://goo.gl/XqxMx))

2\. Bananas only for months (i.e. extreme '30 bananas a day' diet)

3\. Meat only for a year (Vilhjalmur Stefansson)

4\. Water only for > year (in extreme obese case).

5\. Protein powder only (i.e. Last chance diet, case to the point of what could get wrong. )

6\. Synthetics only (I am sure Kurzweill probably experimented with this with his 200++ pills per day).

7\. MC Donalds for a month

... and so on

Its important to note that inventors are young people. Young people are very resilient - they also have full reserves of stem cells that are used to repair damage. Soylent might speed up this process because it certainly doesn't contain 'everything body needs' (because we don't know this at the current technological level) in which case one theory is that body will use its triage system (Ames) to redirect resources to the systems that are most important and shut down or reduce output in less important systems for immediate survival (such as reproduction). More appropriate would be to say 'everything body tolerates for a period'. Its certainly far better move then eating only pasta or rice for entire day.

One may hope that whatever is the eventual damage that could be done to you by use of this food surrogate is going to be fixed by the future medicinal, yet to be discovered, techniques.

<comment><USER18> Also: he ate fastfood for some time, started with 'healthy' Soylent and noticed he felt better. So I think he got the wrong reference. But ofcourse there is also a lot of sweet marketing talk in the article.

</comment></comment><comment><USER19> It's like saying sex toys will eventually replace having sex because why waste time dating, courting and then having sex :) when you could just use a toy + redtube and get it over with in 10 minutes.

<comment><USER20> Watch the Demolition Man " _sex scene_ ": [http://www.youtube.com/watch?v=k80UQWWUIYs](http://www.youtube.com/watch?v=k80UQWWUIYs)

[http://www.imdb.com/title/tt0106697/](http://www.imdb.com/title/tt0106697/)

</comment></comment><comment><USER20> Previous discussions on HN:

* [https://news.ycombinator.com/item?id=5369778](https://news.ycombinator.com/item?id=5369778)

* [https://news.ycombinator.com/item?id=6175079](https://news.ycombinator.com/item?id=6175079)

his blog: [http://robrhinehart.com/](http://robrhinehart.com/)

A short documentary style video: [http://www.youtube.com/watch?v=t8NCigh54jg](http://www.youtube.com/watch?v=t8NCigh54jg)

</comment><comment><USER21> This screams link bait title, similar to "The End of the PC". Who knows what the longterm effects of Soylent are. At best, it's a supplement, and at worst... I don't even want to know. With that said, Soylent has a potential to make a big impact. But you can't scream that this is the end of food just because there is a cheaper and more efficient way of getting your nutrients. Eating food is a staple of our culture and the way we socialize revolves around it.

<comment><USER22> The article addresses your concerns. First - the author lives almost entirely on Soylent for the past year and half. The reporter wrote that he looked and behaved really healthy. Sure, there could be some side effects that will manifest much later, but to this date it seems that there are none. Second - he doesn't believe that Soylent will, or should, replace food entirely. He calls everything else "recreational food" and says that Soylent is a replacement for frozen stuff that you eat because its cheap and gives you calories, not for meals in restaurants that are meant to socialize.

<comment><USER23> > First - the author lives almost entirely on Soylent for the past year and > half.

In other words, a sample size of one.

I haven't an opinion on the safety of the product, but a sample size of one only proves it's not an outright poison. It's likely not actively harmful, but still, nutrition is a complicated thing.

I just think the whole idea of it couldn't be more depressing. People who don't get enjoyment out of food freak me out a bit. There are few things I enjoy more than cooking and eating well prepared food.

<comment><USER24> Whether or not it's a sample size of one, it's the longest soylent trial I've read about so far. You have to start somewhere.

</comment></comment></comment></comment><comment><USER25> I think Soylent is great idea.

It doesn't have to be either/or and certainly the end of food. I love to cook and take a lot of care about what I eat. This probably is never going to change. But on occasion I would love to be able to slurp down a meal and not have to worry about it. I wouldn't consider living on something like this though and I don't know why anyone else would either.

</comment><comment><USER26> _NSFW_

Whenever discussions of Soylent come up, I always wonder - would the same people who want to take Soylent also agree to taking a people that made their sex drive disappear? It will definitely save time and money to not have to worry about sex, after all.

NOTE: This is not a pro- or anti- Soylent sentiment, it's a mind experiment.

<comment><USER27> Food is more expensive and time consuming than sex.

I've never heard anyone say that sex is a chore, but I often feel that way cooking food that I don't want to eat, but have to because I'm hungry.

<comment><USER28> Sex is way more expensive. Most of your life's spending is spent indirectly on sex.

</comment></comment></comment><comment><USER29> Assuming that it doesn't taste as bad as it sounds, it's likely that this will just end being a quick meal replacement here and there. I.e. you're pulling a long night at work and you _could_ order pizza but that's unhealthy so you whip out a Soylent shake.

It's certainly not a revolution by any means

</comment><comment><USER30> I'm a little concerned about the long-term effects. The article states that the founder looks and feels healthy, but since he's the only person living off Soylent for an extended period, this is anecdotal evidence at best. It reminds me of the death of Seth Roberts, a couple of days ago.

<comment><USER31> Many people eat nothing but unhealthy crap for years and still stay relatively healthy. Human body is really good at metabolising even a limited variety of food into many different chemicals it needs. Well-balanced diet hasn't really been a thing in human history until recently. Mix of nutrients designed to our best knowledge can't be any worse than eating pizza/fries/burgers/soda 24/7.

<comment><USER30> You're reasoning that eating fast food isn't all that bad for you, and for that Soylent can't be worse? Because there are countless clinical trials and scientific researches that prove a correlation between fast food and heart disease, cancer et al. All I'm saying is that for a product like Soylent the long term effects haven't been studied yet.

To be clear: I'm not biased against Soylent. If it's indeed just as healthy as regular food it can save a lot of time, and maybe it can even solve some hunger issues in the third world. I'm merely saying it should be studied more before jumping on the bandwagon. I mean, it's made by a 25 year old electrical engineer who failed a cheap cellphone tower startup. In an apartment in The Tenderloin. If you want to put your life in his hands by all means go ahead, but I'll just stick with actual food for now, at least until I see the results of some clinical trials.

On a side note: food as a business is hard. Medicin is even harder. There are so many rules, regulations, government agencies, etcetera. They're mostly there for a reason: to keep food and medicine safe. I hope these guys realize what they're getting into. If there's only a hint of claiming something is 'just as healty as regular food' you're opening a giant can of worms...

<comment><USER32> You know how something gets studied? People _try_ it. There are no "clinical trials" proving eating grapefruit is safe. Or hot dogs. Or bread. Or any specific cracker you find in the cracker aisle. Soylent is food. People eat food; if they like it they sometimes eat more and if they crave something else they eat something else. This is a non-problem.

> I mean, it's made by a 25 year old electrical engineer who failed a cheap > cellphone tower startup. In an apartment in The Tenderloin. If you want to > put your life in his hands by all means go ahead

When you buy a burrito at Chipotle you're eating something that's made by a 19-year-old who's probably failed at something too, but it'd be weird to call eating that burrito "putting your life in his hands".

</comment></comment></comment></comment><comment><USER33> I love this idea not because I like to replace my meals with this, but the fact that it could improve world hunger problem in a big way. Couldn't it?

<comment><USER34> No.

The world food programme already have a wide range of supplements that are produced in developing nations and are much cheaper than Soylent.

Note that Soylent requires clean water to mix, and a large proportion of the world don't have clean water.

</comment></comment><comment><USER35> The article is dated May 12, 2014. Is that because it's going to be in the print edition that will be released that day?

</comment><comment><USER9> money quote from the article:

 _Perhaps the main difference between Soylent and drinks like Ensure and Muscle Milk lies in the marketing_

It's amazing how taking an old product and repackaging it for a different purpose can be a source of business success and glory. I believe this is the story behind both Coca Cola and RedBull.

</comment><comment><USER19> These are what I call "Get me some Press" companies from a VC / YC perspective. Will make the cover of WSJ ... get a ton of attention to the company and investors and eventually be dead in 3-4 years.

</comment><comment><USER36> Food was such a large burden.

</comment><comment><USER37> Tasty Wheat.</comment>


Google Locationhistory -- Tracking Android Phones by default - <AUTHOR> https://maps.google.com/locationhistory

<comment><AUTHOR> I was checking my Google Activity Report and discovered i can see a complete profile of my movement for the last 3 weeks.

This is nothing i want to be visible to anyone! You can disable it but why was it enabled in the first place?

<comment><USER2> Personally, I enjoy having a repository of my location history that I can browse easily. Where was I on such and such a date? Easily looked up.

As for the privacy issues, they are real, but my carrier could easily replicate this data if required to anyway. So if it's going to be generated with or without my consent, I may as well have access to my own copy.</comment></comment>


Show HN: Prime Camera for iPhone – Simple Manual Control - <AUTHOR> http://kickstandapps.com/prime/ <comment><USER2> "Seemless" should be "Seamless" (home page)

</comment><comment><USER3> Looks cool! I'll have to give it a shot.</comment>


Born to Rest - <AUTHOR> http://harvardmagazine.com/2016/09/born-to-rest <comment><USER2> _Harvard "needs to reopen a discussion about…a physical-education requirement."_

It's been said that "love is a better master than duty," and requiring college students to take a physical education course sounds like a duty.

If you want people to exercise without coercion for the rest of their lives, I think you have to tap into intrinsic motivation, i.e., the unique things that get _you_ excited. Depending on your personality and life experience, that might be novelty, socializing, recognition, or competition.

Forcing college students to exercise will backfire, leading many of them to stop as soon as the course is over.

<comment><USER3> Well, at least Harvard has the swimming competence requirement to graduate, installed as a condition of the Widener Library family bequest (since their son Harry Widener died on the Titanic).

At least they did in 1976. ;-) I wonder if they still do?

<comment><USER4> Seems that the requirement being due to the Widener family donation is apocryphal[0], but still an interesting idea. Also, that source suggests that there was a swimming requirement for all students only from fall 1970 to spring 1975. It wouldn't surprise me to hear that it wasn't officially required in 1976 but was effectively compulsory for anyone without a "legitimate" dispension.

[0] [http://asklib.hcl.harvard.edu/faq/81791](http://asklib.hcl.harvard.edu/faq/81791)

</comment></comment></comment><comment><USER5> I'm a fan of bringing back physical education. It seems that more of my younger colleagues (~23-30 y.o.) are trying to be physically active/fit. But exercise is a habit. If you don't build it up when you're young, it can be hard to develop it later. And then there's the physical challenges of returning to a physically active life after years or decades of inactivity. Certain injuries are more likely (like back, knee, tendon injuries) which can be debilitating, leading to months of recovery and more sedentary behavior.

And schools can always give several options to accommodate different interests and capabilities.

<comment><USER6> Bringing it back? Is it no longer offered?

<comment><USER5> Speaking specifically of college, I'm unfamiliar with current K-12 requirements in the US. Physical education is no longer _required_ at most universities. Not even one or two semesters. It is _offered_. But so is going outside for a jog, or dropping to the floor and doing push-ups and sit-ups. The issue is one of motivation. If students (really, people) aren't feeling motivated to be active then they won't. Attaching a grade and a graduation requirement to it gives them an immediate motivator.

The hope, then, is that this pattern of physical activity remains with them throughout their life. Ideally also paired with courses on nutrition and diet.

<comment><USER7> Considering that gym class was hell on Earth for some of us fat people and we greatly looked forward to the liberation of college, I think the idea of mandatory PE is horrible.

PE is usually taught in a manner that's great if you're already fit, and atrocious if you're fat. It's hard on the joints and exemplifies how much weaker/slower/less valuable to society you are compared to the fit kids.

I know this because I WAS fat. Horribly so. I lost nearly 100 pounds in my mid 20's by doing something simple - ignoring the exercises I hated (running, any team sports, etc.) and doing what I loved (swimming and cycling, which no PE class had offered). Instead of agonizing knee pain I experienced the joy of cycling, and instead of being a dripping sweaty pig I was in a nice cool pool, where body heat was transferred away easily.

I also quit eating the poison that some call food on a college campus. Remember that at this time people thought margarine was comparatively healthy!

(Note - I use the word "fat" because I think it's an accurate descriptor. It has unfortunately taken on a lot of emotional overtones. But let's be honest, the simplest word that describes the experience I had is "fat").

<comment><USER5> High school and middle school PE were fucking miserable for me. I wasn't fat, but I was slow and in a group with very fit (naturally or by practice) guys. I was also uncoordinated as I had a _massive_ growth spurt and, being extremely near sighted, wasn't active in any typical sport so my coordination lagged behind for years.

College PE is different. It's not (usually) a general class. You typically pick which one you want. Weight lifting, running, cycling, golf, swimming, etc. This means you're working on a physical activity that you _enjoy_ or have some moderate interest in, unless you truly despise moving and can't even find one thing in the list that interests you (and somehow never fits into your schedule over the 4 years of college).

And all those options were at a cheap, no-name state school.

EDIT: As well, you typically end up setting your own goals within the constraints of the course. Usually an A requires: Be fitter/healthier/faster/stronger than when you started.

<comment> guard-of-terra Where I studied, there were indeed groups of choices, but they had to take you for some reason (either be fast to apply or have good fit already). Most people were raked into general class, where it was as bad as in school, under the fear of dropping you out, since it was a required course.

Seriously makes me want to bulldoze the stadium (bonus points for frags) and set a few extra dorms on top.

</comment></comment></comment></comment></comment></comment><comment><USER8> _" But humans also "evolved with a very large stick: if you didn't exercise, you had nothing to eat." Exercise was mandatory. For many humans today, he points out, there are very few incentives and no penalties."_

Not entirely true - in fact further down in the article:

 _" Furthermore, says HUHS director Paul Barreira, the same surveys show that students' own sense of health and well-being tracks the amount of exercise they report getting. Those with the most depression and anxiety also get the least exercise. The happiest students get the most."_

So the penalties include depression, anxiety, and in future, propensity for heart disease, diabetes, hypertension, etc., etc. So either we haven't evolved enough to consider the penalty of dying young and having a more miserable existence while alive, or is it just that evolution optimizes to the point of reproduction and then doesn't care about much after successfully handing down genes to the next generation?

(Edit: I might also ask, without putting forth a theory or guess, what statistics say about obesity and likelihood of reproduction? If health is poorer than non-obese humans, is there a lesser chance of successful reproduction? If so, shouldn't that be considered the ultimate penalty?)

<comment><USER9> Getting up and going to the gym sucks right now. It sucks in the 10 minutes it takes me to get to the gym. It may suck for some time while I'm there. It sucks after when I think about the time I could have spent playing videogames or doing work ( I'm a notorious workaholic so this is big for me). It feels great in 20 years if I do it consistently enough.

Eating crap feels great right now (endorphin release and whatnot). Sleeping in feels great right now. I know what exercising feels like, I know what being lazy feels like. I don't know what diabetes and hypertension feel like. Getting the will to overcome that and make the right choices consistently takes diligent practice and sometimes a little coercion.

As someone whose parents never forced me to play sports, exercise is a particularly bad struggle. I stepped into the gym for the first time in college. I've never really experienced being skinny (I would love to someday and am working towards that). Motivation is slightly more complicated than people like to acknowledge, especially when you consider that the environment we experience now is so drastically different from the environment we evolved in.

Also, in a social species, reproduction is a less acute need for individuals. See: why condoms exist. Personally, I hate the idea of having children. My act of "reproduction" will be making the world a better and more survivable place for the other children of the world whose DNA is 99.99% similar to mine.

</comment></comment><comment><USER10> Modern solution to campus exercise requirement: Put the dining hall 1 mile away from the dorms.

<comment><USER5> And remove cheap, high calorie, low nutrition foods. No reason schools ought to provide students with tons of mac & cheese, they can do that to themselves. Give them healthy, nutritious options with the meal plans, and let them spend extra on the unhealthy stuff if they really want it.

<comment><USER11> Isn't college where people are supposed to be responsible for their own actions? Or have we extended the hand-holding that far out?

<comment><USER5> Sure, they can be responsible for their own actions. Doesn't mean the schools have to provide them the option to be actively harmful to their health. They're adults (as mentioned in another comment). If they want to buy a few cases of Coca Cola and drink that instead of water every day for 4 years, that's their prerogative. No school should stop them from making such a terrible, stupid mistake, but no school should present that to them as part of the meal plan either.

<comment><USER11> There's a little bit of cognitive dissonance going on here. If they're responsible for their actions, and they're adults, why are you taking an option away from them in fears that they're "making a mistake"? This is universally seen as hand-holding.

<comment><USER12> So should the meal plan serve alcohol, too, to avoid reacting to 'fears that they're "making a mistake"'?

<comment><USER13> That's a bit of a straw man. You can choose alcohol or no alcohol. But you don't choose mac n cheese or not -- you choose mac n cheese or some other food.

<comment><USER12> You can choose alcohol or some other beverage.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Is there actually any evidence that ancient hunter gatherer groups didn't do what we now call "exercise" on a voluntary basis? Some ideas come to mind: dancing, walkabouts, spirit quests, right of passage rituals, ?ancient forms of martial arts? I am compelled to believe that early man did things merely to do them, including exercise.

<comment><USER15> Interestingly, all of those activities you list seem to have an underlying purpose beyond merely "fitness for the sake of fitness" \- exercise (and the fitness which follows) are incidental side-effects of completing those activites.

Not sure what you mean by "actual" evidence, but we know a lot of what we know from studies of modern-day hunter gatherer groups and it seems unlikely that the ancients would have significantly different behavior around needless calorie expenditure.

<comment><USER14> Modern day hunter gatherers do not "play"?

<comment><USER15> "Play" isn't exercise for the sake of exercise.

<comment><USER16> Which leads us to the current situation of the only people who stuck to exercise long term are those for whom "exercise for the sake of exercise" is play.

</comment></comment></comment></comment></comment><comment><USER17> Let me die in peace. For some reason, it is always assumed that everyone wants to live as long as possible. If you don't have a cola each day, you can dribble into a towel for another 3 years when you are older.... No thanks, Ill take the coke. put it on the reapers tab.

<comment><USER18> It's so strange to me how people in this country seem to share a common expectation that an adult should be a productive member of society, but have no expectations about being a healthy member of society. The amount of taxpayer money spent on providing treatment for type II diabetes alone is staggering, and even the amount that isn't paid by the government is paid by insurance, so still amortized across the population: "The total estimated cost of diagnosed diabetes in 2012 is $245 billion" "Most of the cost for diabetes care in the U.S., 62.4%, is provided by government insurance"

[http://www.diabetes.org/advocacy/news-events/cost-of- diabete...](http://www.diabetes.org/advocacy/news-events/cost-of- diabetes.html?referrer=https://www.google.com/)

<comment><USER19> The cost is so high precisely because our healthcare toolkit is so poor.

If we could restore our systems to full functionality, our healthcare cost will drop like a rock.

Though we will be faced with new questions. If everyone are fully healthy, then our mortality rate will also drop like a rock, destroying social security as it currently exists.

<comment><USER20> Solution: Extreme sports.

</comment></comment></comment></comment><comment><USER21> I've noticed that, at least in photos & videos, it seems cheetahs are exclusively either sprinting at 75mph, _or_ lounging on the ground like a housecat.

It certainly makes sense for them, considering how much energy they expend to run that fast. So perhaps there's precedent.

</comment><comment><USER22> If we have always been lazy, why is there now an obesity epidemic? It seems to me that means the cause of obesity lies elsewhere.

<comment><USER5> Prior to the 20th century (depending on where you live, but this is for the areas hit with obesity epidemics):

1\. You had to walk most places (often miles a day).

2\. You had to physically labor. Most people didn't have desk jobs or the period's equivalent.

3\. Food was relatively scarce, and harder to transport.

4\. Social activities (we are social creatures) required physical effort (walking to, dancing, walking around). We can now be social while being remarkably physically passive.

5\. Entertainment options were more scarce, so boredom meant you _did_ something. Now, you can entertain yourself by sitting on a couch or at the desk playing a video game or watching a movie. You could make a case for books being similarly physically passive, but that requires widespread literacy and access to books which wasn't true of the general population.

<comment><USER23> The expansion of ranching and farming in the great plains of the Americas was sufficiently bountiful to lead to the first 'health food' crazes of the early 20th century--more people had enough to eat, and increasing automation was already reducing labor. So the roots of our current problem were already being planted, as it were. But in the last 30-40 years, there's been a noticeable uptick in average caloric consumption, even more reduced activity, and more of those calories (at least in the US) coming from processed foods with ever- larger quantities of sugar.

</comment></comment></comment><comment><USER24> Delightful cause behind muscle atrophy. Annoys me on a daily basis because I have to keep working out to maintain my shape, but makes so much sense once you see why it is the way it is.

</comment><comment><USER18> I was very frustrated by PE. At my college, you could take a class about making maple syrup for a wellness credit. It's a joke. But how many people can brace correctly, or squat, deadlift, and press with proper form? These are archetypal movement patterns that we repeat 100s of times a day without thinking about it, and most people don't even know what they're supposed to look like. How many people know how to use a foam roller? How to read a nutrition label? Forget about trying to impart healthy habits, most people don't know what healthy habits are and wellness classes, whether they're making maple syrup or playing soccer for 2 hours a week, aren't going to fix that.

<comment><USER25> In my opinion the maple syrup class could be a fantastic wellness opportunity -- making maple syrup involves hiking around the woods, often in the snow, carrying heavy buckets full of sap around. It's this kind of real-world, ecologically reasonable movement that most people are so profoundly deficient in. To say nothing of spending time in a beautiful natural setting, which is often quite restorative and de-stressing.

It would be awesome if everyone knew proper lifting form, but it's just not going to happen. Getting people to engage in an activity that they like, and that involves lots of real-world movement, is the biggest win you can hope for, practically speaking.

<comment><USER18> >> It would be awesome if everyone knew proper lifting form, but it's just not going to happen

I'm not talking about proper lifting form, I'm talking about proper human movement. We perform these archetypal movement patterns every day throughout our entire lives. Every time you sit down in a chair you're performing a squat. Every time you pick up your bag off the ground, you're performing a deadlift.

And to say "that's just not going to happen" is a self-fulfilling prophecy. We're not teaching people these skills, therefore we wouldn't be able to teach these skills.

I completed a minor in music and I still had to take more general liberal arts classes for my CS degree. They can afford to include a class that teaches about how to correctly use and care for one's body.

<comment><USER25> It's less a self-fulfilling prophecy than drawing on a bunch of professional experience with trying to get people to do things that would be vitally important to their health.

</comment></comment></comment></comment><comment><USER26> That debate is already extremely toxic due to the diet craze nutjobs, it really doesn't need any of the unscientific hunter-gatherer lore that extrapolate bits of data to romantic premedieval fantasy novels.

</comment><comment><USER27> This explains why I am unable to bring myself to exercise without getting exhausted instantly.

But if I need to do something like fix a car, build something, whatever, I'll work myself to utter exhaustion and not even notice.

</comment><comment><USER28> Interesting.

This thread is a good argument that many people don't choose effective health without coercion.

It's probably a good idea for public health for the government to sponsor to the point of "effectively free" adult sports such as running/soccer/swimming, along with requiring such each semester for state-sponsored schools.

It's probably past time for me to put on my running gear and take regular evening jogs, myself. :-)

</comment><comment> guard-of-terra Can we just turn off those "lazy/slacking" mechanisms on genome/cell level? A pill will be nice.

Make your organism always maintain optimal muscles and circulation system while burning a lot of calories? Seems like a win/win to me.

Some people are strong and healthy without excersice, I think we are all entitled to same package?

I had PE in University (and before that school), hated it and now I'm turned off from any excercise.

<comment><USER21> Could be upsides to that, but cranking up the global food requirements (remember, a muscular body requires a lot more energy to maintain) would be a negative.

Also, what's optimal? Your body adapts to what you are doing, something we take for granted today. E.g. If you like bicycling, a permanent linebacker build may be undesirable.

It would probably be more ideal to set some kind of fitness "floor", rather than remove the mechanism completely.

<comment> guard-of-terra Most people already eat much more than they need, might as well make use of those calories.

Of course there should be a "floor", we don't want to grow into blobs of slithering muscles and veins, just well-built.

</comment></comment></comment><comment><USER29> If they think it's a good idea to require physical education for the students, why not require it for the faculty?

</comment><comment><USER30> The part of making exercise more fun is interesting. I remember growing up and being shocked that adults did not play team sports in such a team-sport obsessed country.

Also, probably could to have some sort of multiple-person activity in ones schedule that isn't work (c.f. church attendance dropping faster than religiosity).

</comment><comment><USER31> It is odd for a piece like TFA to fail completely to mention the role of internal combustion engines. Harvard has many shuttle buses, which might make sense when the snow is deep but seem actively harmful while serious consideration is being given to a phys ed requirement. A less coercive and cheaper solution might be to scale back the shuttle buses and introduce some sort of bike-sharing scheme. They could make uphill rides like Mather to the Yard free, and charge a nominal fee on the fun downhill rides. (Otherwise all shared bikes would end up in the river forever.) I guess they'd want to subsidize both directions for those poor souls stuck in the Quad.

[EDIT:] It occurs to me that this is the sort of amenity that would be basically free for Harvard to provide. Because it would be a highly visible part of students' lives, hordes of donors will line up to sponsor individual bikes (with donor nameplates), pay stations (likewise), named chairs for bike mechanics, or even the whole program.

</comment><comment><USER32> Quite interesting that Harvard Magazine would publish something like that. Doesn't that pretty much imply that Harvard is unnatural and abnormal</comment>


Snapchat CEO 'Mortified' Misogynistic Frat Emails Were Leaked - <AUTHOR> http://www.huffingtonpost.com/2014/05/28/snapchat-emails-evan-spiegel-ceo-cofounder_n_5405389.html

<comment><USER2> I'm not justifying what Evan said in his emails whatsoever, but why are we chastising him for something he said many years ago when he was a stupid fraternity bro? Oh, he's a CEO now and that means we need to go bringing up his past and using it against him? That doesn't make it right. The real issue here is Evan's privacy was compromised by the leak of those emails. If he said what he said in 2014, then yeah, cause a storm of controversy, but come on.

And without casting stereotypes, the fraternity bro culture isn't just localised to one person. It's a cultural thing that has been happening for decades. People that attended college in the 60's were doing hard drugs and talking in the same vulgar speak as Evan. Sure, not everyone is like that, but this isn't new behaviour only limited to billion dollar app running CEO's.

</comment><comment><USER3> "I have no excuse. I'm sorry I wrote them at the time and I was jerk to have written them. They in no way reflect who I am today or my views towards women."

I would like to believe that this statement is true, but that frat boy mentality is the exact kind of mind that would produce a product like Snapchat.

</comment><comment><USER4> Personal views totally aside, I absolutely despise people for leaking stuff which is intended to stay private as long as it isn't whistleblowing about actual law breaking.

That whole attitude of the tabloid press and its web spinoffs "press, press and if pressure doesn't help then hack, as long as we get something that MAY look newsworthy" aka "we don't care where the stuff comes from or how it was obtained, as long as we can squeeze page impressions and money out of it" is vile, disgusting and every single person "leaking" this kind of stuff should be thrown in jail, keys put away and the "media" closed down.

You and your fellow "media" friends are undermining the trust in EVERYONE. Go fuck yourself or invest some money to obtain _news that matter_.

<comment><USER5> Don't you realize now that your professional and personal success hinges on the fact that you've never said anything ever even to your most private and intimate friends that might in any way be offensive to anyone at all at any arbitrary point in the future?

It seems like it's once a week now that someone is run out of town on a rail for saying/doing/thinking something that the Teeming Millions disagree with. And that for some reason that completely justifies a career death sentence.

I don't get it, and I don't like it. And it's not because I have any particular skeezy email skeletons in my closet I'm afraid will come to light, I'm just bothered by the current trend that the Thought Police can come in and be judge, jury, and executioner, and anyone who even deigns to say "hey wait a minute, that doesn't seem right..." gets thrown into the pariah patch.</comment></comment>


Show HN: Hone for iPhone 4S - never lose your keys again - <AUTHOR> http://kck.st/KEDJ6B Hone is a Bluetooth low energy device for the iPhone 4S and new iPad that erases the annoyance of losing your keys - my partner (Louis Gerbarg) and I built it for ourselves, and then decided to put it on Kickstarter. It lasts for 6+ months on a CR2032 battery, has a 50 meter range, and we're making the whole thing in the USA - case, PCB, and component assembly. We've gotten lots of help and support from HN users (esp. Eric Migicovsky's Solid State Hardware talk, curtgeen's hardware design guide) and we'd love feedback now that we're launching. <comment><USER2> I find it funny how much emphasis creators are putting on how things are developed in the USA. If someone puts that up as a feature of their product all I hear is "We have issues in our product but you'll put up with them because we are playing the demagogic card."

<comment><AUTHOR> We set up our supply chain in the US because it sped up our development and saved us from customs issues and the like, not for jingoistic reasons; we don't have an issue with global manufacturing. I assure you, we will stand behind our product!

</comment></comment><comment><USER3> What do I do when I can't find my iPhone?

Snark aside, this looks great. The beauty is that it's not necessarily for your keys, its for anything you lose often, if you can figure out a way to attached this to said object.

<comment><USER4> That would significantly increase its value: a button on the unit to cause the phone to buzz, so that one can always be used to find the other.

</comment></comment><comment><AUTHOR> My description got eaten - anyway, Hone is a Bluetooth 4.0 low energy device for the iPhone 4S and new iPad that helps you find your keys when they're lost. 6+ month battery life on a single CR2032, 50 meter range, weighs 0.6oz. HN posts inspired me and my partner (Louis Gerbarg) to do this, especially Eric Migicovsky's Solid State Startups video and curtgeen's guide for the design of physical products. We'd love your feedback!

<comment><USER5> Awesome, I've just backed it.

One question: What happens when there are multiple devices around?

<comment><USER6> The app will support multiple devices when we ship.

</comment></comment></comment><comment><USER7> I still have a key-chain thingy that beeps back when you whistle. No Bluetooth and no proximity sensor, but it runs on the same battery I put in in early 90s :)

</comment><comment><USER8> Are there any iPhone/Android + Doorknob solutions out there? I don't want keys at all! I would love to replace my doorknob and just have my phone transmit a key to unlock the door as I approach.

One problem could be if the phone ran out of battery. Perhaps a small usb charger near the door for emergencies?

Man I would love that.

</comment><comment><USER9> I thought I recognized his sister (featured in the video). Turns out she is Kat Dennings, a relatively successful actress. - <http://www.imdb.com/name/nm0993507/> . Anyway, nice project...most of the key locator products currently on the market are junk.

</comment><comment><USER10> Simpler solutions have been around for years. For less than a dollar, you can buy a keychain that responds to claps or whistles... [http://www.alibaba.com/product- gs/535558081/clapping_key_fin...](http://www.alibaba.com/product- gs/535558081/clapping_key_finder.html)

</comment><comment><USER11> I knew I recognized his sister. She starred in 'Nick and Norah's Infinite Playlist.' Awesome, awesome movie.

</comment><comment><USER12> I love how they used the celebrity sister. Why didn't he explain that further in the video or something?

<comment><USER13> Explain what? The video is about the product, not about the sister.

</comment></comment><comment><USER14> This is a great idea! I'd love to see how much slimmer they can make the fob though. Between my car key and my apartment fob, it's almost impossible to lose my keys because they take up so much room!

</comment><comment><USER15> how does this differ from phone halo? <http://www.phonehalo.com/products-2/>

<comment><USER5> It appears to work with iOS and low power bluetooth 4.0.

<comment><AUTHOR> Exactly right. The Cobra Tag uses traditional Bluetooth, so it has to be recharged every couple of days; we use Bluetooth low energy and a removable battery that lasts 6+ months. They don't support iPhone and we (currently) don't support Android.

<comment><USER16> I'd love to have this but it being iOS-only kills it for me. Any thoughts on opening up the spec? I'd love to implement this for both my laptop (Linux) and my Boot2Gecko phones.

<comment><AUTHOR> It uses a custom UUID for advertising, but other than that it uses standard Bluetooth LE profiles for notification and battery status, so you should be able to do it...

</comment></comment></comment></comment></comment><comment><USER17> Seems like you could pick up another large pool of users for almost no cost, the on-the-phone software can't be hard and new Android devices have BT4 support.

<comment><USER6> We would love to support Android, but two issues are currently preventing us from doing that:

1) A low percentage of currently shipping devices support Bluetooth low energy. We expect this to change in the future, but it's hard to explain to consumers which models will and won't work. We think we would probably get a lot of users buying are device for incompatible handsets, which would result in a lot of returns and unhappy users.

2) BLE isn't supported in the base Android distribution. In order to support it you need to use different BLE stacks depending on the device vendor, which would be a huge potential ongoing maintenance and QA burden.

Having said that, we're hopeful Google will add support for BLE with a standard API in a future release, and at that time we will seriously evaluate supporting it.</comment></comment>


Show HN: Pipecat – Connect message queues with Unix pipes - <AUTHOR> https://github.com/lukasmartinelli/pipecat <comment><USER2> This looks really cool. I went ahead and added it to the Arch User Repositories[0].

[0]: [https://aur.archlinux.org/packages/pipecat- bin/](https://aur.archlinux.org/packages/pipecat-bin/)

</comment><comment><USER3> Ok, I know message queues[1] are not the most popular part of System V IPC, but was it really necessary to overload the name?

(I mean ActiveMQ _et al_ overloading "message queue" in general, not you pipecat project)

[1] [http://beej.us/guide/bgipc/output/html/singlepage/bgipc.html...](http://beej.us/guide/bgipc/output/html/singlepage/bgipc.html#mq)

</comment><comment><USER4> Great job! I am definitely gonna use this daily!

</comment><comment><USER5> hey guys can u provide usage scenario ?

</comment><comment><USER6> Why no 0mq? :)

<comment><AUTHOR> I don't have any experience with zeromq as I mostly deal with AMQP protocols. But good that you mention it - if there is a demand I can dig into it.

There are quite a few other message queue systems out there. I can also imagine that pipecat could work as a kind of migration systems between them. But this probably should be a separate tool as it requires shuffling around metadata as well.

</comment></comment><comment><USER7> Very cool!

</comment><comment><USER8> That logo... yikes.

<comment><USER9> Well it's...memorable?

<comment><AUTHOR> Now that I look at it again it also seems offensive. Switched it to a more boring but more suitable logo.

Thanks.

<comment><USER10> I didn't see anything wrong with the old logo. Too bad you felt the need to change it.

<comment><USER11> Yea, cats have/are prominent assholes. It's just how they are!</comment></comment></comment></comment></comment>


Twitter co-founder Biz Stone is returning to the company - <AUTHOR> https://techcrunch.com/2017/05/16/twitter-co-founder-biz-stone-is-returning-to-the-company/ <comment><USER4> I know a lot of the early history, I worked there when we hired both Jack and Biz. I can tell you that it's a really good thing that Biz is back on board. Biz was able to articulate what twitter was as the human voice of the company in both directions. From day one the question was always, what does Biz do.

Biz bounced around but when he was there things were better. Kind of like the basketball player that Nate Silver likes to love, who doesn't have any stat which makes them star, but everybody else around them plays better when they're there. Biz isn't a business guy, nor product, nor code, nor support, nor really marketing. But when he's in the room, working with people, everybody's better at all of those things.

He can play a kind of court jester role, which is disarming, but he's super damned sharp. He uses stories and humor to bring people forward.

Having him there, working on twitter means there are now two people in senior roles who aren't afraid of breaking twitter, because they created it in the first place.

In recent years, talking to twitter employees you get this amnesia over the company's culture and history. People don't know where things came from, they don't know the story of how the came to be. The myth's are complicated and messy. And eventually go so messy the company stopped telling the story of how twitter came to be where it is now all together.

With Biz back, he can take on that internal story telling, creating a hero's journey that the company can believe in. Because he's there, as an equal to Jack in understanding the origin, he can tear things down without fear of destroying somebody else's house of cards.

<comment><USER5> Thanks for your post. I'm curious about what you mean by "I was there when we hired Jack and Biz" though. Were they not founders?

<comment><USER6> Rabble was the first Eng hire at Odeo, the podcasting startup founded by Noah Glass and Evan Williams. When Odeo didn't gain enough traction and they were figuring out what to do next, a small Odeo team (which included Jack) prototyped Twttr. And the rest is history.

<comment><USER7> Well, we still had to scale it.

<comment><USER4> Yeah, huge amounts of work from prototype to world changing platform happened after i left. Nobody created twitter, rather it was the collective work of many.

</comment></comment></comment></comment></comment><comment><USER8> "Stone explained that his top focus will be guiding company culture. 'It's important that everyone understands the whole story of Twitter and each of our roles in that story. I'll shape the experience internally so it's also felt outside the company.' – Biz Stone

Of the things twitter needs - strategy? product direction? revenue? - <USER3> "guiding the company culture" really that high on the list?

<comment><USER9> > Of the things twitter needs - strategy? product direction? revenue? - <USER3> > "guiding the company culture" really that high on the list?

This is very typical in dying large companies, you bring back the old guard to try and save the day. Rarely works, but at least it's an attempt.

<comment><USER10> Steve Jobs.

<comment><USER11> I dispute this. Steve Jobs didn't save Apple. Apple saved Next. Today's Apple is not the Apple of the 90s - it is Next with an Apple logo on the box.

<comment><USER12> Nonsense. NeXT had largely failed in the marketplace. The best the NeXT people could do without Steve at the helm was Rhapsody, which Steve hated. And Jony Ive worked for Apple, not NeXT.

That said, I fail to see what unique vision either Twitter founder brings to the party.

</comment></comment></comment></comment></comment><comment> aaron-lebo Semi-related. Does anyone know how accurate _Hatching Twitter_ is?

<comment><USER7> It has many correct facts, but rarely the correct interpretation.

</comment></comment><comment><USER13> Twitter's market cap moved up $200 million on the news that Biz is back.

<comment><USER14> Alternatively, twitters market cap went up 1.4%, which is well within the usual ups and downs of normal trading throughout the day.

Maybe it was because of this. Maybe it was because some analyst put out a piece on a private feed that said "Twitter might make more money someday".

<comment><USER15> " _more_ money"? They've yet to make _any_ money.

<comment><USER16> Looking at their financials, they make tons of money. They just spend gigatons of money. The revenue side of the equation is looking quite healthy, actually.

</comment></comment></comment></comment><comment><USER17> I have to imagine he wouldn't be going back unless he felt he could do something to improve the company's fortunes. It'll be interesting to see if he does have something up his sleeve.

</comment><comment><USER18> It's possible the role is fluid because he's being groomed to take the company over from Jack. I don't think anyone can argue that Jack is doing a good job as CEO but Biz would carry some of the same goodwill that put Jack in the role and might actually be better equipped to turn the company around.

Or he's just "putting the band" back together and next month they will launch their own version of Snapchat's filtered selfies. Who knows.

</comment><comment><USER19> I hope this is a predecessor for him taking over from Jack Dorsey.

I don't understand how anyone thinks Dorsey can run both Twitter and Square at the same time.

<comment><USER20> Lieutenants. Elon.

</comment></comment><comment><USER21> What skills does Biz have exactly? It always seemed like Ev was a product genius, Jack was the business brains, but Biz?

<comment><USER22> Cool name. "The Biz is back, baby"

</comment></comment><comment><USER23> Net positive for twitter. Net negative for Medium.

<comment><USER24> Ev Williams started Medium, so this is completely unrelated.

<comment><USER25> Biz is a board member and investor of Medium.

[https://angel.co/medium](https://angel.co/medium)

</comment></comment></comment><comment><USER26> Just what Twitter needs-- another employee.</comment>


DDoS any Wi-Fi network with just 7 lines of Python - <AUTHOR> https://github.com/pirate/mesh-networking/blob/master/bringitdown.py <comment><USER2> The "D" in DDoS stands for "distributed". Using one interface to flood packets is not distributed.

<comment><USER3> also, the "any" wifi network is misleading since you have to be connected to the the wifi network which for most networks implies knowing the password.

<comment><AUTHOR> Not necessarily, if you know the channel you can create a network on the same channel and then flood that one with noise. It should cause a bunch of packet collisions and retransmits which would slow down the victim network. Haven't tested this yet, so this is pure conjecture, I don't actually know if this style of attack works. If someone does try it please do post back.

<comment><USER4> That's not how wifi works you better off buying a cheap sdr and just transmit actual noise.

<comment><AUTHOR> Yes, but that kind of defeats the purpose of having a simple code snippet with few dependencies. I don't know much about 802.1 and how it deals with noise, you're probably right about it not working. As I said I haven't tested this specific attack and have no clue if it has any effect.

</comment></comment></comment></comment></comment><comment><USER5> I can't really see anything in dnet ([https://github.com/dugsong/libdnet](https://github.com/dugsong/libdnet)) that does anything useful with the string "HOST:all|GET:spam"; it appears all you're functionally doing is spraying data (in the form of that string) at your NIC PHY.

Still, that's fun to play with! Here's my go.

My router (a flaky Netgear DG834GU I'm definitely upgrading when I get the chance) does NOT like the following snippet which floods NULs over ICMP (it took a few seconds for Google to become re-pingable):

      yes "$(printf '%16s' $'\0')" | pv | sudo socat stdin ip-sendto:$(route -n | \
       sed -n '/^\(0\.\)\{2\}/{s/[^ ]\+ \+\([^ ]\+\).*/\1/;p;q}'):1

The call to pipeviewer is only to show I/O rate and can be removed, and the subshell call to `route' and `sed' is just to fish out the first listed gateway; you might want to substitute a specific IP address.

IMHO the moral of this post is that it's a testament to socat's power that I got to the above point with only a few minutes' tinkering; I know nothing about networking. :D

(Sidenote: `yes`ing different string widths produces wildly different results; 16 bytes produces 1GB/s+ of data on this old box, everything either side of that is just MB/s. I wonder why?)

<comment><USER6> If you really want to be evil you could just open TCP sessions to random valid ip addresses. You'll need a custom stack to make it stateless so you can just "forget" about the TCP sessions. You'll fill the NAT table at some point.

<comment><USER5> I've been meaning to learn more about TCP and IP in general, so that's actually quite an interesting idea.

Not too practical in real-world use - if the link speed is slow, getting the connections successfully ESTABLISHED is going to take a few minutes - but the educational value is certainly noteworthy.

<comment><USER6> Here's the great thing. You create your own or use some one else's stateless implementation and you can just slam the interface with MBs upon MBs of syn packets to random ips.

Whenever you get to the next step from any of them you just advance. It's beautiful.

</comment></comment></comment></comment><comment><USER7> Just tested with my X220 generating packets using both this Python script and i336_'s simpler socat command. Both killed the connection of the X220, but neither affected other connected hosts (in particular, my X1 Carbon and Nexus 6 were both able to ping Google just fine).

I'm running a TL-WDR3600[1] with OpenWRT.

[1]: [http://www.amazon.com/dp/B008RV51EE](http://www.amazon.com/dp/B008RV51EE)

</comment><comment><USER8> Kind of off-topic, but since I see it (mis-)used here, I kind of dislike how the term 'router' has become synonymous with 'SOHO Wifi Access Point / Modem / Router thingy'...

In most 'professional' settings, all of these things are actually completely different devices with clear purposes, and this script would actually do nothing against a router. It might freak out some APs, but that's about it.

<comment><AUTHOR> Yes, this will only affect the AP and not the router. The devices that I tested it against were consumer home routers that also serve as APs. My title might be better as 'DoS any home Wifi network with 7 lines of Python and 1 library'.

</comment></comment><comment><USER9> > 7 lines of Python

...and two libraries. Still cool, but come on

<comment><AUTHOR> Two libraries? It's just dnet, and many systems have it preinstalled, the pip library is just Python bindings.

<comment><USER10> reverse runtime deps (RDEPEND) of libdnet: [https://qa- reports.gentoo.org/output/genrdeps/rindex/dev-lib...](https://qa- reports.gentoo.org/output/genrdeps/rindex/dev-libs/libdnet)

edit: also, pip isn't a library, it's a package manager.

<comment><AUTHOR> I know pip is not a library haha, I meant the package you install with `pip install dnet` is just bindings.</comment></comment></comment></comment>


The Race Gap in America's Police Departments - <AUTHOR> http://www.nytimes.com/interactive/2014/09/03/us/the-race-gap-in-americas-police-departments.html

<comment><USER2> This is not to minimize a plethora of issues that I have with modern day policing (i.e. their militarization, lack of dash/body cameras, etc). I do feel like this is an unfair way to analyze this information.

As far as I'm aware, most police forces require at least an Associates Degree, if not a Bachelors Degree. So to compare racial makeup of the community as a whole and that of the police force is unfair. It suggests, without explicitly stating it that the police force is inherently racist. If they were just comparing the racial makeup of community members with degrees to the police force, I think it would be a more fair comparison. The largest disparities in racial makeup are going to tend to be in poorer cities, which will naturally lead to fewer people with degrees.

It is also worth stating, that from a psychological perspective, a poor city with more crime is going to be harder on a cop mentally and make them more likely to misbehave and overreact. Coupled with the racial disparity (which as aforementioned is likely caused by outer forces), could lead to racial tensions.

<comment><USER3> That's some pretty weak apologetics.

</comment></comment><comment><USER4> John Lott found that the more minorities on a police force, the higher the rates of murder, manslaughter, violent crime, robbery and aggravated assault will be. Violent crime increased by a minimum of 3.3 percent every year after affirmative action policies went into effect—and the spike in crime was highest in black neighborhoods. [http://papers.ssrn.com/sol3/papers.cfm?abstract_id=231100](http://papers.ssrn.com/sol3/papers.cfm?abstract_id=231100)

The problem was not with black cops, Lott's study showed, but rather with the lowering of standards across the board, resulting in less-qualified officers of every race. To get more of MSNBC's "voices of oppression" on police forces, requirements are reduced for all recruits. (Just as quality declined at MSNBC when "voices of oppression" had to be added to their lineup.)

<comment><USER5> Did he define exactly what these "Standards" are?

<comment><USER6> Based on the paper at [http://www.law.uchicago.edu/files/files/56.Lott_.Police.pdf](http://www.law.uchicago.edu/files/files/56.Lott_.Police.pdf) , "standards" here refers to various tests, including tests for cognitive skills, that police departments used during employee selection. The tests ended up with a negative bias towards minority groups. This is important because of the 1989 Supreme Court decision in City of Richmond v. J.A. Croson Co. which imposed a strict scrutiny test to ensure that job qualifications fit the goal 'so closely that there is little or no possibility that the motive for the classification was illegitimate racial prejudice or stereotype.'

Eg, and quoting from the paper:

> After spending "$5.1 million to have consultants develop unbiased exams, > only to have minorities fare poorly again," Chicago moved to a heavily > weighted seniority system for promoting police officers and a lottery system > for hiring firefighters (Spielman, 1996, p. 16)

Thus, the supposition in the paper is that those cognitive tests _correctly_ measured skills needed for law enforcement, and with that in supposition in place, it tries to estimate the impact of the changes, concluding that they were negative.

I don't find the paper persuasive. I don't think the hypothesis is tested well enough, I don't see a model for how the police force composition could change so drastically after only 5 years, and I don't like the short baseline of the analysis.</comment></comment></comment>


Being Funny: How Steve Martin got his act together - <AUTHOR> http://www.smithsonianmag.com/arts-culture/funny-martin-200802.html <comment><USER2> The really interesting thing to me is how much work it took him to earn his success. In his book "Born Standing Up" he goes more into his adolescence, where he learned stagecraft, acting, and improvisation. Then as a comedian, years of refining his odd formula until he became great at it.

<comment><USER3> Amazon link to _Born Standing Up_ \- <http://www.amazon.com/o/ASIN/1416553649>

</comment></comment><comment><USER4> A great quote from the article:

 _The consistent work enhanced my act. I learned a lesson: it was easy to be great. Every entertainer has a night when everything is clicking. These nights are accidental and statistical: like lucky cards in poker, you can count on them occurring over time. What was hard was to be good, consistently good, night after night, no matter what the circumstances._

<comment><USER5> Is this not another example of the "10,000 hours to be good" meme that's been floating about for a while? Desire gets you going, and of course it's good to have talent, but then practice and luck... Doesn't work with everything (no way I was ever going to play roundball well, but I could nicely handle soccer), but it's always amazing to see it when it works.

This should be read by everybody thinking an overnight sensation merely took one night.

</comment></comment><comment><USER4> Steve is a master at making his shows odd yet familiar. His mastery of the awkward sparks curiosity, which leads to attraction/engagement from the audience.

This was succinctly outlined in Scott Adams blog post:

<http://news.ycombinator.com/item?id=1164323>

</comment><comment><USER6> I actually have had this steve-martin-ism floating around in my head for more than a couple of years now.. that is truly inspiring (not sure if he ripped it off from another). It's related to show-biz, but I apply it to my classical music ambitions and now my entrepreneurial projects, too: "Be so good, that they _can't_ ignore you." It's quite a zen thing, actually, and it dissolves perhaps some of the complexity of breaking through to the other side with one thing to keep in mind.</comment>


HP and other companies on the hook with banks for $200B - <AUTHOR> https://medium.com/@cloudstories2017/hp-and-other-companies-on-the-hook-with-banks-for-200-billion-on-benprise-tech-site-3fb381468198 <comment><USER2> Is this a real article? It reads like it was written by some kind of poorly- trained AI. While it has complete sentences that, by themselves, make sense. That all falls apart when going from sentence to sentence though.

Either that or it's stream of consciousness rant written by someone with too little sleep on too much caffeine...</comment>


Deep Learning with Keras: EuroScipy 2016 Tutorial - <AUTHOR> https://github.com/leriomaggio/deep-learning-keras-euroscipy2016 <comment><USER2> Are there videos available as well?</comment>


Why becoming a data scientist is not easier than you think - <AUTHOR> http://www.josephmisiti.com/why-becoming-a-data-scientist-is-not-actually <comment><USER3> I couldn't agree more with this. The ML courses that Ng and Koller teach are really missing a lot of the statistical tools you need to do real-world data mining and ML.

My experience: I had basically zero math background, but I took ML with Ng and probabilistic graphical models with Koller, and later was a TA for Ng's ML class, during my Masters' degree and thought I was all set to go into machine learning jobs. To my surprise, I consistently found myself in interviews stumped by questions from basic stats, particularly significance testing, which people with more traditional stats backgrounds assume is basic knowledge (and it should be), but which wasn't taught in any of my ML classes.

I'm in a job now that involves some machine learning, but the ML component is 50% marshalling data (formatting, cleaning, moving), 40% trying to figure out how to get enough validated training examples, and 10% thinking about the right classifier to use (which someone else already implemented). Which to be honest is not very interesting.

So yeah, becoming a real data scientist is hard, requires a lot more knowledge than you get in one ML course, even from Andrew Ng, and the reality of the work often doesn't make it some dream career. And the competition for jobs isn't from other people who also just took that course -- it's from PhD statisticians and statistical physicists who might have taken one ML class to show them how to use all the mathematical tools they already have to do the new hot thing called machine learning.

<comment><USER4> I was pretty gung ho about getting into ML two years ago and put a lot of time into online courses like Ng's, books, and ground-up implementations of a lot of the common algorithms. I enjoyed it, but after a while it became clear to me that a lot of this stuff is better described as applied statistics.

And this can be powerful, of course, but it doesn't really have much of the magic of AI.

<comment><USER5> "a lot of this stuff is better described as applied statistics"

This is a _key_ insight. Bravo!

To generalize a bit more, most of ML is applied _mathematics_. Getting a good grounding in the underlying math is the most illuminating step to learning ML (spoken as someone who wasted a lot of time doing other things thanks to an irrational fear of learning mathematics and am still bad at it)

 _Deep_ math/stat understanding _combined_ with the engineering bits(like programming, cleaning the data, running clusters) and the communication bits, (like visualization) brings you to (what should be) 'data science' (imvvho ymmv etc etc).

I am still not sure one person can pull it all off-it probably needs a solid team of specialists. But hey 'data scientist' is a hot job description, and so you can't blame people who know bits and pieces (sometimes _very_ small bits and pieces ;) ) calling themselves 'data scientists' or whatever. "Machine Learning for Hackers" and all that jazz. We've seen all this before with "HTML coders" from the nineties.

<comment><USER6> Work as a search quality engineer at Google and you do pretty much all of that.

Except for running the clusters[1], I've done pretty much all of those steps myself. I started with a nice statistical idea, built some simple models, played with feature selection and learning algorithms, built model viewers, built classifiers, validated classifiers, built demos, validated demos, built a production implementation[2], optimized the production implementation to make it small/fast enough, and finally launched a big search quality improvement.

[1] I certainly write distributed code that runs on them, but maintaining the DCs definitely isn't part of my job description.

[2] Validation of the final quality in prod is actually someone else's job, not because I couldn't do it, but you might not want me to tell you how good my stuff is, cause you know, I might be biased.

</comment></comment></comment></comment><comment><USER7> I think the idea is... at the moment there is this elite few people that actually have the entire skill-set of 'data scientist,' to one degree or another... but we need more of them. And the way to achieve this isn't to say, "Get a decade's experience across an enormous area of math, computer science, domain experience, and then come talk to me." The way to achieve this is to make data science seem more approachable than it is at the moment... in the hopes that it will be more approachable as we build courses like the one you critique.

This of course devalues your own skills, as you are one of the elite few. Unless you start writing textbooks. Which you should do, if you're one of the few. And self-promote like hell. If that course doesn't cover it - what does? Do you acknowledge that in a few years some shortcuts might be possible - <USER2> budding data scientists might not need to hae read every book that you have? I bet if you try, you can make your own shortcut in the form of a book.

Which is followed by a link to my own book on this topic, Agile Data: <http://shop.oreilly.com/product/0636920025054.do> which attempts to demystify as much as teach.

<comment><USER8> There are different classes of skills. For lots of organizations simply adding someone who can run a MongoDB mapreduce into a t-test (or SVM or whatever flavor you like) and print out a bar chart is going to be an upgrade.

To fill that kind of need we can use a lot of bodies with Coursera degrees.

</comment></comment><comment><USER9> I still don't understand why it's important to have all these areas of expertise embodied in a single person called a "data scientist". Rather than hire one of these rare and expensive people, why couldn't a business hire a statistician and a couple of computer science people and have them work as a team? Given how few data scientists there currently are and the high demand for them, you might even be able to get these three people for less money than one data scientist.

Also, someone who has to constantly shift their attention between statistics and database servers might get less done than somebody who can concentrate on the mathematics and let their co-workers handle the implementation details.

<comment><USER10> Not only that, "data scientist" is just a marketing gimmick for consultants. It's a standard set of skills that anyone trained in science or engineering has. It's ok to use it in front of pointy headed bosses but in front of nerds it's slightly dishonest.

<comment><USER11> ^this. I've often wondered where the science in data science is. I don't see it. If you're an engineer or scientist, you probably don't either.

<comment><USER12> The science in data science involves testing hypotheses, like anyone following the scientific method would do. It's mandatory for validating ML models.

<comment><USER13> But most public examples of "data science" don't do this. They just publish pretty graphs that a great completely unrigorous.

<comment><USER12> OK, but that doesn't mean "if you're an engineer or scientist, you don't see the science in data science" is a true statement. It's categorically false.

</comment></comment></comment></comment></comment></comment><comment><USER14> Well written, but I believe you missed the point of the original article.

No one ever claimed that taking one class made someone an expert "data scientist." Instead, that single class wetted Luis, Jure, and Xavier's (the three competition winners) appetites, and pushed them more to learn more about machine learning and natural language processing. They then went on to dive much deeper, and excelled specifically in one area of applied NLP.

However, without that first class, there's a good chance none of them would have ever focused on (or heard of) machine learning. Their story is growing increasingly common. Like the Netflix Prize, Andrew Ng's first Coursera class did its part in shining a spotlight onto our dark little corner of the universe.

I'd be very cautious about a long checklist of items that are necessary to be a successful data scientist (which is a pretty ill-defined and encompassing term at this point). That is a decent summary of many useful tools of the trade, but they are by no means useful for all problem domains. For example, I could spend years working on machine learning for EEG brain-computer interfaces without a good reason to use databases or "big data" NoSQL technologies. I especially enjoyed MSR's take on the matter in "Nobody ever got fired for using Hadoop on a cluster" [http://research.microsoft.com/pubs/163083/hotcbp12%20final.p...](http://research.microsoft.com/pubs/163083/hotcbp12%20final.pdf)

When we're hiring data scientists or seeking successful ones, we've found focusing on demonstrated excellence in one relevant area plus general quantitative competencies and the curiosity and tenacity to learn new tools and techniques works far better than a laundry list of skills and experiences.

<comment><USER15> Absolutely agree, also author didn't take into account that there are software engineers with good math background who don't know how cool it is to do machine learning. I'm one of them and I've spent last month and a half doing a lot of ML and enjoying the heck of it! My next step is Kaggle.

</comment></comment><comment><USER16> As someone who writes software for data scientists (<https://github.com/snowplow/snowplow>) I definitely agree with his analysis. But I would go further: without _domain knowledge_, a data scientist is really just a ETL guy who cleans up big data for the real analysts to make sense of. Applying the whole toolkit to a specific domain (and SaaS B2B looks totally different from supermarket loyalty schemes and from F2P mobile games) is key.

</comment><comment><USER17> Look -- as long as "data scientist" is a sexy job title, a lot of different jobs are going to claim they fall under that umbrella. I have an applied math background, and I'm fine with scientific computing, but I have much less experience with databases. I'm a very different candidate than a software engineer who took a machine learning course. Maybe in a few years we'll have more intelligent language for making those distinctions.

It shouldn't be surprising or bad news that some "data scientists" have deeper knowledge than others. We're going through a quantitative revolution -- many fields and industries are nearly untouched by statistical analysis/machine learning, and so there's a lot of low-hanging fruit in going from "nothing" to "something." Even somebody who only knows a little can add value at these margins. But, of course, that won't be true forever -- look at quantitative finance, which is very competitive and requires a lot of education, because the low-hanging fruit was picked in the 90's.

There's room in this world for the statistician, the mathematician, the database engineer, the AI guy, the data visualization expert, the codemonkey who knows a few ML methods, etc.

</comment><comment><USER5> "Coursera skipped over Bayesian learning"

This probably needs to be clarified a bit to say that Ng's course skipped this. Daphne Koller's "Probabilistic Graphical Models" (running now at Coursera) covers this in great detail.

Minor tweak in an otherwise nice post.

<comment><USER12> "This probably needs to be clarified a bit to say that Ng's course skipped this."

That seems weird, considering Bayesian learning/methodologies are a big cornerstone of ML.

<comment><USER13> How much can you expect someone to learn in 10 1-hour sessions, starting from a college freshman background?

<comment><USER12> A fair amount; I've seen some of his work and I think it's pretty good. I don't really like Octave (his program of choice), but I understand why he used it. I would have gone with something higher-level, since a lot of decent tools are out there that add a layer of abstraction to ML.

Anyway, I consider Bayesian logic a cornerstone of ML modeling. It's not so much the content that needs to be memorized/understood as much as it is the way of thinking that Bayesian methodologies present over frequentist statistics.

Still, I'm sure it's a good class.

</comment></comment></comment></comment><comment><USER18> As I argued in the comments of "becoming a data scientist might be easier than you think", the attitude that entering the field is easy is dangerous because it is very untrue. I really wish there were more qualified people in the field (do you know how hard it is to hire?), but entering it without the proper knowledge doesn't help anyone.

I'm excited about what the future brings. Many industries has seen the value in data sciences, and Universities are following (see, e.g. Columbia's new data sciences institute).

</comment><comment><USER19> Really though, unless you have a strong understanding of both calculus and statistics you'll never be a "data scientist", you'll just be a library jockey.

<comment><USER8> And probably database theory, probability calculus, matrix analysis, graphical models, stochastic processes, information theory, &c &c...

<comment><USER15> Yes, yes, it's not like you started from something. Were you born data scientist? Just try to accept that there may be people with almost full skill set who didn't know where to apply it? I'm pure mathematician by education and I really didn't know where to apply my skills, having followed ML, NLP, Big data, PGM courses I have much better understanding now.

Now even if I weren't a mathematician it doesn't mean that following Coursera courses for a couple years and doing a lot of work at home wouldn't get me somewhere. You don't have to switch jobs as soon as you finish ML course but you can certainly practice your skills at home.

<comment><USER8> I'm not saying the Coursera classes are bad---just that there's a chasm between being able to implement or derive Naïve Bayes and being able to do meaningful work or study in applied statistics.

<comment><USER15> The way I see it you also learned it somewhere sometime ago. Coursera offers more and more courses some of which are quite in-depth. Motivated person may be capable to finish university degree in couple years. Coupled with the fact that this person may be gainfully employed in similar area, it will not be that hard to imagine him or her to eventually switch to a position that requires all the old and new knowledge. IMHO, examples that were given in the original article are in line with what I described.

</comment></comment></comment></comment></comment><comment><USER20> Maybe I'm mistaken but I think most of the people interested in becoming 'data scientists' are either currently doing lots of software with an interest in stats, or people doing lots of stats with an interest in software. Given one or the other half of this list is probably already very familiar territory.

I actually found this list encouraging because the things I don't know well on that list are things I'm working on and am aware are holes in my knowledge.

But in the end the reality will always be that the people who are "real" data scientists will be the people that are actually solving real problems whether or not they can check off every bullet point on a check list.

</comment><comment><USER21> I think it's interesting to note the number of professions that may already fulfill a "data science" role, just with a different title. I worked a job where my primary role was data analysis: parsing data with Unix commands, feeding it to classifiers, applying standard algorithms, drawing meaningful conclusions, etc.

Sound familiar? The entire team I worked with had a similar workflow, but we went by life science domain specific titles rather than "data scientist." I'm willing to bet that other professions have similar roles, merely called something else.

I think "data scientists" are out there in the sciences. They just don't go by the latest buzzword.

</comment><comment><USER22> Replace scientist with analyst and all of a sudden 75% of the people interested in this career path don't care anymore. I don't get the absolute obsession and sexification of a role that has existed for a long long time already.

<comment><USER23> Data scientist: in many companies, this means a software engineer with an additional credibility that gives him dibs on the most interesting projects. A lot of data scientists end up working on distributed systems problems that would typically be considered closer to hard-line engineering than machine learning or data analysis.

It's an XWP vs. JAP issue: <http://michaelochurch.wordpress.com/2012/08/26/xwp-vs-jap/>

Once you're at or near 30, you realize that you won't be able to stand the software career unless you get an edge in picking projects, because the vast majority of the engineering workload is line-of-business bullshit that you don't learn much from. To grow as a programmer, you have to beat (or cheat) the project allocation game. One avenue is to go into management, but that doesn't work because bosses who take all the interesting work for themselves get undermined. An alternative is the "architect" designation, but becoming an "architect" is even more political than moving into management. Right now, "data science" is a title that has enough of a "+1" to it that it gives engineers the ability to put themselves on the most interesting projects.

<comment><USER17> This makes sense.

It's a "The way to Tara is via Holyhead" kind of thing.

</comment></comment></comment><comment><USER24> what's missing from any discussion here or in many of these "this is the new hotness" posts is this: science.

where's the science? it is, after all, a data scientist role. where is learning to do actual science?

what the world's been describing is an analyst or an engineering position, not science. if you don't know how to ask questions, interpret results, structure experiments - then you don't know science, so quit calling yourself a scientist. science involves a rigor of thinking and doing that has been omitted here.

<comment><USER12> Indeed. It may come as no surprise to you, then, that I push people with a degree in physics far ahead of the line when hiring for my Data Science team.

</comment></comment><comment><USER25> While it is not easy to become an expert in any field or pursuit, one should neither overplay the "it is a very hard field / not very easy" argument nor should one underplay the effort involved in becoming good. 10000 hours (equates to roughly 5yrs at 40hrs per week) to expertize seems like a good rule of thumb to keep in mind. Someone said - "The test of a vocation is the love of the drudgery it involves."

For any field, one has to provide positive encouragement (and a good platform/set of tools and techniques) to people seeking to get into that field, while being grounded in reality.

</comment><comment><USER26> Isn't the real skill for a Data Scientist one of scalability and abstraction, from Data? While its critical to be able to get the data and make it more plastic, for measurement of metrics, real-time pricing, or even various weak- form predictive variables, its ultimately the analysis and understanding that is Critical to monetization/value extraction. And to build a good system for this level of data transparency, you need some good high-level understanding for clarity of vision. There are lots of people good at all manner Quants, but like the interview question, how much complexity can you explain in 5 minutes? is not one all answer equally. The ability to scale from granular detail to abstract levels of organization, meaning, and pattern recognition, are critical to extracting value in these contexts.

Also, its not clear folks are using consistently the term vis a via scale/scope. Consider an anaolgue of knowledge and expertise (real estate example):

L1 Architect>

l2 Contractor>

L3 Sub-contrator>

L4 Builder/Laborer>

Is Data scientist an Architect? Or the person that builds the building? Is he the guy that does the plumbing? Although the up and coming "quantitative system analyst" probably doesn't quite ring the same tune on a biz card. And most refer to lower level quant mastery, eg. social engineering or quantitative finance, as a "black art" not a science. Without a high level vision, the concept/title seems...grandiose, until you get to very extreme levels of skill. And then it makes sense.

<comment><USER27> I think quite a few data scientists are taking the position that they're a specialist plasterer in your metaphor, someone who does one job very well, but can't help design the skyscraper they're working inside.

Good plasterers get paid (and treated) pretty well, but they're never going to be the architects who designed the building.

<comment><USER26> You raise an interesting point here. A good (old world) plasterer is best thought of as an Artisan. This word has been lost in the modern era (Artist!=Artisan), but thats arguably at our loss. The "black arts" are inherently those that require practioners to get their hands slighltly dirty in the details.[1]

_____________

[1] Alternatively, i'm starting to see the clearer picture: "Data Scientist" is more a contraction than a description. viz: "Data-Analytics-Literate Compter Science" becomes _"Data_Science"_ if you just omit the middle terms. For marketing purposes, moniker "Scientist" is highlighted as status signifier (Academically ~akin to "Artist").

</comment></comment></comment><comment><USER28> I have to really thank you for this article, but it may have had the opposite affect on me. I've always felt like I have a disjointed skill set, but this makes me a bit more confident in looking into this field and it give some good ideas of what I should brush up on. I know this may not be what the author intended, but it's appreciated never the less.

<comment><USER29> I came away with similar thoughts. The article made it seem like becoming a data scientist is far easier than I ever imagined.

</comment></comment><comment><USER30> What supposed to be a rant turned out to be a nice little list of things worth learning in data analysis field. Thanks!

</comment><comment><USER31> Thanks for validating I have been pursuing the right skill set. I just wish that list existed 20 years ago when I started college. Instead, I have been teaching myself off and on since 1999.

</comment><comment><USER32> If I have learned anything from ml-class, pgm-class, nlp-class, and now neural-nets, is that becoming a data scientist is one of the hardest things I'll ever eventually succeed in doing.

<comment><USER33> The good man was trying the opposite. Which is to make things as simple as possible for early students and get them do the hardest thing they will ever do: "solve some problems using these tools"

Instead of chasing after some title ("data scientist ..."), find ways of solving some useful problems with whatever you learnt. The article argues that becoming an expert is difficult, which may be true. But that does not mean you don't know enough to start digging at your problems in hand

</comment></comment><comment><USER34> What are the best tools to visualize table data?

I've been testing Tableau, but it's only for Windows and I'm on a Mac.

I'm looking for something which easily connects to your SQL database and allows you to produce all kinds of fancy graphs, with a easy user-interface. Tableau comes close to what I'm looking for, but my gut feeling is there should be a whole bunch of good commercial and free software in this field out there, but my googling haven't gave any good results yet.

So if anyone have any good suggestions, please let me know.

</comment><comment><USER35> being a good data scientist is about having enough intuition about the dataset to _ask the right question_ aka form the hypothesis.

working out the question is what makes it a hard(and creative) process, and then you can apply your ML toolbox.

edit: whats different from a data scientist vs analyst/statistican is they build their own tools as the datasets are too massive & non-standard for the usual toolset.

</comment><comment><USER36> So is there a good middle ground between being a web developer and a data scientist? If so what would be the most useful problems for such a skillset to solve?

</comment><comment><USER23> I was a math major in college, with a focus on pure math. I did a year or grad school (math PhD program) and left to work on Wall Street (and worked for a couple startups, and Google, in that mix). In all, I spent 6 years as a mix of quant, trader, software engineer, startup entrepreneur, data scientist.

The software engineering career is in somewhat of a mess right now. It comes down to the "bozo bit" problem. Being a software engineer (even with 10+ years of experience, because there are a lot of engineers who only do low-end work and don't learn much) is not enough to clear the bozo bit, and you won't be able to prove that you're good unless you have a major success, and it's hard to have that kind of success without people already trusting you with the autonomy to do something genuinely excellent.

It's not enough to write code, because LoC is a cost and source code is rarely actually read at the large scale. At least for backend developers, the only work-related (as opposed to political) way to establish that you're worth anything as a engineer is to have an architectural success, but it's very hard to have architectural successes unless you've established yourself as an "architect" to begin with. So there's a permission paradox: you can't do it until you've proven you can, and you can't prove you can do it until you've done it. Hence, the vicious politics that characterize software "architecture" in most companies.

Functional programming is one way to put yourself head-and-shoulders above the FactoryFactory hoipolloi. The problem is that most business people don't understand it. They just think Haskell's a weird language "that no one uses". Elite programmers get that you're elite if you know these languages, but most companies are run by people of mediocre engineering ability (and that's often just fine, from a business standpoint).

It is true that functional programming is superior to FactoryFactory business bullshit, but not well-enough known. Good luck making that case to someone who's been managing Java projects for 10 years. What is better known is that mathematics is hard. It's a barrier to entry. I doubt more than 5% of professional programmers could derive linear regression.

So I see the data science path (and yes, it takes a long time to learn all the components, including statistics, machine learning, and distributed systems) as a mechanism through which a genuinely competent software engineer can say, "I'm good at math, and I can code; therefore, I deserve the most interesting work your company can afford to fund." It's a way to keep getting the best work and avoid falling into that FactoryFactory hoipolloi who stop advancing at 25 and are unemployed by 40.

<comment><USER37> I don't know how you can say that knowing functional programming inherently makes you a better programmer than another. A bad programmer is a bad programmer regardless of domain.

I have seen some ninja FP artists that have written some really terse solutions in a FP style which are completely unreadable to the uninitiated.

Also, I do not think that there is anything intrinsically difficult about FP. Knowing it proves that you are ambitious and that you have mental plasticity but not necessarily that you are a good developer.

You're just as likely to become that weird dude that writes code no one understands as you are to become Mr Wizard.

<comment><USER13> Mr Wizard was accessible to children!

</comment></comment></comment><comment><USER10> The author clearly has a bias (I'm a data scientist so respect me and pay me a lot). He's then gone on to describe some standard programming and maths skills that a huge number of people have (taught to engineers/scientists/programmers). I'm going to get downvotes and be labeled troll but I just have to plainly disagree. Data science isn't some magic new career field, it's simply the application of standard scientific tools to tables of numbers. As netflix and kaggle competitions have clearly demonstrated, literally anyone from anywhere has a shot at be the best on any particular spreadsheet of numbers (that what it boils down to, (possibly large) spreadsheet of numbers).

<comment><USER38> You actually do not contradict the author that much. The thing is, the skills that are required are somewhat standard programming and graduate level maths, which narrows down the number of people to graduate level computer scientists. As he mentioned, the data science is NOT JUST application of someone's algorithms to the data, you need to have skills to preprocess the data, be able to use distributed systems for big data, actually know what is going on under the hood, etc. Also, I doubt that the netflix and kaggle competition winners were anyones from anywhere, they probably already had quite a bit of experience with ML.

<comment><USER39> Agree, unless your arguing that graduate level math(s) are only 'truly understood' by individuals with post-graduate math(s) degrees. In my opinion, reading (and practicing) is a viable alternative to the same skills. Or, as more famously said by Matt Damon... (<http://www.youtube.com/watch?v=ymsHLkB8u3s>)

<comment><USER12> Actually, the strongest candidates I've screened have been self-learners who list Coursera or their own OSS projects on their resume with little academic background. (Our best analyst is a guy who is qualified to repair VCRs with his trade skill diploma in rudimentary electronics.)</comment></comment></comment></comment>