Web Scraping with Electron - <AUTHOR> https://en.jeffprod.com/blog/2019/web-scraping-with-electron/ <comment><USER2> > Is there a better way to surf the web, retrieve the source code of the pages > and extract data from them ?

Yes, of course! To get the source code of a web site you don't need a browser and all its complexity. It makes me so sad how far we have come in terms of unnecessary complexity for simple tasks.

If you want to extract data from web pages without requiring hundreds of megabytes for something like Electron, there are lots of scraping libraries out there. There are for example at least two good Python implementations: Scrapy[1] and BeautifulSoup[2].

[1]: [https://scrapy.org/](https://scrapy.org/)

[2]: [https://www.crummy.com/software/BeautifulSoup/](https://www.crummy.com/software/BeautifulSoup/)

<comment><AUTHOR> Sure, but i meant to build a portable app, for end users who are not coders, with a GUI, and for a dedicated purpose, like for exemple navigating on facebook.

So i will edit this question to this : Is there a better way to code a portable application with a graphical user interface to scrape a given site ?

Thanks for your comment.

<comment><USER3> You can access the html of the website and use regular expressions.

<comment><AUTHOR> > You can access the html of the website and use regular expressions.

Yes but using regular expressions is the last and least recommended solution, please read : [https://stackoverflow.com/questions/3577641/how-do-you- parse...](https://stackoverflow.com/questions/3577641/how-do-you-parse-and- process-html-xml-in-php/3577662)

<comment><USER3> If you read that link, it's only not recommmended because people don't know how to use it. Regular expressions are powerful.

<comment><USER4> Read the link. Just wondering how you managed to interpret this:

> regular expressions is a waste of time when the aforementioned libraries > already exist and do a much better job on this.

as this:

> it's only not recommmended because people don't know how to use it

<comment><USER3> > [https://stackoverflow.com/questions/3577641/how-do-you- > parse...](https://stackoverflow.com/questions/3577641/how-do-you-parse-and- > process-html-xml-in-php/3577662)

It says "can make regex fail when not properly written" etc.

There are different circumstances where using a premade parsing library versus using raw regular expressions are going to make sense.

The answer is not binary.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> I wish there was an easy way to send commands to the console of a browser.

That would be all I need to satisfy all my browser automation tasks.

Without installing and learning any frameworks.

Say there was a linux command 'SendToChromium' that would do that for Chromium. Then to navigate to some page one could simply do this:

SendToChromium location.href="/somepage.html"

SendToChromium should return the output of the command. So to get the html of the current page, one would simply do:

SendToChromium document.body.innerHTML > theBody.html

Ideally the browser would listen for this type of command on a local port. So instead of needing a binary 'SendToChromium' one could simply start Chromium in listening mode:

chromium --listen 12345

And then talk to it via http:

curl 127.0.0.1:12345/execute?command=location.href="/somepage.html"

<comment><USER6> While not currently "easy", there exists the Chrome Devtools Protocol.[0] I'm not aware of a CLI utility that communicates with it, but it wouldn't be impossible to make one that fulfills what you're looking for. A second tool could then act as a REST proxy, if calling the commands via curl is really your jam.

I think you've given my weekend some purpose. Lemme see what I can pull together...

[0] [https://chromedevtools.github.io/devtools- protocol/](https://chromedevtools.github.io/devtools-protocol/)

<comment><USER5> Yes, that might work. Maybe an even better approach is to use chromium- chromedriver.

I just got it working like this:

        apt install chromium-chromedriver
        chromedriver

This seems to create a service that listens on port 9515 for standardized commands to remote control a chromium instance. The commands seem to be specified by the W3C:

[https://www.w3.org/TR/webdriver/](https://www.w3.org/TR/webdriver/)

I got it to open a browser with this curl command:

        curl  -d '{ "desiredCapabilities": { "caps": { "nativeEvents": false, "browserName": "chrome", "version": "", "platform": "ANY" } } }'  http://localhost:9515/session

I have not yet figured out how to send javascript commands though.

</comment></comment></comment><comment><USER7> Interesting but seems less powerful than my current setup:

\- I have mitmproxy to capture the traffic / manipulate the traffic

\- I have Chrome opened with Selenium/Capybara/chromedriver and using mitmproxy

\- I then browse to the target pages, it records the selected requests and the selected responses

\- It then replays the requests until they fail (with a delay)

I highly recommend mitmproxy, it's extremely powerful: capture traffic, send responses without hitting the server, block/hang requests, modify responses, modify requests/responses headers.

Then higher level interfaces can be built on top, Selenium allows you to load Chrome extensions and execute Javascript on any page for instance. You can also manage many tabs at the same time.

I could make a blog post/demo if people are interested

<comment><USER8> A blog article would be nice. Sounds interesting, but I'm having a hard-time understanding. If it's replaying requests, how do you get it to do things like go to the next pagination and click on all of the next paginated results?

<comment><USER7> In my case I can't do the pagination automatically so I have to fetch the pages myself to then have them replayed.

In most cases you would capture the request and change the "page=" parameter (either for an HTML page or an API).

You could also use selenium to click on each "next page". Could be parallelized with multiple tabs / windows.

The only website that blocks me is Bloomberg because they detect mitmproxy (I didn't care enough to make mitmproxy harder to detect).

Another detail is that regular Chrome doesn't let you load insecure certificates while chromedriver allows that.

Anyway, I will write about all that, I already posted some code on my Twitter: [https://twitter.com/localhostdotdev](https://twitter.com/localhostdotdev) (that I will turn into a blog).

</comment></comment></comment><comment><USER9> I'm going to plug my app that does scraping with Electron: [https://github.com/CGamesPlay/chronicler](https://github.com/CGamesPlay/chronicler)

To the commenters who don't understand why this is necessary:

\- It reliably loads linked resources in a WYSIWYG fashion, including embedded media and other things that have to be handled in an ad-hoc fashion when using something like BeautifulSoup.

\- It handles resources loaded through JavaScript, including HTML5 History API changes.

<comment><USER10> Could you explain what electron offers here over for example a browser plugin? I'm not that familiar with limitations of WebExtension APIs

It looks like an interesting project but only for a few selected sites. For more random browsing I believe I would be too security conscious ([https://electronjs.org/docs/tutorial/security](https://electronjs.org/docs/tutorial/security)) to allow it

Might be unlikely that some random script on a random site will target Electron but you never know

<comment><USER9> Well in Chrome you can't hook into the network layer to record/replay requests like I've done here (you could fake it by overriding the ServiceWorker, possibly, but this feels brittle and I'm not sure if it's possible either). I'm not familiar with Firefox extensions but I'm given to understand you likely could implement this project as a Firefox extension.

Locking down Electron apps to be safe on the larger web is certainly one area where I think electron could do a lot better. I think my project has followed all of the recommendations and should be safe, but I agree with you that it feels like a bigger attack surface. I personally wanted it to support offline browsing of documentation sites, which are generally pretty "safe" from that perspective.

</comment></comment></comment><comment><USER11> Are there any other advantages over things like webdriver or puppeteer?

<comment><USER12> Not really.

I also have no idea what cheerio brings to the table here.

Seems like a hefty solution to web scraping</comment></comment>


The next big blue collar job is coding - <AUTHOR> https://www.wired.com/2017/02/programming-is-the-new-blue-collar-job/ <comment><USER2> Ok this article actually gripes me quite a lot due to its wrongful interpretation of what a blue and white collar job is. TBH I read it whilst I was on the toilet so I'm going to go as far as to say it gave me the shits.

Unless we are going to start referring to or reclassifying typing on a keyboard for 8 hours a day as strenuous physical activity (which it is not), coding will never be a blue collar job. Blue collar jobs by definition are classed so by the large amounts and types of physical exertion required to perform them along with the location in which that job is undertaken (mostly outside or in workshops). White collar jobs on the other hand refers to work that is generally of a cognitive nature and doesn't require much in the way of physical labor and are generally done in offices(don't get me wrong blue collar jobs require cognitive processes too but they also involve PHYSICAL LABOR as a KEY component).

The article would be better if it had said "blue collar work is going to be replaced by white collar work as the pillar of civil middle-class society". The title of the article currently is just plain wrong unless they are incorrectly referring to white and blue collar as some form of reference to a class system/exclusivity system based upon how many workers perform a certain type of task (e.g white collar jobs only employ a small high earning portion of the population whilst blue collar jobs employs the masses on average incomes).

<comment><USER3> This is a very excellent point. Blue collar means manual labor, while white color jobs are anything professional.

Heck, most of the developers I know rarely even wear a collared shirt to begin with.

<comment><USER4> Please don't imply that labor that is predominantly manual cannot be professional. A better dichotomy would be: Blue-collar means manual labor, and white-collar means mental labor (with a spectrum between both poles).

<comment><USER3> I'm not implying anything... I'm going by the dictionary definition.

Wikipedia link has good references, if you care to look them up. I agree that you can be a "professional" who does a lot of manual labor, but the blue/white collar dichotomy is rather established in definition. [https://en.wikipedia.org/wiki/Designation_of_workers_by_coll...](https://en.wikipedia.org/wiki/Designation_of_workers_by_collar_color)

> A "white-collar worker" is a salaried professional...

</comment></comment></comment></comment><comment><USER5> From Scott Adams:

"If you want an average successful life, it doesn't take much planning. Just stay out of trouble, go to school, and apply for jobs you might like. But if you want something extraordinary, you have two paths:

1\. Become the best at one specific thing. 2\. Become very good (top 25%) at two or more things.

The first strategy is difficult to the point of near impossibility. Few people will ever play in the NBA or make a platinum album. I don't recommend anyone even try.

The second strategy is fairly easy. Everyone has at least a few areas in which they could be in the top 25% with some effort. In my case, I can draw better than most people, but I'm hardly an artist. And I'm not any funnier than the average standup comedian who never makes it big, but I'm funnier than most people. The magic is that few people can draw well and write jokes. It's the combination of the two that makes what I do so rare. And when you add in my business background, suddenly I had a topic that few cartoonists could hope to understand without living it.

I always advise young people to become good public speakers (top 25%). Anyone can do it with practice. If you add that talent to any other, suddenly you're the boss of the people who have only one skill. Or get a degree in business on top of your engineering degree, law degree, medical degree, science degree, or whatever. Suddenly you're in charge, or maybe you're starting your own company using your combined knowledge."

[http://dilbertblog.typepad.com/the_dilbert_blog/2007/07/care...](http://dilbertblog.typepad.com/the_dilbert_blog/2007/07/career- advice.html)

<comment><USER6> Becoming moderately good at two things is how I ended up where I am. Before I went to college I did freelance graphic design. In college I ended up studying computer science, simply because that's what my brother majored in. The two combined meant I was getting job offers in my sophomore year. I can't stress just how much it helps when I'm applying for jobs - not only do I have a wider selection (design and programming jobs), but it also opens up niche jobs which are typically in high demand and low supply (prototyper, my current role). Scott's advice here is spot on and is the first thing I recommend to people when they're looking to prepare to enter the workforce.

<comment><USER7> I have a similar skill set combining graphic design and programming, but I find those niche jobs of prototyping to be in very short demand, where are you seeing the opposite?

The lack of opportunity to flex my creative muscles actually drives me to create my own startups because it's the only type of project where I feel I can utilize all of my strengths instead of being pigeonholed into a developer role.

</comment></comment></comment><comment><USER8> Why coding and not mechanical engineering? MEs often have jobs that could be considered blue collar if not for their degree: designing, installing, & building the same equipment that many blue collar workers use or maintain.

It's because coding is seen as a trivial job despite requiring the same amount of education as an ME. Society has this imagine of 14 year old hackers that can do it with minimal training. So it's easy to convince them that some mechanic that lost his job can retrain to become a coder, because, if a 14 year old could do it...

Computer Science is poised to get more difficult in the future, not easier. New companies/technologies will come along offering products that eliminate the need for low-skilled programmers, leaving the more difficult, high-skilled jobs. The tasks that get easier will no longer require a specialized developer, they can instead be done by a person with an understanding of that specific domain. Leaving the rest of the programming jobs a little bit more difficult. (I think the skewed ratio of senior to entry level programming positions is evidence of this phenomenon in action)

I got my start as a teenager doing small e-commerce sites for people, but I'm not sure the "dedicated developer for a small e-commerce site" job exists much anymore. They have been replaced by Shopify, WIX and even Amazon. And once these companies out-grow their platforms, they need more experienced people to build something more robust.

I think we are doing society a disservice by convincing people that anyone can A) learn to sling some JS, and B) make $80k a year doing it. Easy jobs that anyone could do are not jobs that pay 2-3 times the median salary of a region.

<comment><USER2> >> but I'm not sure the "dedicated developer for a small e-commerce site" job exists much anymore. They have been replaced by Shopify, WIX and even Amazon.

Mate I thought this was the case until over the last 2 years I have met a number of mum and dad "coding" companies that literally make their bread and butter from building websites using these platforms and shit like wordpress. I was actually blown away that a few of them had managed to make enough to maintain/pay off houses in Sydney (and in nice parts of the city too) and still have enough to buy a bimmer and a audi to boot. Like considering over the last few years I have been trying to get myself out of the laboring market and into I.T, I had to actually stop my jaw dropping in obvious exclamation when I found out these people support their lavish lifestyles providing products that are literally DRAG AND DROP forms of production and not exclaim "you get paid for doing that shit?!" its been a real eye opener...there are plenty of luddite's out there willing to pay for crap.

<comment><USER9> Developers need to get out of their bubble and realize that even a basic website is sorcery for 99% of the population. I'm thankful that I started off in online marketing and then learned to code because I remember what it was like to know nothing and can see opportunities that people like yourself overlook.

This post reminds me of the infamous HN dropbox comment in terms of how out of touch it is.

<comment><USER10> The downside of the 'sorcery' impression is that they actually think it's magic and once you make magic happen, you can do anything. There are no engineering trade offs to make. Your request isn't unreasonable or infeasible. If you don't cast spells for me, I'll look around for someone who will (try). Cheaper.

</comment></comment></comment></comment><comment><USER11> There has been a "blue collar" coder for a long time, maybe always. It's all about "sandboxes", by which I mean the area the programmer is concerned with. If the sandbox of a programmer's job is in one language, say Java, and it's only writing unit tests to test other people's code, the programmer is very limited in what they are responsible for, and the cost of their mistakes is very low. Yet just writing unit tests that cover code, even if they aren't brilliant tests, certainly has value. And the number of people who can be taught to write unit tests is much higher than the number of people who can be taught to write the application itself. And the person is still a professional programmer, but they are paid less, and are less talented than other programmers. But they still make decent money and likely find this work more intellectually stimulating than selling insurance or whatever.

I think a lot of people who are professional programmers find programming itself scary. Programming is a hard thing to do, and ace programmer's are very opinionated, smart, logical, and can spot non-ace programmer's pretty easily. Or maybe it's just that people are naturally lazy and don't want to get out of their comfort zone. Whatever the reason, many programmers work in one language, in one product, then try and stay in it forever. Then they get laid off after 10 or 15 years and find it quite hard to find a new job.

When hiring people, I'll come across this type of resume, where they focused in some super-niche proprietary tech for many years and are unfamiliar with the latest trends, and usually are asking for a below-average salary. I have interviewed a few, and their lack of talent is so glaringly obvious I feel bad for them. But someone will probably hire them, probably the companies that think that all programmers are replaceable widgets.

In summary, "blue collar" programmers already exist, it's just the author who thought he made some stunning realization, which isn't new.

<comment><USER12> > If the sandbox of a programmer's job is in one language, say Java, and it's > only writing unit tests to test other people's code, the programmer is very > limited in what they are responsible for, and the cost of their mistakes is > very low. Yet just writing unit tests that cover code, even if they aren't > brilliant tests, certainly has value.

Not disagreeing with your overall point, put I'm fixing some terrible unit tests (that aren't really unit tests) right now that do nothing but add negative value. So many otherwise experienced programmers write terrible unit tests and/or untestable code that I don't think it could ever be silo'd like that.

</comment></comment><comment><USER13> While I'm all for coding literacy and job retraining, I think the hype around code schools misses the fact that the field requires extremely deep knowledge along with curiosity. A few months of an intensive code bootcamp can get you the basics of writing some JavaScript, basic git workflows, and maybe some API development on top of a database, but it does nothing to develop the intuition a lot of us developed over years and years of tinkering.

I've seen lots of code bootcamp grads get completely lost when deploying an application to a Linux environment. Or when they have to figure out what's going on with a malfunctioning system. Or when they have to think about networks. Or when they encounter performance issues in their code.

Then I realize that a lot of my peers that I pair with were like me. They broke the school computer and fixed it when they were 12. They spent their summer job money to build a computer from parts because they were curious at 15. They taught themselves to code, then got better at college or through continued self-study. They ran Linux as their desktop for fun.

Now I'm not saying that bootcamp coders can't get there, but it's hard to compare to someone that at 21 had 10 years of autodidactic learning experiences, maybe with a college degree on top of it. I'm all for being more inclusive and we really should be making efforts to get more people involved, but I think we forget how broad of a domain we work in and how long it took us to get to our skill level.

</comment><comment><USER14> A presumption of this article, and many like it, is a backwards-looking and somewhat elitist view of the economy and the workforce.

Consider these quotes:

* "What if we regarded code not as a high-stakes, sexy affair, but the equivalent of skilled work at a Chrysler plant?"

* "teachers and businesses would spend less time urging kids to do expensive four-year computer-science degrees and instead introduce more code at the vocational level in high school. You could learn how to do it at a community college"

The problem is that manufacturing jobs, or their equivalent, is going backwards. We want to go forwards, improve opportunity, income and quality of life for each generation. We want higher-paying, higher-skill jobs.

Rather than find ways to not go to college, what if we continued the trend of each generation being more educated than the last? What if we sent people to college? Sanders proposed college for all, arguing that high school was sufficient a century ago but now college is needed. Clinton proposed something similar, free college for families making <$125K/yr (IIRC the details); she got the majority of votes (which doesn't qualify her for President, but does say something about the popularity of her ideas).

The idea that 'college isn't for everyone' is elitist. It really means that 'college isn't for the children of the working class, only those of the wealthy'. Think about it: Will the children of Senators go to college? I think so. The children of most people reading this? Of SV millionaires? Of Apple and GM executives? Do they tell their kids, 'college isn't for everyone; learn a trade at community college'? Somehow college _is_ for all their kids (with few exceptions), so the evidence seems strong that with the proper resources, it's for everyone.

<comment><USER15> >The idea that 'college isn't for everyone' is elitist. It really means that 'college isn't for the children of the working class, only those of the wealthy'.

I actually think you're right that it's elitist, but I don't think this "college is not for everyone" attitude is about money, but about intelligence and personality characteristics. These things happen to also correlate with "wealthy," for values of wealthy above "working class."

Some people just aren't going to be able to make it through college no matter how many resources you give them. And college as we do it now is not right for everyone either. For evidence of this, go to your local state run university and observe how many students are failing out of school because they prefer to smoke weed every day to studying. I think something like 70% of people I met in my freshman year failed out of school in this particular way. Free college wouldn't have helped them.

<comment><USER14> > this "college is not for everyone" attitude is about money, but about > intelligence and personality characteristics. These things happen to also > correlate with "wealthy,"

You're claim is that children born into wealthy families are naturally more intelligent? And that 70% of people in state universities flunk out, and they flunk out due to marijuana usage?

Hmmm .... I think we'd need to see some basis for these theories.

<comment><USER15> No, I'm claiming that intelligent, conscientious people are more likely to be wealthy. There are many studies that support this.

I'm also sharing an anecdote that many people who have the opportunity and resources to go to college, waste it. That's an anecdote from my own experience of going to a state university. The point of sharing it wasn't to tar everybody who goes to state universities, but to point out that there are people who will waste opportunities that they're given.

<comment><USER14> > intelligent, conscientious people are more likely to be wealthy

We're talking about kids; their access to wealth depends almost entirely on their parents, not the child's own intelligence and conscientiousness.

</comment></comment></comment></comment></comment><comment><USER12> Considering the quality of the code I see currently and the costs created by that technical debt I'm not sure if becoming less elitist will be a good thing. Instead of all these code camps, I can't help but wonder if we could get 90% of the value and 10% of the problems if we taught people shell scripting instead of programming.

It looks like wireds ad-blocker blocker has been circumvented now at least.

<comment><USER16> The nice thing about software projects is that they're easily replaceable. They can be treated as any other business asset in that you can hire a shop to develop them or do it in-house. So long as you have enough business eyes on the process then it works. Expensive, but it works. Businesses are used to expensive.

The real value of good code is way, way less than developers would like to believe. This is coming from someone who is damn near a perfectionist about code. I have to constantly check myself and ask whether whatever I'm demanding we spend time and effort on is really worth it from a business perspective. Occasionally it is, most of the time it's not.

If coding wasn't the thing I could make the most money with at this point in time, I'd do something else and code free software as a hobby. At least free software really benefits from great code practice.

<comment><USER12> I've never seen management buy in for a rewrite, especially when the code is so bad that the company starts circling the drain. The only time I've seen it is when there's been a fundamental technology shift, like from VB6/C++ to java/.net.

Network effects also make it hard. Sometimes you can't upgrade X because A, B and C feed into it and D, E and F feed out of it and they're all tightly coupled (another thing that's even worse with less skilled programmers). Replacing a single component can mean rewriting the software stack of the whole company.

> If coding wasn't the thing I could make the most money with at this point in > time, I'd do something else and code free software as a hobby. At least free > software really benefits from great code practice.

Completely agree here. The money is the only thing keeping me in the industry, if that went away I'd do something much more blue collar.

<comment><USER16> Sometimes I wonder if the money is the only thing that keeps me coding. It seems too hard to do well otherwise.

</comment></comment></comment></comment><comment><USER17> Most 'blue collar' jobs are probably about 1000% more complex than 'white- collar' workers realize.

This discussion is about social class more than it is about practicalities.

There is a divide between the ownership class and the worker class, just as there has always been. The owners always find ways to justify their unfounded belief in their superiority because that is necessary to rationalize the unequal distribution of profits and unfair burden of work.

The 'coders' are the intelligent and capable hard workers. The owners, managers, 'entrepreneurs' are the ones that happen to have enough money to avoid doing real work.

This article is largely just another caste justification.

</comment><comment><USER18> This is actually how programming started in the first place, and still is. It's only because of highly publicized Valley-style successes, mysticism ("these twenty-somethings wrote an app and got rich? Surely they are geniuses!") and remnants of the dotcom era that programming still has a glamorous air.

Programming used to be women's work (i.e. "low level" work, similar to typists according to 1960s stereotypes), and programmers programmed a business' computers to do whatever tasks they wanted to accomplish. The vast majority of companies in the world do not have software as a core business, so this still applies to a large extent.

</comment><comment><USER19> I'm sure large corporations would love some cover to depress wages. The reality is the blue-collar jobs the article references had low barriers of entry, and provided good salaries. SE jobs with good salaries rarely go to boot-camp graduates, and with a glut of CS majors on the market, there's no reason that'll change.

</comment><comment><USER20> I'm going to make an assertion that will like make some of you angry. Let me start with this... Life is a bell curve. Everything you observe from weather to color to taste to whatever, measured multiple times across a large sample tends to land into a bell curve. Now we all know this also applies to people's intelligence. We tend not to emphasize it, because as empathetic members of society we know categorizing people can lead to unethical behavior. But ignoring the spread of human intelligence does not make it false. So what does this have to do with the article? Think back to high school where we were all constantly reminded of our ranking amongst our peers. Some got A's, some B's, most got C's, then the D's and F's. Now think of the friend you had that fell into the D/F group. Which do you think they are better suited for? Pouring concrete or writing secure Javascript logon pages? Let's move across the bell cure into the C's. Do you think they are better suited to hanging sheetrock or writing discrete transactional SQL in a multithreaded environment? See what I mean? Now we move to the B's, a much smaller group. They can likely code, but do they have the aptitude? I know people consistently making the Dean's List in college who can barely understand variable assignment, let alone pointer arithmetic or referential integrity. So to say that we just shift the "blue collar" workforce from "industrial" tasks to writing code, I can't see it.

</comment><comment><USER21> I've had this thought before. What's software development if not a trade? I don't want to say that work gets dumbed down, but one hundred years ago a store clerk was a solidly middle-class job, now it's not.

Maybe a more relevant example. My grandfather was an accountant. What he did for most of his career, bookkeepers do now. What he did at the end of his career was basically law and banking. The electronic spreadsheet took care the gruntwork he used to do

<comment><USER14> What's the difference between a system administrator and a plumber? I don't mean it as the first line in a joke.

<comment><USER12> The plumber can't execute change-washer.sh and have his job done in 1 second. The plumber can't use "down time" to proactively solve future issues. A sysadmin does have a higher level of abstraction available to them.

<comment><USER22> > The plumber can't use "down time" to proactively solve future issues.

Oh, but he surely can, replacing a valve that is known for going to break in near future for instance, or putting a different one with additional safety features.

> A sysadmin does have a higher level of abstraction available to them.

Mainly because sysadmin works with immaterial things (software) (and some of sysadmins also work with hardware).

Sysadmins really have similarities with plumbers.

<comment><USER12> > Oh, but he surely can, replacing a valve that is known for going to break in > near future for instance, or putting a different one with additional safety > features.

Doing that change will cause downtime. The sysadmin can write a script to make the change and avoid the downtime.

</comment></comment></comment></comment></comment><comment><USER23> This seems to depend heavily on the specific role and type of software! Building LLVM, cutting edge vision systems, self driving cars, sublinear time algorithms, ..., all probably take a bit more study than the standard definition of blue collar would demand.

<comment><USER24> As a counter example of what is hard or not, a bit more study is easy compared to 'the standard definition of blue collar'. As an example. Pick up a 50 pound box. Walk ten feet with it, and set it down. Repeat. For 12 hours a day, 6 days a week. That makes advanced systems design, such as self driving cars look trivial because theres a ton of talent lifting one enormously complex box, one time - then it ships.

</comment></comment><comment><USER25> >But any blue-collar coder will be plenty qualified to sling JavaScript for their local bank.

Yeah, that sounds dangerous.

<comment><USER12> Do you have any idea how many billions of dollars are moved around everyday by excel and VBA scripts? Sometimes I've replaced these with real code and they complain it doesn't replicate the VBA errors.

</comment></comment><comment><USER26> I wonder where this fits on the continuum to end-user programming: [https://en.wikipedia.org/wiki/End- user_development](https://en.wikipedia.org/wiki/End-user_development)

One can see Rails and the later enthusiasm around Node driven by the idea that people could come to be programmers, not necessarily through a dedicated process of education in computer science.

</comment><comment><USER27> The big problem with this is that we're already competing to some extent with outsourcing shops in China and India and this trend is more likely to grow than not. While I believe there will always be an incentive to keep these jobs in-house and local, there is no reason to think it will grow to the extent that middle class blue collar work did through the industrial revolution.

<comment><USER28> I work in China as a programmer and the local outsourcing shops tend to throw tons of very cheap labour at anything. Unless you're building critical or sensitive systems this model can work (even for physical products and civil engineering if "QA'd" enough through brute force). It's a complete pain in the ass for the in-house team who have to pick up the pieces and maintain the code, though.

<comment><USER27> I've worked on code that came from those shops (though maybe India more than China) and I completely agree. But it's still going to be hard to make the case for coding to expand the way auto manufacturing did, for example, when we're already outsourcing a huge chunk of work. And there's no reason to think they won't get better at the work there in any case.

</comment></comment></comment><comment><USER29> I found the statistic interesting that for 11 jobs there were 950 applicants. How does this sort of competition compare to traditoonally blue collar jobs?

</comment><comment><USER30> I wish I could just manage a login page all day, sheesh</comment>


Google 'to end' Pentagon Artificial Intelligence project - <AUTHOR> http://www.bbc.com/news/business-44341490 <comment><USER2> Already discussed today:

[https://news.ycombinator.com/item?id=17208892](https://news.ycombinator.com/item?id=17208892)

[https://news.ycombinator.com/item?id=17209691](https://news.ycombinator.com/item?id=17209691)

</comment><comment><USER3> "executives saw the contract as a huge opportunity while being concerned about how the company's involvement would be perceived"

Pretty damning they are concerned about how they may look to the outside world, instead of being actually worried about the consequences of being complicit to the development of a new breed of killing machines, and therefore also to how they will be used.

<comment><USER4> So now the motto is "do not look evil".

That is a pretty good ethical principle.

</comment></comment><comment><USER5> So they happily continue for the next 10 months. Well played.

<comment><USER6> Don't they have a contract with the federal government for the current period? Are you suggesting they should be breaching it?

<comment><USER7> They should just try to cancel the contract as soon as possible.

</comment></comment></comment><comment><USER8> Don't worry, other companies and/or militaries will do that

Do you think China will have any qualms about making selling AI enabled drones?

I'm not saying Google is doing the wrong thing, but there are other players to do the same thing.

<comment><USER9> None says US should not use high tech, just that an international company like Google doing this will have big consequences to it's business abroad and inside US and people , employees and customers have expressed their right to tell Google management they don't want to be involved with this.

I personally think is a bad idea that a big company like Google to be associated with military,NSA, CIA , it will end up with losing customers outside the US and even in US, like the Dropbox image issue where some people quit using it (I don't remember exactly the names involved)

</comment></comment><comment><USER10> 1e^100 kills

<comment><USER11> More effective warfare in the right hands would kill less people. We're fine when America keeps the military upper hand, we're fucked if nations who care less about keeping people alive do decide that using neural nets for warfare is a sound idea.

<comment><USER12> [https://en.wikipedia.org/wiki/List_of_countries_by_military_...](https://en.wikipedia.org/wiki/List_of_countries_by_military_expenditures)

[https://en.wikipedia.org/wiki/Arms_industry#World's_largest_...](https://en.wikipedia.org/wiki/Arms_industry#World's_largest_arms_exporters)

Do you think this technology will really "disrupt" which nation is both the largest manufacturer and consumer of weapons?

I think who the "right hands" are to handle military power is something that reasonable people can (and do) disagree on...</comment></comment></comment>


Abraham Wald's Work on Airplane Survivability (1984) [pdf] - <AUTHOR> https://people.ucsc.edu/~msmangel/Wald.pdf <comment><USER2> It's a work that people love to make Youtube videos out of, and is interesting. "Where are the missing bullet/flak holes?" is the question people should be asking, rather than just armoring the places that are obviously bullet-ridden. The paper is much more technical than you would think from the popular tale (Lagrange multipliers, etc).

I believe they did other story-worthy statistical feats, like estimating the size of the German tank fleet by observing the serial numbers of captured tanks.

<comment><USER3> Aahh, yes, the German Tank Problem[1]. Also more complex than it would initially seem.

[1] [https://en.wikipedia.org/wiki/German_tank_problem](https://en.wikipedia.org/wiki/German_tank_problem)

</comment></comment><comment><USER4> I read about this in Daniel Kahneman's "Thinking Slow and Fast" where this was listed under possible "suvivorship biases" i.e. we draw conclusions from data that survives but forget about the unseen data. For example, we read stories about how someone with a simple startup idea became a millionaire but never hear about those who tried and failed hence creating a biased view on startups.

</comment><comment><AUTHOR> (submitter)

Wald's WWII memos were reprinted in the 1980s and are available here[0].

[0] [https://web.archive.org/web/20140814122810/https://www.cna.o...](https://web.archive.org/web/20140814122810/https://www.cna.org/sites/default/files/research/0204320000.pdf)

</comment><comment><USER5> I study WW2, and the US had such fundamental problems with bomber design that you could say they were of doctrine in nature.

The B-17 "Flying Fortress" had lots of guns and a crew of 10, but couldn't defend itself against German fighters. It's payload ended up being the same as the 2-man British Mosquito bomber.

The B-29 was the first pressurized bomber, yet that just made it expensive to build since bombing accuracy at 20k feet is random, and Curtis Lemay ended up using it at 5k feet, with napalm.

<comment><USER6> > The B-17 "Flying Fortress" had lots of guns and a crew of 10, but couldn't > defend itself against German fighters. It's payload ended up being the same > as the 2-man British Mosquito bomber.

To make a more apples to apples comparison, one should replace the B-17 with the British heavy bombers, who often flew on the same missions as the Mosquito. The British heavies were similar to the American ones (though somewhat higher bomb load and less defensive guns). The interesting thing is that the loss rate for the Mosquito was an order of magnitude less than for the heavies (which was beyond appalling). It was just so fast that it was very difficult to catch for the Luftwaffe night fighters, and of course by being faster it spent less time in flak barrages as well.

Of course one could argue that the slow heavies presented much juicier and easier targets for the Luftwaffe and the flak, so they didn't bother going after the Mosquitos. And thus if Bomber Command had switched from heavies to fast unarmed ones like Mosquito the loss rate would have climbed. Still hard to see how it could have reached the levels it factually did for the heavies.

An interesting tidbit here is from Freeman Dyson, who served as an analyst in Bomber Command during the war. He had calculated that the loss rate would be reduced by removing all, or most, defensive armaments. This was based on reduced drag and weight increasing the speed, and also that interviews with aircrew who had been shot down and (miraculously) survived and made it back to England revealed that most never knew what hit them. But this proposal was shot down, as the myth of the heroic gunners protecting their mates was so strong.

<comment><USER7> _interviews with aircrew who had been shot down and (miraculously) survived and made it back to England revealed that most never knew what hit them_

The particularly depressing part of this story is that for months RAF crews were reporting attacks they couldn't see coming, and the RAF ignored these reports and chalked these losses up to flak. What _was_ happening was the Luftwaffe had observed that RAF heavy bombers lacked belly gun turrets and had essentially zero downwards visibility, and had developed the upwards-facing _Schräge Musik_ gun mounts to attack from below. What was also happening was that RAF bombers were being equipped with _Monica_ tail warning RADAR in an attempt to detect pursuing night fighters more easily, but the Luftwaffe had quickly developed detectors that allowed night fighters to directly home in on these bombers instead, while still avoiding being detected by _Monica_ sets. Together, these were responsible for much of the severe losses suffered by Bomber Command in the winter of 1943-1944.

<comment><USER6> > Luftwaffe had observed that RAF heavy bombers lacked belly gun turrets and > had essentially zero downwards visibility, and had developed the upwards- > facing Schräge Musik

My understanding was that the point behind attacking from below wasn't so much the bombers lacking a belly turret as the bomber being silhouetted against the night sky, whereas detecting a plane by looking downwards towards the dark earth was almost impossible.

And yeah, the history of early electronic warfare is fascinating. All the things they came up with, and the counter-measures developed amazingly quickly etc.

<comment><USER7> There were plenty of other reasons to attack from below - interest in the tactic predates the war - but I don't think the tactic would have been anywhere near as effective if RAF heavy bombers had been equipped with belly turrets (and armed with something better than the basically useless .303 machine guns). Entirely different tactics were used against the USAAF formations in daylight, where a slow overtaking approach from below would be far to vulnerable.

 _whereas detecting a plane by looking downwards towards the dark earth was almost impossible_

This would be true on the long approach to the target, but once near the target area, bombers would be silhouetted from below by searchlights, flares, and the burning city beneath them. Enough light was available from these sources for the Luftwaffe to use _Wilde Sau_ single-seat "day" fighters at night.</comment></comment></comment></comment></comment>


What's the ultimate way to defy depression, disease and early death? Exercise - <AUTHOR> https://www.theguardian.com/lifeandstyle/2017/oct/03/exercise-depression-disease-death-sit-less-move-more <comment><USER2> I reversed diabetes with diet alone by doing my own research[1], taking charge of my health. I lost 180lbs in the process. No exercise (disabled). Very much a work in progress.

I'm not rejecting the importance of exercise for health - I follow the general principal of eating to get healthy, exercising to get fit - but I think we skip over the absolute tragedy that is the Standard American Diet at our peril.

Recovering from the consuequences of three/four generations of preaching the lipid hypothesis[2], as fast as we damn well can, is the ultimate way to defy depression, disease and early death.

[1] [http://www.thefatemperor.com/blog/2015/9/5/fat-emperor- produ...](http://www.thefatemperor.com/blog/2015/9/5/fat-emperor-productions- present-the-kraft-interview-decoding-diabetes)

[2] [https://en.wikipedia.org/wiki/Lipid_hypothesis](https://en.wikipedia.org/wiki/Lipid_hypothesis)

<comment><USER3> I'm on the same track as you, I've lost about 80lbs so far and I've done zero exercise.

I've found that when I tell people this, they worry about my health even more than they did when I was 80lbs heavier. It's bizarre.

Never mind the fact that my blood pressure is now a pristine 110 over 70, all my pre-diabetic symptoms have disappeared, my sleep apnea is cured, my gastroesophageal reflux disease is completely gone, my cholesterol and triglycerides are back to normal, etc.

The fact is exercise has an almost negligible impact on weight loss.

Let's be very generous and say an hour of typical exercise burns 350 calories. If you're overweight, compare that to the 120 calories you burn per hour spent sitting in your chair. That's an extra 230 calories you burned by exercising for an hour.

That 230 calorie expenditure costs you:

* 1.5 to 2 hours of disruption to your daily life

* An hour spent being extremely bored and uncomfortable

* An ever-growing amount of generalized aches and joint pains as you put together consecutive days of doing this

And at 230 calories per hour, doing an hour of exercise every single day nets you 1610 calories per week, which is less than 1 pound of fat loss every 2 weeks. You turn your whole life upside-down, make yourself miserable and in pain every single day, take significant time away from work and studying and family, and all you get is LESS THAN 1 additional pound lost every 2 weeks.

I honestly think the net impact of promoting exercise as a weight loss option is people getting and staying fatter. People have a tendency to dramatically overestimate the calories they're burning, people want to use exercise as a way to avoid having to eat less, and people use exercise as an excuse to eat more. It also causes an amount of disruption and discomfort in people's lives that's probably the number one cause of people giving up on their weight loss goals.

As far as health impacts, I'm still not convinced on the cause-and-effect here. I want to start doing some physical activities soon, but that's not going to cause me to be less depressed, that's a _result of_ me being less depressed because of all the weight I've been losing. All the studies I've seen on exercise and health draw the same sort of "playing basketball makes you taller"-conclusion.

<comment><USER4> > An hour spent being extremely bored and uncomfortable

In the beginning, this is often true. But exercise is addictive! When your body gets used to it, you really want it and it is not uncomfortable. That "runners high" is also not a myth.

<comment><USER5> I'm willing to grant that this must be true for some people. However, I don't think it's true for everyone, or maybe even most people. In the early 90s I was in the US Army for a couple years, and exercise never got better: it was always a struggle: boring, painful, and stressful. I suppose one good thing about it was that it was very rare that anything worse happened to me later in the day... :)

Looking around at the PT groups I was in during that time, I do not think most of them were looking forward to it.

<comment><USER6> I've never been able to reach a point where weight training is anything but boring and painful. The trouble is it requires mental focus, and I cannot think about other things while lifting.

I try to mitigate the boredom by watching a movie while doing it.

<comment><USER7> I listen to headphones, music, podcasts, ebooks. I find it a very stimulating time and I feel better about it at a gym, even though I don't socialize much.

Just seeing the same people every day makes me feel like a community member and helps me stay motivated and connected.

</comment></comment></comment></comment></comment></comment><comment><USER8> I think it's amazing the way we've completely sidelined physical activity - not exercise per se, but any physical activity - in our western lives. Unless you live without a car, there is no requirement for you to walk more than 1000 steps during the day. Wake up, drive to work, sit at your desk, drive to a restaurant for dinner, drive home, plop down on your couch. Rinse, repeat.

Even if you do decide to work out, it tends to be the first thing you drop when you don't have time. Or you worked late. Or you forget your gym clothes. Or you're just too exhausted to go.

We used to be active, our jobs required motion, we had to walk or cycle to work, or even take public transportation - which requires you to at least walk to the stop and back.

Now we literally sit on our asses, day in, day out, and wonder why so many people are obese.

<comment><USER9> This sounds particularly American, rather than for the entire west.

<comment><USER8> We like to think so, but there are swaths of Western European where this is true. Replace the Dodge with a Citroen, the Walmart with Tesco, and the Olive Garden with a pub. Even in the most pro-bike and public transportation nations in Europe 20-30% of adults drive to work.

</comment></comment></comment><comment><USER10> Just to avoid confusion, not _all_ clinical depression is _fully_ treatable by exercise; I've known several people who were severely depressed while also being non-car-owning regular cyclists.

Sometimes antidepressants are required. Sometimes therapy is required. Please avoid turning this into "why don't you just exercise".

<comment><USER3> You can see this if you just look at the last few hundred years of human history. Humans used to have to engage in hours of labor every day just to survive. If the sun was up you were exercising. Were they so much happier and healthier?

There are people in the world who still have to do that labor. They're not happier or healthier or longer-lived than the western world. In fact, the overall high-level trend appears to be that happiness is correlated with being sedentary.

Correlation doesn't mean causation of course, but as long as articles like this one are going to imply that it does, I might as well point out that I can make the exact same argument in the opposite direction.

<comment><USER11> > _They 're not happier or healthier or longer-lived than the western world. > In fact, the overall high-level trend appears to be that happiness is > correlated with being sedentary._

Source?

<comment><USER3> This is completely self-evident and not even controversial, but here you go:

[https://upload.wikimedia.org/wikipedia/commons/thumb/0/01/Wo...](https://upload.wikimedia.org/wikipedia/commons/thumb/0/01/World_happiness.png/1200px- World_happiness.png)

Agrarian populations spend all their time doing hard labor to earn basic necessities like food, water and shelter. That sucks and they're less happy.

People in western nations live longer and move less. The correlation is pretty obvious.

<comment><USER11> That chart is not a source for your claim. Not really interested in arguing the point further.

</comment></comment></comment></comment></comment><comment><USER12> Just in case you need to read this again, exercise changed my life. It simply get rid of the fatigue. Dont take drugs to stay awake, go run half an hour (it really doesn't take more than that to feel tremendous effects).

<comment><USER13> I would rate my life satisfaction as at least moderate, but is nothing about being alive that could possibly justify the experience of exercise, particularly of running. Given the choice between a lifetime of regular running or death, I'd choose death without even blinking.

<comment><USER14> You would be missing out on some great experiences. Running is what humans are good at: our bodies are evolutionarily adapted for it. It makes you feel good. Yes, running is hard at first, if you're sedentary. But I can assure you that regular running is considerably more enjoyable than death :)

</comment></comment></comment><comment><USER15> Went from 320 pounds to 260. Back problems basically go away unless I have been sitting a really long time. After I go to the gym I feel great no matter what day I have had. Since I haven't died I will just rely on the stats in the article.

<comment><USER16> I did go back to sport, I used to be moving 6h+ per week then stopped. Amazing help. One thing I should do is swimming, the feeling of complete exhaustion and relaxation after is overwhelmingly good.

Also fixes your bad diet.

<comment><USER17> oh man that time when I'm exercising enough that my body starts _insisting_ I eat better is _the best_

<comment><USER16> I stopped eating shit 3 years ago, but since I started long jogging sessions, I can sense another level, I don't eat, I devour. Apple, beef, veggies, whatever, it doesn't spend more than 15 seconds in my mouth.

</comment></comment></comment></comment><comment><USER18> I agree with the article but put it differently.

To say exercise lowers depression, disease, and death normalizes inactivity.

You can define normal how you want, but I prefer to think of exercise as normal and that lack of it contributes to depression, disease, and death.

</comment><comment><USER19> Ah yes, let me just conjure up some motivation to get rid of my lack of motivation.

<comment><USER20> Well, depression is a positive feedback loop where a negative mental state leads to actions that amplify that state. To break the loop, something has to change.

<comment><USER21> Eating works :)

</comment></comment></comment><comment><USER22> There is a simpler explaination to this: If you spend a few hours doing some exercise (changing cloth, showering, going to the sport area) you are not sitting alone, as usual in front of a display or in your couche. It distracts your mind from your negative thoughts. Also beeing able to walk normally and without pain, going up stairs etc. is a good and normal feeling. When you get older and those basic things stop working, that really sucks. Or being overweight when 'normal' things become tedious.

</comment><comment><USER23> I'm a great fan of my daily walk. So not saying this is necessarily wrong.

But there's no theory! e.g. mental health isn't understood, so whether exercise is of lasting benefit in that department is unclear. Consider that people pay good money to _sit_ for a week or more to try to elevate their well-being. Consider that athletes have their fair share of mental health issues, and that even top athletes live less than 3 years longer than average. (Which seems like a small difference.)

Consider that people who move about constantly will eventually drop dead of heart failure:

[https://en.wikipedia.org/wiki/Dancing_Plague_of_1518](https://en.wikipedia.org/wiki/Dancing_Plague_of_1518)

Examples such as this are perhaps misleading edge cases. But theory would give some insight into _why_ they are misleading. At present we are blind.

</comment><comment><USER24> For those still unconvinced, I'll add another anecdote to this thread.

Morning exercise completely cured my intractable insomnia.

<comment><USER25> Morning exercise specifically? How about lunch?

<comment><USER24> Lunch works too, but the earlier and the harder the better.

</comment></comment></comment><comment><USER26> It's great to have a dog that will be walked every day on every weather. Ok if you have a cold it's not such a great thing but as a desk worker you must go out and walk at least one hour per day around.

<comment><USER27> I agree with this.

If you like animals and are responsible enough to have what amounts to a light version of a child, then dogs are excellent for encouraging activity since they require daily walks.

<comment><USER26> "light version of a child" is funny :) It's like a child but without the stress ;)

<comment><USER28> I agree and disagree. A well-treated animal does require attention and some effort, but I agree it causes no stress, to the contrary. But they're like children who never grow up and will always need you or _someone_ who loves them from the bottom of their heart when you're gone (maybe not so much with, say, lizards, but certainly with dogs and to a lesser degree cats). I don't mean this as a negative per se, I just needed to say that as soon as I saw "light version of a child". When my cats died, I thought what it might be like to have children, and I nearly swooned. But then I thought how much better it is for them to have lived a full life and die on me, than the other way around, and that's certainly not how it would feel with children.

</comment></comment></comment></comment><comment><USER29> I've moved from an apartment to a house, which means I have two staircases I traverse daily, a small lawn to mow, and a few other things like that.

It's amazing to me what even a slight bit of regular activity can do. It just feels good to be even a little active on a regular basis. I feel so much more ready to move.

</comment><comment><USER30> Exercise does nothing to lighten my mood. I hate exercise in virtually all forms. Never experienced the "runner's high" or any kind of apparent dopamine production (not saying it didn't happen, just that I don't ever feel better with exercise).

</comment><comment><USER31> This is a UK article, and I'm going to post a UK-centric, male-centric comment (as a UK-based man), but I suspect my comment applies to both genders, and to many health systems around the world.

I don't doubt exercise is extremely valuable, and that everyone should be following the advice in TFA. However, I am starting to think that lifestyle recommendations are a way for a public health system to wash its hands of its responsibility to actually, proactively care for people. As in: "Well, we're sure you can make changes on your own to improve your health, so just go and do your bit while we quietly neglect to do ours".

Yes, you should exercise, and eat right, and not drink too much, and sleep right, and not stress too much. You should do as much as you reasonably can in all these areas. But if you're looking to "defy depression, disease and early death", you'd also better learn a bit about the way your health system operates, and the common issues it is inclined (presumably for cost reasons) to ignore.

I've written in past comments about thyroid conditions, which are a big deal, and can easily go undiagnosed for a long time (due, basically, to cost-driven neglect). I'm becoming aware of more issues:

1) Testosterone: I'm a middle-aged man and I've been complaining to my GP for months about feeling weak while exercising, and about sleep disruption. I've been doing my own reading, and have just discovered that hypogonadism (inadequate testosterone levels) is incredibly common amongst middle-aged men:

"Hypogonadism affects approximately 40% of men aged 45 or older, although less than 5% of these men are actually diagnosed and treated for the condition" [1]. Even discounting the fact that the author of this article declares that he consults for pharma companies, that's a staggering statistic. So why given my symptoms do I have to ask my GP to check my testosterone levels, rather than him having ordered the test months ago?

2) Prolactin: it seems a very significant fraction of the population (e.g. 6-25% of the U. S. population) have pituitary tumours, forty percent of which produce prolactin [2], which is a sex hormone. There's evidence that people whose prolactin levels are in the upper normal of the physiological reference range, suffer from insulin resistance [3] and have significantly increased risk of mortality [4]. Oh, and prolactin stimulates the immune system and high levels are associated with autoimmune diseases [5]. I had a test done for prolactin levels recently (I requested some tests to check for hypopituitarism), and levels came back quite high but in the normal physiological range (see above, of course). GP response: "no problem, levels are normal". Which is true, in that I appear to be suffering from quite normal disease that the health system intends to do nothing about.

Bottom line: Do your homework, folks. View your health service as a resource, a gateway to services. When it comes to chronic conditions and diseases of old age, you (or nobody) are the principal investigator.

[1] [http://www.bcmj.org/articles/testosterone-deficiency- practic...](http://www.bcmj.org/articles/testosterone-deficiency-practical- guidelines-diagnosis-and-treatment)

[2] [https://en.wikipedia.org/wiki/Prolactinoma#Epidemiology](https://en.wikipedia.org/wiki/Prolactinoma#Epidemiology)

[3] [https://www.ncbi.nlm.nih.gov/pubmed/28384295](https://www.ncbi.nlm.nih.gov/pubmed/28384295)

[4] [https://www.ncbi.nlm.nih.gov/pubmed/22843444](https://www.ncbi.nlm.nih.gov/pubmed/22843444)

[5] [https://www.ncbi.nlm.nih.gov/pubmed/16411065](https://www.ncbi.nlm.nih.gov/pubmed/16411065)

<comment><USER32> I think if 40% of middle aged men have a testosterone level that's below normal, we've probably misdefined 'normal'.

(And conveniently misdefined it in a way that exploits men's fears about aging and masculinity to sell them things.)

<comment><USER33> I have no idea about the health claims being addressed, but I do feel obligated that normal != healthy and perhaps the original comment is conflating the two.

</comment></comment></comment><comment><USER34> Exposure to weather conditions, hot and cold is also critical.

<comment><USER35> First time I have heard that. Got any references, I would like to read more?

<comment><USER36> tu7001's comment shouldn't be down voted, it is correct. The human body has to expend energy to heat or cool itself when the ambient environment is outside a certain narrow range. Sweating burns some calories, even if you're just sitting around. When we spend our time inside climate controlled rooms then we burn fewer calories. The effect is small, but measurable.

[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3975627/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3975627/)

</comment></comment></comment><comment><USER37> Improv.

</comment><comment><USER38> These articles are such exquisite forms of class warfare that it approaches the sublime.

1\. Upper class people seek to differentiate themselves from lower class people by status markers (like reading the guardian)

2\. In the past, food was scarce, so upper class people were fat when lower class were lean, hence terms like "rubinesque beauty" or the figures of Botticelli's art.

3\. Now food is common though, so the status marker is to be thin, because being thin shows you have leisure time to exercise, (you try being on your feet for 12 hours a day and then going to the gym), have money to eat the right kinds of food, and have the good form to have discipline not to eat when you need comfort (or in general need comfort).

4\. but it's not enough to do it, the upper class needs their virtues reinforced.

5\. Since the upper class is atheistic due to the lower class being religious (unless they are the right kind of religion, something either punishing to follow or neo-atheistic as it is), we must focus on the atheist's heaven, good health.

6\. Your virtue in being thin is rewarded through lack of depression, disease, and long life! Your priestly newspaper has given you a scientific sermon.

Think about this. Why is this article in the Guardian, who I'm willing to bet already exercise, eat well, etc? Wouldn't it be in the Daily Mail instead, if the point was to convince people to exercise? That's because it's there to reinforce the idea of exercise and thinness as a status marker among the Guardian readership. The same thing happens in lower class media too, on different subjects, but the upper class usually never even gets how they are being led around.

<comment><USER14> You realise that by rejecting health and exercise on political grounds, you are condemning the lower classes to shorter, unhappier lives? Good food and exercise really are good for your body and mind, whatever you think of those effete Guardian readers.</comment></comment>


Ways to Get Better at C++ - <AUTHOR> http://www.fluentcpp.com/2017/06/23/7-ways-better-cpp-summer/ <comment><USER2> [http://www.ensisoft.com/temp/cplusplus_stack.png](http://www.ensisoft.com/temp/cplusplus_stack.png)

Just replace that Josuttis's C++ standard library with C++11 version and throw in Mayers new effective book for good measure and you're all set

<comment><USER3> Surely you are joking right? I just can't imagine one has to go through all those books to be proficient in one language.

<comment><USER4> One is never proficient in C++. One merely... learns to adjust ones ignorance to practical constraints. I.e you choose such a subset that you can deal with it. But which subset? Hence, a shitload of books. The point is to read about all sorts of styles and pick whatever feels practical for you.

And I'm completely serious. I've earned my living with C++ for over a decade, am considered a valuable contributor and still... so clueless towards so many details.

<comment><USER5> Then C++ has failed. If it has so many competing standards and subsets, then it is too fragmented. This is why when you see C++ written in one standard it looks like a whole other language when you look at other C++.

If after a decade there are so many details still obscured from you, then that's pretty much again a fault of the overly-engineered and tacked on language called C++.

Other mainstream language work fine and allow you to still write clean and modern code without having literally dozens of subsets and implementations. So why is it that C++ feels the need to have so many?

<comment><USER6> On the contrary, the one nice thing about C++ is that it is 'multi-paradigm'. Don't like boost's philosophy of overengineering every little detail? Then don't use boost. C++ without exceptions, RTTI, or even the whole STL? Perfectly fine. Other languages don't have this freedom. It's a double-edged sword of course. Less freedom also means wasting less time with pointless discussions about which combination of C++ features is 'right'.

<comment><USER7> > Then don't use boost.

After using boost in several projects and regretting it each time (slow compile times, painful to configure), I finally learned my lesson and "don't use boost" is now one of my guiding principles when programming in C++.

<comment><USER8> While I would mostly agree, I would humbly suggest a small nudge in perspective: Boost does not equal Boost. While I passionately hate most of Boost, I find boost::optional to be one of the most useful template classes. So you shouldn't treat Boost as one library, but as a collection of different libraries. Same goes for STL in my opinion.

<comment><USER7> I think boost has definitely had a positive impact on c++ and the fact that large amounts of it are now in (or had a major influence on) the std library certainly makes it much easier to forgoe using it.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> So I hate C++. What I really want is the speed, flexibility, universality and simplicity of C. But I wish structs had destructors, and a way to bind namespaced related functions. Only since I basically now have classes, I should really think about adding inheritance. And I also find prototyping, testing and non-speed critical programming is a pain without a good generic container library. Containers can be implemented fairly elegantly with templates, but then I probably also need some compile-time processing ability. Oh crap, I just ended up at C++ again. And so I persist with C++ as my main language, with a constant voice in the back of my mind saying, "this is just stupid, there must be a better way. Why is there SO g __-d __* much accidental complexity in this language? ". In my view, C++ has probably wasted more programmer hours, and added more sadness and despair to the world (at the very least MY world) than any other technology. But given my language wish list it's very, very hard not to be tempted to fire up c++-mode for just one more hit.

<comment><USER10> Other than C, are there any languages that interest you? Are languages like Rust or D or other systems languages not what you're looking for? How come?

<comment><USER9> I use Python a great deal, especially for tooling and prototyping and appreciate the fact that programming is fast, easy and focused on the inherent complexity of the problem. Similar positive experiences with Smalltalky and Lispy languages as embedded languages. But I write a lot of performance- critical libraries for programs that need good C/C++ interop and where C++ is pretty much the lingua franca. Swift, D and Go, for example, all have their strong points, and I plan to do more experiments when I have some time ... after this one last C++ project, though!

</comment></comment></comment><comment><USER11> > Improve your knowledge on Boost

I might be extremely biased about this, but I can't not recommend to follow Folly as well ([https://github.com/facebook/folly](https://github.com/facebook/folly)). There is some very good C++ there (and some bad C++ too), and it is generally more readable than Boost. Some ex-Boost people work on it too.

The implementation of Function ([https://github.com/facebook/folly/blob/master/folly/Function...](https://github.com/facebook/folly/blob/master/folly/Function.h)), for example, is mindblowing. I learned lots of tricks from it that I ended up using in a bunch of places.

There's also a lot of interesting things happening in concurrent data structures under experimental/ (and the widely used MPMCQueue and LifoSem are also excellent).

</comment><comment><USER12> Can anyone recommend a good compiler/IDE learning setup?

I studied C and C++ back in university (I was in the wave of students that came through after they switched from Pascal as the teaching language, and before they switched to Java). And my first job or two back in the early-2000's was using C++ in HP-UX servers, to talk to other systems via CORBA.

But that was ages ago, and once I stumbled into a Java job I never left. Every so often decide to scrape off the rust and check out C++ again, but the ENVIRONMENT setup scares me off.

Should you use GCC or LLVM? If you're on a Windows box, do you try one of those? (there's at least three different distributions of MINGW available) Or do you use Microsoft's compiler? (VS has potential licensing issues, while the freeware build tools are pretty arcane and it's hard to even find an installer or know which exact thing to install) Or Cygwin? (I'm not 100% sure on the current state of its licensing)

I've tried to build the wxWidgets library at least 100 times. It gives you dozens of options for compiler and environmental setup, and yet I've never found a choice that successfully built all the sample apps.

Once you have a compiler setup, what about an IDE? (Don't even start with the " _just use Vim or Emacs_ " crap, I'm not an animal!) I love IntelliJ in the Java world, but don't want to pay for JetBrains CLion just for hobby/personal use. The last time I tried Eclipse's C++ support it felt pretty half-baked. Qt Creator is actually really nice IDE (a C++ IDE _should_ be written in C++!), but it feels very specific to writing Qt apps.

What about build tools? It's hard enough for Java newbies to wrap their heads around Maven or Gradle. But makefiles and "configure" scripts are far more arcane, and _which flavor_ of make should you be using in 2017? It's hard enough to build a project that uses one of these large frameworks with opinionated build systems. I wouldn't even know where to begin in _combining_ two opinionated build systems as dependencies of a single project, or creating a new build system from scratch for a greenfield large project.

Once you have a compiler, IDE, and build system and are ready to start coding, what about a standard library? Is Boost still the thing everyone uses? STL? Something else?

I really can't imagine that learning C++ syntax is THAT challenging in comparison to Java/C#, Rust, D, or anything else. But I'd love to see an "Effective C++ Environment" book.

<comment><USER13> On Windows, just use Visual Studio Community. You don't need anything else, Microsoft's support for C++11/14/17 features is excellent. I'm not aware of any killer licensing issues with the free version, the only limitation is that if you're an enterprise (> 5 users, > $1M revenue, that sort of thing) you need to buy a license. There isn't a problem with commercial development using the free version. You may also need to provide the VC++ redistributable with your application though.

As for standard libraries, the STL is _the_ standard library and you 'should' use that where possible. Boost is a mix of potential features for C++, a few modules that lots of people rely on (e.g. json, timers, asyncio), and a whole lot of undocumented and unmaintained code. I get by without it most of the time.

Qt Creator is pretty decent even if you don't use Qt. You can use the qmake system which does lock you in somewhat, but it's easier to configure than cmake for a newcomer. You still get debugging/profiling and intelligent autocomplete/linting.

If you don't mind the large library overhead, you can use Qt for a lot of brick-and-mortar code like dealing with filesystems, sockets/serial comms, parsing XML, etc. It's very well documented, updated often and it's GPL which is nice. Or you could statically link it. Personally I've found the Qt GUI framework much more intuitive than the native Windows stdafx stuff.

<comment><USER14> Qt is LGPL nowadays btw.

<comment><USER13> Yep, typo!

</comment></comment></comment></comment><comment><USER15> Ask HN: Do I miss much by skipping "Effective C++" and only reading "Effective Modern C++"?

<comment><USER16> In addition to the other replies, you might end up working on a codebase that precedes C++11.

<comment><USER17> I would say that this is very likely.

</comment></comment></comment><comment><USER18> So what are the best ressources for learning C++11/14/17 for someone who already knows the _old_ C++ ?

<comment><USER19> _Effective Modern C++_ by Scott Meyers

<comment><USER20> One of the best technical books in existence (afaik).

</comment></comment></comment><comment><USER21> it is easier to write fluent modern c++ code than read modern c++ code. The big obstacle of modern c++ is to communicate between c++ programmers.

</comment><comment><USER22> Good advice on learning functional language, learning OCaml helped me a lot to advance my template meta-programming skills.

<comment><USER23> As a non-C++ programmer with a background in functional programming, this similarity was a big surprise to me when I read a recent article on C++ template metaprogramming -- it's all basic functional programming techniques, with recursion, (kinda) pattern matching, etc. Very interesting!

I could see how mixing templates with other C++ paradigms could quickly get messy in the wrong hands, since it's very powerful.

</comment></comment><comment><USER24> Bookmarked, thanks for this! You've covered books, course pages, tutorials and videos as learning resources. Do you have any audio/podcast recommendations with C++ material?

<comment><AUTHOR> Thank you! Well there is one podcast dedicated to C++: cppcast.com. Also I'll publish a post dedicated to books about C++ that shows more recommendations, during this summer.

</comment></comment><comment><USER25> Way to get better C++: write code that everyone could understand.

</comment><comment><USER26> Just a nit pick for the author, take it or leave it: You use "STL" quite a bit through your blog, when you probably mean to say "C++ Standard Library". They are different beasts. You probably know the differences but beginners might not and could become confused.

Otherwise, great article--that's a lot to dive into over a single summer though!

<comment><USER27> "STL" is a perfectly valid way to refer to the C++ Standard Library through metonymy. Source: I am STL, an STL maintainer, with the sovereign right to name what I work on.

<comment><USER2> Cool to have you here!

</comment></comment></comment><comment><USER28> I don't really see any advice here that's not generic.

</comment><comment><USER29> This sounds like a lot of fun, but I'm concerned about the limited ROI. How many practical, modern use cases does Cpp have at this point for those of us who don't work in a few very specific verticals like finance or game dev? Even in those aforementioned industries, C++ is reserved for niche roles / use cases.

Of course not all hobby projects need to be "practical" or directly useful - but time is a valuable resource and I at least would like to have an actual non-toy project goal in mind when choosing to better grok a language. What is C++ uniquely good for in 2017?

edit :

Useful answers to this question -

        * extending libraries for higher level languages
        * embedded dev

<comment><USER12> Are you asking about the ROI in terms of professional career opportunities? Probably three orders of magnitude greater than it is for Rust, D, Haskell, Lisp... or any of the other flavor-of-the-month darlings that HN uses in their hobby projects, but for which few jobs exist in the real world.

Are you asking about the ROI in terms of learning and personal growth? Then it's tremendous. The language that you probably are using (Java or C#) was written because most average programmers couldn't handle the power and flexibility of C++, so industry developed those languages as lookalikes with training wheels. If you've ridden a bike with training wheels your whole life, then aren't you at least curious at some point to see what it's like with them taken off?

If nothing else, then you'll learn why decisions were made the way they were in derivative languages, and become a better programmer overall. It gives you certain amount of confidence to have experience in a language that most people are afraid of.

<comment><USER30> Do you think the suggested books in OPs link are accessible to me if I've only a couple of intro to C++ and Java under my belt?

Universities are a much longer and unfortunately financially inaccessible route for me. Going to try the "build-a-portfolio-and-start-freelancing" approach.

<comment><USER31> It's hard to know without knowing what your intro to C++ looked like, and what your background is. If you can get through Accelerated C++, then some of those books may be useful once you've gotten some practice.

From [http://www.ensisoft.com/temp/cplusplus_stack.png](http://www.ensisoft.com/temp/cplusplus_stack.png) (why is this a PNG and not a JPEG?), Modern C++ Design and C++ Template Metaprogramming are likely to be the most challenging. Don't get me wrong, they are fantastic works, but are probably not what you need right now.

C++ Primer, C++ Common Knowledge, and API Design for C++ are books that I think you should consider that are not listed in the post above. If you have never studied computer architecture it is an important subject to understand. There are good books like Computer Systems: A Programmer's Perspective, but others may have different opinions.

<comment><USER30> Well I think my intro was pretty rudimentary. My knowledge of computer architecture is as well, I can tell you what pieces do what, and what a buffer is, but I couldn't really tell you how memory addressing, logic gates, instruction sets work.

Is the JPEG question a test? If so I don't know!

The last ones you mentioned sound good, and I think I'll add those to my list and start with Accelerated C++ and see how that feels.

I appreciate the info!</comment></comment></comment></comment></comment>


Earth-sized planet around nearby star is astronomy dream come true - <AUTHOR> http://www.nature.com/news/earth-sized-planet-around-nearby-star-is-astronomy-dream-come-true-1.20445 <comment><USER2> Should be merged with [https://news.ycombinator.com/item?id=12353441](https://news.ycombinator.com/item?id=12353441)

<comment><USER3> Yes. Although this post was submitted a minute sooner, the other one got the more active discussion and is closer to being an original source, so we've left that one up.</comment></comment>