CppCon2014 Presentations - <AUTHOR> https://github.com/CppCon/CppCon2014/tree/master/Presentations

<comment><USER2> These are only the slides, and while it's nice to have them all together, slides to talks generally provide very little value unless they're exceptionally† well made. From flipping through a random selection of them, i can tell they're just average examples of the species, good for when someone is talking over them, but fairly useless taken on their own.

Will this repository be amended with links to the videos to make it useful?

† exceptional would be slides in this format: [http://rjbs.manxome.org/talks/moose/](http://rjbs.manxome.org/talks/moose/)

<comment><USER3> What do you think of my slides? ("STL Features And Implementation Techniques") I try to make them readable by themselves.

<comment> alok-g I once made a presentation that was much more self-contained than this. Everything wanted to say was spelt out in plain English. The take-home message in each slide was color coded and simplified to levels you could deliver to VCs. The title matched the contents (unlike yours it seems to me) by claiming no more and no less than the contents. The outline and flow of the presentation was made explicit after section.

Then I received the feedback. While I was seeing nods from several people during the presentation, many people later reported that they could not understand one single thing.

I learned presentation skills that day (i.e., this experience taught me the same).

Imagine surveying all the audience members about percentage of the material they absorbed from your presentation and averaging across people. If you think the number would be more than say 30%, I do not think you know typical audience well.

PS: Please see these as constructive comments.

<comment><USER3> > The title matched the contents (unlike yours it seems to me)

How do you think my title differs from the contents? As far as I can tell, it is completely accurate.

</comment></comment></comment></comment><comment><USER4> Looking forward to see these in parallel with the actual video s of the presentations.</comment>


Tesla's 'Autopilot' Will Make Mistakes. Humans Will Overreact - <AUTHOR> http://www.bloomberg.com/view/articles/2016-07-01/tesla-s-autopilot-will-make-mistakes-humans-will-overreact <comment><USER2> That's part of the problem.

Tesla has two basic problems with their "autopilot". One is hype, and one is bad technical design.

Tesla's "autopilot" is just automatic lane keeping and automatic cruise control. Mercedes and BMW have been shipping that for years. The other car companies have hardware to check if the driver's hands are on the wheel. Tesla not only didn't do that, they hyped those basic functions as being an "autopilot", as if they had something comparable to what Google has. Tesla has significantly overhyped their technology, to a dangerous level.

Here's another Tesla crash, from yesterday.[1] This is a Tesla rear-ending the corner of a stalled van on a highway. This may be an "automation in wrong mode" error, and complicated excuses from Tesla will probably be forthcoming. That's not good enough. Airline pilots get into "wrong mode" errors, and they get extensive training, including simulator time. Drivers don't get that. This driver presumably thought the automation would do something sensible in that situation. It didn't.

The other big Tesla problem is a pure technical design error - the bumper- mounted radar is blind at windshield height. This is the immediate cause of the fatal accident involving under-running the semitrailer. It's also the cause of the parking accident with the truck with overhanging load sticking out the back.[2] Tesla's implementation has deadly blind spots. They need a radar at windshield-top height. It's quite likely, now that the NHTSA is looking into this crash, that Tesla will have to do a recall and retrofit one. (Along with hands-on-wheel sensors.)

Tesla was relying on stereo cameras too much. Depth from stereo is always iffy; there needs to be some detail on the target to get range. You can't range a uniform surface with stereo vision. (Human flyers have the same problem flying over ice.) A big white truck under some lighting conditions has that property. Cameras are useful, but backed only by today's algorithms, not enough. Teslas come with a sensor suite inadequate for the job.

From a software perspective, Tesla's software seems to be overoptimistic that there's no problem ahead. 3D vision systems know when they don't have range info for a large area. Tesla's software was willing to drive into that. It didn't note the absence of range info for the road surface; it may not even be profiling the road ahead. (We all had to do that in the DARPA Grand Challenge because we were operating off-road. That's a solved problem. Google does it with their roof-mounted LIDAR.) Tesla owners have asked for "pothole detection" as a feature. Lack of this may eventually take someone off a cliff.

The press is gradually picking up on this. Until recently, Tesla's autopilot was treated as a magic black box in the press. Now some automotive writers are starting to ask the right questions.

This is not a setback for automatic driving. This is just a Tesla problem.

[1] [https://www.youtube.com/watch?v=qQkx-4pFjus](https://www.youtube.com/watch?v=qQkx-4pFjus) [2] [http://www.roadandtrack.com/new-cars/car- technology/news/a29...](http://www.roadandtrack.com/new-cars/car- technology/news/a29133/tesla-self-driving-crash-summon-autonomous/)

<comment><USER3> > The other big Tesla problem is a pure technical design error - the bumper- > mounted radar is blind at windshield height. This is the immediate cause of > the fatal accident involving under-running the semitrailer. It's also the > cause of the parking accident with the truck with overhanging load sticking > out the back.[2] Tesla's implementation has deadly blind spots. They need a > radar at windshield-top height. It's quite likely, now that the NHTSA is > looking into this crash, that Tesla will have to do a recall and retrofit > one. (Along with hands-on-wheel sensors.)

This is just completely false. We've been putting them at bumper height for 10 years because that's where the system ends up working best with the greatest number of targets to see. NHTSA has been regulating these sensors for years and designing the tests for them; they are not new to this. Tesla is not going to have to do a recall on these, and spare me the idea that they are going to "retrofit a sensor at windshield height." This is just mindless speculation you made up.

<comment><USER2> Volvo has placed radars at the top center of the windshield. They had a blind spot at bumper level, so later Volvo systems added a second radar down there. And a LIDAR. [1] One radar at bumper level is just enough to keep from rear- ending the car in front. That's not enough for a hands-off "autopilot". You need at least two, or beam steering in elevation, like the Fujitsu unit.[2] (Although that has only ±9° of elevation, not enough at bumper level to eliminate the need for a higher sensor.)

[1] [http://www.pcworld.com/article/2025386/volvo-v40-watches- the...](http://www.pcworld.com/article/2025386/volvo-v40-watches-the-road- with-camera-radar-laser.html) [2] [http://www.fujitsu- ten.com/business/technicaljournal/pdf/38-...](http://www.fujitsu- ten.com/business/technicaljournal/pdf/38-1.pdf)

<comment><USER3> I had not seen some of that Volvo tech, thanks for the link. Ultimately the radar portion is going to be more three dimensional tech still in the bumper. It's a struggle financially to get a single one of these sensors on a commodity vehicle, so the tech is focused on how to do more with that. Still, what Volvo is doing with the technology available is very interesting transitional technology.

</comment></comment></comment></comment><comment><USER3> I used to work on the radar systems in these vehicles. It's a problem that the feature is called "autopilot," yes, but the larger issue is that the car does not _enforce_ what the company often says in press releases, which is that you need to remain alert and with your hands on the wheel. There have been similar systems in other vehicles for years - I'm most familiar with those in Mercedes vehicles that are not as advanced but perform similar actions. Those systems will _automatically turn off_ after just a few seconds of removing your hands from the wheel. That is what the Tesla should do. The autopilot feature is a toy; it's not to be used in place of a human driver, nor would a human driver be able to step in quickly and take over for the system were it to suddenly fail had the person not been paying attention. It should not be possible to watch a Youtube video of someone driving completely hands free for minutes at a time.

Yes, Tesla should not call it "autopilot," but if you want to really prevent people from using the system in a way in which it's not intended, you need to enforce it in hardware.

<comment><USER4> Is Tesla's autopilot using radar? How is radar fooled by a white truck against a bright background? I was surprised by this point of failure, which suggests to me that they aren't even using radar.

<comment><USER3> As of July 2016, the autonomous driving system in the Model S (& X, presumably) is _not_ using radar, only cameras. The vehicles do have radar which is used for other things (various crash avoidance programs), but not this, and these things presumably did not fire correctly. I do not know if Tesla's software update capabilities would allow it to push a future autonomous driving update in the future that could interface with the radar sensors; I'd assume so. When I was working on these things the software build that went to the factory was what was on the car for 15 years, obviously this way is superior.

As a side note, radar in cars and semi trucks are a difficult match. The height of tractor trailer trucks makes it difficult to figure out if they are actually obstacles or if you're really seeing a bridge or something. Normally there's enough hanging down in back that it's not an issue, but in this case the truck was perpendicular to the car. It would not surprise me if the radar went right underneath the truck and either didn't see it, or saw it but thought it was something like a bridge that the car could go under. These are difficult and complex systems to engineer.

<comment><USER5> :o) Tesla is not using radars?!

I am driving an '14 Audi A6 and I can tell you, the radars + software on it are the most advanced I ever saw. They even catch a small cat when was trying to cross the street. The car actioned a emergency break before I even realized. The only time when this wasn't working, was during a snow blizzard, when the radar sensors got iced and they couldn't see anything. Otherwise, they are pretty wide range, at least 100m.

It also keeps the car in between the lanes - this is done with a camera, but if you take your hands of the wheel, you get a max of about 30 seconds of free driving to maybe open a bottle of water, before it get's really loud.

<comment><USER3> > :o) Tesla is not using radars?! I am driving an '14 Audi A6 and I can tell > you, the radars + software on it are the most advanced I ever saw.

TIL everyone on Hacker News drives a much nicer car than I do. The crash avoidance measures run in tandem with the autonomous driving features, but the system (which is mainly licensed technology from Mobileye) just doesn't get any depth intelligence from it. Presumably if autonomous driving pointed you into a brick wall, the radar stuff would still trigger the AEB features (autonomous braking).

</comment></comment></comment></comment></comment><comment> tuna-piano I'm saddened, though not surprised, with how many people are so quick to come to Tesla's defense here.

Autonomous cars will have issues. But we should hold the software makers accountable for those issues the same way we hold airplane makers / maintainers responsible in the rare event their planes crash.

As other commenters pointed out, Tesla likely released a half-baked technical solution too early, and created misleading (and in this case dangerous) marketing to go along with it: calling it "autopilot" and not requiring hands on the steering wheel. Tesla screwed up here, and shame on them.

This is not a binary world: I'm not saying that they should be bankrupted or the autonomous car industry shut down. When a plane crashes due to poor maintenance - I want that company to suffer market+legal consequences, I don't want the entire aviation industry to be shut down.

White trucks on a sunny sky are not rare enough to call this an anomaly. If Tesla had installed sensors on their cars to collect data but not enabled autopilot yet - would this man still be alive today?

The MVP model is great, but not for things where life is on the line. Google seems to be doing this right.

<comment><USER6> The product Tesla has built is a perfectly fine assistance mechanism. The problem is that it's not an autopilot and nowhere near autonomous driving, nevertheless Tesla is naming it and marketing it as such.

The problem is that Tesla's marketing here is (borderline) fraudulent. There should be and need to be legal consequences for that.

<comment><USER7> > The problem is that it's not an autopilot…

An autopilot is a system that assists, but does not replace, the human operator in operating a vehicle. Tesla's system does exactly this.

> … and nowhere near autonomous driving…

Nobody ever claimed it was autonomous.

<comment><USER6> A plane can depending on the airport start, fly and land on autopilot, you just need to tell it where to go. There is pretty much no unplanned intervention necessary. This is what the term means to people, autonomous driving. I don't think most people would draw any distinction between the two.

In a Tesla it would be irresponsible to take your hands of the wheel or not to pay attention to the road for even a minute. Tesla's "Autopilot" is nothing like what people imagine an autopilot to be like.

</comment></comment></comment></comment><comment><USER8> This entire article is based on the premise that two very different situations are directly comparable.

In the former, people sometimes accidentally stomp the gas instead of the brakes, then report the opposite.

In the latter, Tesla markets a feature as "Autopilot". It pilots a car into another vehicle. The feature had apparently been demonstrated at events with key safety features disabled: attendees are able to use it without their hands on the wheel. "Autopilot" fails to account for a common real-world traffic occurrence and its driver has an inadequate mental model of the feature, which is made up of a complicated mesh of sensors and software.

There may be something to the argument that self-driving car features will reduce traffic fatalities even as they become responsible for a distinctive (but hopefully small) subset of those fatalities. But this article doesn't make that point.

The problem people have with features like Autopilot is that it may not be reasonable --- at least, as those features are currently designed --- to expect human drivers to model their behavior accurately enough to use them safely. This is an HCI problem that people in the computer industry should be very familiar with. I'm surprised to see it dismissed so easily.

<comment><USER9> > without key safety features disabled

Did you intend this to be a double-negative? It's pretty confusing as written, but would make sense as a single-negaitve.

<comment><USER8> Nope, thanks. I shuffled the sentences around in that paragraph but should have just rewritten it.

</comment></comment></comment><comment><USER10> >It appears to have been something of a freak accident -- white trailer riding high against the bright sky, so that the autopilot didn't detect the truck in its path.

It wasn't a freak accident. The AI failed. The human driver was distracted and failed to respond in time. Two failures Tesla has to answer for. IMO the second one is the gravest. They shouldn't have deployed the technology, primitive as it is.

<comment><USER11> How come Tesla has to answer for human failure?

<comment><USER12> It depends on what kind of failure. If there is a red button next to an slightly darker red button and you get the instruction from your boss to press the red one, you have a 50% of failing. But maybe the designer of said buttons should have thought about giving the buttons different colors instead of similar ones.

"The Design of Everyday Things" talks about this, that sometimes it's the designer of something's fault, rather than the user of the something. Also called "Human Error" by statements.

Search for "[https://archive.org/stream/DesignOfEverydayThings/DesignOfEv...](https://archive.org/stream/DesignOfEverydayThings/DesignOfEverydayThings_djvu.txt") here [https://archive.org/stream/DesignOfEverydayThings/DesignOfEv...](https://archive.org/stream/DesignOfEverydayThings/DesignOfEverydayThings_djvu.txt)

Point being, unless Tesla makes it safe to be used as autopilot, you probably shouldn't say it's autopilot. Otherwise it's not good enough and you would fault Tesla for the design rather than the users of the autopilot.

<comment><USER11> I agree that the naming is poor - that doesn't make them liable for human error though, the driver has to opt in and is made well aware of the operation modes.

Incidentally I'm carrying that exact book in my backpack right now. I think the problem here is much deeper, and even opposite to what Norman discusses in the book. An autopilot is the epitome of "don't make me think" \- I want to go from A to B, no need to worry about the driving. The problem at hand is that the technology is not mature enough to fully take over from humans, and we are not good at staying alert in this half-focused state - I don't own a Tesla but it must be incredibly boring. It's an interesting conflict of tech vs human nature.

</comment></comment></comment></comment><comment><USER13> Humans "overreacting" is the reason why airplanes are so safe. This reaction (IMO "fair" not "over") is important for the development of the driverless cars tech. It will make them much, much safer and won't kill any innovation.

"Underreaction"[1] is much more dangerous for society than overreaction. I would cite digital privacy as an example of this.

[1] i'm not sure _underreaction_ is a word or if i invemted it. Is underreaction so underreacted to that it is not even a word?

<comment><USER14> From a utilitarian perspective one can invest too many resources in making specific things more secure. That happens when other causes of death dwarf the probability of incidence to cost of fixing ratio of a particular failure mode.

If you're accounting for micromorts then we might be at the point where TSA checks destroy more human life-hours than terrorism and normal plane crashes combined. Simply reverting airport security to pre-9/11 levels could save more lives than _infinite_ spending increases for airplane crash safety.

So yes, at some point it becomes an overreaction when resources would be better spent on other ways of saving lives.

I'm not saying that we're there yet with self-driving cars. We probably are with planes.

<comment><USER15> This is how I feel about car insurance in Ontario (think $5000 a year for a teenage male).

On one hand, it's great that if I get in a crash I get crap loads of money for physical therapy and what not. On the other hand I wasted 2h every day of high school waiting and sitting on the bus and so did everyone else. Hours I'll never get back. Would rather have taken my chances :/

</comment></comment></comment><comment><USER16> The biggest elephant in my view is that Tesla has known about this accident and has been hiding it from investors since May. The only reason why this came out is the NHTSA investigation, and if it were up to Musk, this crucial information would have stayed secret until he had completed the dumping of SolarCity onto Tesla shareholders.

</comment><comment><USER17> This is the most horrifying part to me:

 _Eventually, the National Highway Traffic Safety Administration got involved, and wrote up a report which found that … yup, these drivers were stepping on the gas instead of the brakes, often with horrific results. That didn't save Audi, for which sales collapsed and which almost pulled out of the U.S. market._

Despite thousands of hard working engineers, QA, Marketing, Sales, Legal and design people earnestly working to put out a safe, quality vehicle, Audi was almost undone by histrionic consumers. Really depressing when you think about it as a founder.

<comment><USER18> Far as I remember, that Audi had pedals that were narrower than usual and closer than usual. So it's not about the customers being histrionic, more like somebody in engineering not really understanding what he was doing.

</comment></comment><comment><USER19> The tesla incident highlights the exact technical/ethical problems of using such systems. I was just at a conference speaking with the Google automated car team and the handoff problem is yet to be solved conceptually. The tesla accident is just a first example of it. Obviously the utility of an autopilot system becomes questionable if you can't guarantee safety. It's very similar to medical advances in this way. 1% errors is not a good error rate if it leads to death or serious injury even though such error rates for autonomous systems are remarkable.

That said , if we look at the statistical likelihood of death in a car autonomous systems may very well be much better than humans, but whether we as a society are ethically okay with coming to terms with a piece of software that kills 1 / 100M is another question. A drunk driver that kills someone or themselves is obviously to blame. In this case who's fault is it? The drivers , tesla or the safety board for allowing autopilots to be sold?

<comment><USER20> "In this case who's fault is it? The drivers , tesla or the safety board for allowing autopilots to be sold?"

Obviously the insurance. In Europe you need to have insurance to pay whenever is your fault.

With autopilot the only difference will be that the event of having an accident will be less probable. This means that paying the normal driver fee the insurance company will be making money if you autodrive.

But most of the time the fault will be on the other side, because the autopilot is not going to drive like a drunk or slept person. It will be deer or dogs or children on the road when they should not be there.

<comment><USER19> I wouldn't be so quick to say insurance companies would pick up the tab. I don't know which country you're from but in the us getting insurance companies to pay for a claim is very hard and until a set of laws are established I would expect dramatic pushback from them. Furthermore in most accidents insurance companies try to seek out the party at fault if it's not crystal clear to avoid paying.

But an interesting situation would occur if the incentives from the insurance companies would be directed towards automated driving because of reduced risks and costs to the driver. That would be an interesting scenario

</comment></comment></comment><comment><USER21> I think Tesla screwed up by hyping it like they have an autonomous driving system. Mercedes, BMW, etc. did not make that mistake with similar technology. Just google "tesla autopilot" and you'll see the phenomena: people sleeping, playing games, recording videos of themselves.

From that search, here's an interesting article from WIRED warning about the dangers (and lawsuit proneness) of the technology:

[https://www.wired.com/2015/10/obviously-drivers-are- already-...](https://www.wired.com/2015/10/obviously-drivers-are-already- abusing-teslas-autopilot/)

</comment><comment><USER22> Best outcome of NHTSA investigation would be financial penalty and order prohibiting use of AutoPilot brand on NON autonomous system. It was clearly picked to mislead customers into thinking they are buying more than there really is.

</comment><comment><USER23> A bad joke, but this somehow reminded me about: Darwin Awards movie, Autopilot Cruise Control scene. Afaik, Tesla Autopilot is an assistant, not an fully autonomous system. Which inherently means that it requires constant supervision. Using boats or planes autopilot won't either relieve you from monitoring where you're going to end up. Afaik, some of the news articles were titled with pure lies. 'Autonomous car', hmm, nope. Tesla isn't such. It's always important to acknowledge the true capability s of a system and not blindly trust it. This applies to any more or less automated system. People start blindly trusting it even if it really shouldn't be trusted. Let's say robot or AI stock investment bots. Luckily those are't going to kill anyone. I know, quite rude. It's sad that what happened happened. But in technological sense, we all knew it was going to happen sooner or later. We're going to see more and more things like this in future. - I've blogged several times about this same phenomenon, even before this case.

</comment><comment><USER24> I don't get the argument that using the name Autopilot implies that the driver need not supervise the car. In aircraft autopilots require pilot supervision who can take over in the event of autopilot failure. In marine applications autopilots are even simpler. No-one would think that an autopilot in either of these fields where the word has been traditionally applied can function completely independently.

<comment><USER25> I disagree. Besides specialists, I think most people think that Autopilot means it doesn't need supervision. Wiktionary's definition:

A mechanical, electrical, or hydraulic system used to guide a vehicle without assistance from a human being.

"I set the autopilot to due south, so I could get some rest."

</comment></comment><comment><USER26> As Tesla's sales mix shifts from mostly early adopters to more 'normal' consumers we are going to see a lot more complaints.

Normal consumers aren't interested in technical details or complicated explanations. They just want what the marketing promised them.

</comment><comment><USER27> How about $500 million fine for Mr. Elon Musk with five years supervision of public statements by or for Tesla by government appointed lawyer.

</comment><comment><USER28> \- Tesla has collected 130 million miles of autopilot data in 8 months

\- Google has collected 2 million miles of data in 5 years

\- others have collect 0 million miles of data

One might assume that each 10x of data doubles safety. So, a billion miles of data is perhaps 8x better than a million. It was bold of Tesla to ship autopilot, but it gives them a significant data lead while Google's caution means they may be squandering their once-thought-unassailable technology lead.

My thinking is that first you eliminate the 1% problems, then the 0.1% problems, then 0.01% and so on. So each 10x of scenarios tested is roughly one step of improvement.

<comment><USER29> You have to define what data they're collecting that's so valuable. Their system is so terrible at sensing the world around it that it will run right into construction barricades if you don't take over:

[https://www.youtube.com/watch?v=iSasPVYzSQ0&t=150](https://www.youtube.com/watch?v=iSasPVYzSQ0&t=150)

It's a fancy cruise control, not self-driving. It doesn't understand anything but how to follow lanes (sorta, it doesn't work everywhere) and it has some ability to recognize obstructions in front of it and stop (again, sorta).

Google is teaching their car how to navigate city streets, deal with human drivers and pedestrians, navigate intersections, handle construction zones, deal with emergency vehicles and cops directing traffic, etc. All with a _much_ better sensor package than in a Tesla Model S. Their system doesn't even need lane markings on the road, as they compare LIDAR data with a pre- marked reference map. Tesla's system, and every other system that doesn't do this, breaks down when lane markings are faded or non-existent.

Google's system would not have been prone to this type of accident five years ago, much less today.

<comment><USER30> > Google is teaching their car how to navigate city streets, deal with human > drivers and pedestrians, navigate intersections, handle construction zones, > deal with emergency vehicles and cops directing traffic, etc. All with a > much better sensor package than in a Tesla Model S.

The interesting thing is that no matter how much one anticipates situations a priori and builds them into the AI/model, there will likely exist situations not anticipated by the engineers.

For example, it is not clear to me how exactly a situation like [https://en.wikipedia.org/wiki/Driving_direction#Sweden](https://en.wikipedia.org/wiki/Driving_direction#Sweden) will be handled. Will this be "patchable" (never mind the logistics and security issues of providing updates to a nation's fleet), or will it require a full retraining of the AI? Or more generally, how robust/modular will the designed AI be to these kinds of situations?

The reason I like this example is because it is a classic instance where humans have inertia and thus have difficulty in consistently applying the "switch driving side" rule - I have seen international travellers requiring at least a few minutes at the wheel to reorient themselves.

In principle one would hope that a modular software solution can handle things more consistently.

</comment></comment></comment><comment><USER31> There were 1.07 fatalities every 100M automobile miles last year. Compare that to 1 fatality in 130M autopilot miles. This is not going to be the last fatality we see but it's not clear that autopilot is any worse than a human and statistically this still looks to be the safer option.

<comment><USER32> For comparison, that is 1.391 fatalities per 130M automobile miles. So 0.391 people were saved by Tesla, but that doesn't account for the fact that this is a new, eerie technology and that people are possibly using it in the safest of conditions until they get comfortable with it. Also is the average Tesla driver really representative of the overall population, i.e. would Tesla drivers have had that many crashes without Autopilot? How much of that 0.391 is due to the safety features of the car itself, and not the success of the autopilot system?

At any rate, there isn't much room for error in those numbers, and considering the above this almost looks like evidence that Autopilot is less safe than normal driving.

</comment></comment><comment><USER33> This is how things evolve.

<comment><USER34> It shall get a chapter in the orange catholic bible, the prelude to the butlerian jihad. ;)</comment></comment>


Phones that secretly listen to us are a myth - <AUTHOR> https://www.bbc.co.uk/news/technology-49585682 <comment><USER2> Fixed headline: synthetic study that did not even try to reproduce normal phone usage patterns fails to find evidence of audio tracking.

Wasn't it just last year that a wide network of android apps using inaudible cues for ad tracking was uncovered? What are they trying to prove here?

<comment><USER3> It's the BBC. Who knows what their agenda is today.

</comment></comment><comment><USER4> In other words, not a myth at all. "It's not the phones listening, it's apps on the phones. And maybe also the phones, when somebody cares to snoop."

Likely snooping happens more when phones are moving, and not lying about in a room with recognizable recordings playing, and that belong to people somebody cares to snoop on.

But it is true that readily available metadata being deliberately sent by users, via clickstream, is much easier to extract usable detail from.

</comment><comment><USER5> The notation that phones that secretly listen to us, because the big tech companies are evil and they do it because they can - may be a complete myth.

But the notation that phones, due to their system insecurity, widespread of malicious applications, the reliance of voice recognition over 3rd-party servers (and in turns, their insecurity), and the ethical problems of handing training data and samples, are all avenues that can be easily exploited to secretly listen to us, is very true.

Overemphasizing the first notion while downplaying the second one is a way to manipulatively shutdown the public concerns over privacy issues.

</comment><comment><USER6> > Interestingly, the study found that most of the android phone apps seem to > consume significantly more data in the silent rooms with many iOS apps using > more in the audio-filled rooms.

Disturbing. The fact that there is a difference suggests that they are doing _something_ with microphone input...

<comment><USER7> Or that they are different operating systems. Perhaps iPhones ship with shazam enabled, for example (given Apple bought them).

<comment><USER6> Less concerned that there is a difference between ios and android, more that there is a difference in a quiet room vs a noisy room in both cases.

To your specific point, Shazam has to be activated (hey siri, what song is this?) on iOS.

</comment></comment></comment><comment><USER8> Even if there were not already ample evidence that phones do listen to us, the very publishing of this article would basically be a blaring confirmation.

This feels like it was dictated to the writer directly by a not-super- competent intern at an intelligence agency.

</comment><comment><USER9> > For 30 minutes they played the sound of cat and dog food adverts on loop.

I feel like this could've been more similar to what was being suggested, adverts or background TV/radio isn't what any ad-driven telemetry is going to be interested in. Definitely not discounting the research, just seems like this element could've been improved.

</comment><comment> m-i-l Anecdote time.

Short summary: I received in my Google news feed a story about something I would never expect to see in my news feed. The only rational explanations I could think of were that (i) Google had eavesdropped on an earlier conversation, or (ii) Google had received the full data for an offline credit card purchase in a bricks and mortar store made with a credit card which had been saved in my Google profile.

Full story: My wife went to a bricks and mortar bookshop to buy a boy's birthday present. In the bookshop, our daughter said he liked football, so she bought a children's book by an English footballer. She did not know what to buy beforehand, and did not look the author up online afterwards, so there was zero online footprint for the transaction. When she got home, I asked what present she'd got, and she told me it was a football book. When she told me the author's name, and I said I'd never heard of him, joking that the only footballer's names I knew were Scottish footballers from the 1980s that I'd memorised to try to fit in at school. Again this conversation had zero online footprint. The next day there was an article in my Google news feed about that specific English footballer. I don't recall ever seeing any football related story in my newsfeed, so would never expect to have seen that specific footballer appear by chance.

I'm inclined in this case to believe it was Google buying data on credit card transactions for credit cards linked to the Google account. I keep on meaning to ask around if anyone knows if Google do this. Perhaps someone here knows?

<comment><USER10> A quick Google search suggests that this might possible.

[https://www.theverge.com/2018/8/30/17801880/google- mastercar...](https://www.theverge.com/2018/8/30/17801880/google-mastercard- data-online-ads-offline-purchase-history-privacy)

It's interesting to see how slowly we are all catching up to what is going on behind the scenes, and how we all struggle to come to terms with the implications.

<comment> m-i-l Thanks for the link. Certainly sounds feasible. FWIW the credit card used for the purchase was a Mastercard. This would definitely go beyond what is described in the article though because (i) I'm pretty sure neither of us were shown any online adverts for football related children's books prior to going into the store (if we had they'd have stuck out like a sore thumb among all the unicorn, pony, fairy etc. children's books we normally see based on our purchase history), (ii) to target an individual's newsfeed the Mastercard data can't have been anonymised.

</comment></comment></comment><comment><USER11> My wife and I were watching John Wick and got to talking about owning a handgun. The next morning my facebook feed was showing me marketplace listings for holsters.

<comment><USER12> Frequency illusion or Baader–Meinhof effect: The illusion in which a word, a name, or other thing that has recently come to one's attention suddenly seems to appear with improbable frequency shortly afterwards (not to be confused with the recency illusion or selection bias).[51] This illusion is sometimes referred to as the Baader–Meinhof phenomenon.[52]

[https://en.wikipedia.org/wiki/List_of_cognitive_biases](https://en.wikipedia.org/wiki/List_of_cognitive_biases)

Or maybe you watched john wick on netflix or amazon and boom.

<comment><USER13> Or read a John Wick review, bought the movie tickets online or paid for the tickets with a credit card.

</comment></comment></comment><comment><USER14> I'm not really sure who this article is trying to convince. Besides, who thinks it's the "phones" listening to us? It's the apps that do the listening. Only recently are iOS and android finally clawing back errant permissions on these apps to prevent such behavior.

<comment><USER15> The baseband firmware almost certainly has backdoors exploitable by certain national intelligence agencies to listen in on you, selectively.

<comment><USER4> I know someone who worked at Qualcomm with access to that baseband code, who had a personal project to scan it for backdoors. He came up empty. So if they are there, they are subtle. At least in Qualcomm basebands. Exploits probably provide all the backdoors the spooks need.

<comment><USER16> Two alternatives:

* Firmware is patched after the fact either by inserting a backdoor into the build process, or binary patching the resulting firmware image. Such an operation is likely to be compartmentalised to avoid employees knowing about it.

* Intelligence agencies seed online discussions with false assurances (note that I'm not suggesting any wrongdoing on your part - merely pointing out that an anonymous forum post is hardly verification that can be relied upon).

That said, I do agree that exploits are likely sufficient, and that planting a mass backdoor is probably not worth the risk of it being detected.

<comment><USER15> I was in fact referring to exploits in my post. That's typically how NSA and GCHQ operate these days: either find exploits, or if a sufficient number can't be found, have some hard-to-detect exploits inserted into the code by operatives or as part of unrelated bug fixes.

There's also hardware exploits. The firmware doesn't need to have a back door if a certain sequence of packets sent to the network controller will cause key memory to be overwritten and live patch in a backdoor.

<comment><USER4> Agreed, hardware backdoors are very possible, but software is much cheaper to mess with. The spooks have in the past shown a preference for hardware jimmying, but those have been one-off cases.

That said: monkey-patching is much more easily concealed than hardware or source-code changes, and is more in line with how US spooks have been seen to operate.

</comment></comment></comment></comment></comment></comment><comment><USER17> So, there are hardware in phone whos job is to recognize trigger words and phrases like "Hello Google" , "Alexa" and "Siri" and wake up the phone so it can listen to the users request. They probably have limited memory, but I would not be surprised if they can store a few hundred low quality rules for detecting certain words. The ad-network already have a profile on you, and could compile a list of trigger words you are likely to use based on your profile. They upload this list to the phone, and lets the phone build up a frequency list that they can use to better target you with ads you are likely to click on.

All this would just require a few thousand bytes transferred between the phone and the ad-network. Since they are using information they already have about you, and there are little or no additional CPU or network usage it would be really hard to detect.

</comment><comment><USER18> Why phones that secretly listen to us are a myth?

Because we _know_ they're doing it. Therefore, not a secret.

</comment><comment><USER19> This is a misleading title. Intelligence agencies turning phones into listening devices is not a myth

<comment><USER20> Not just Intelligence agencies, if I remember correctly FBI used switched off phones to record mobsters. But I can't find the link now, hope it was not a hoax.

<comment><USER21> The FBI is an intelligence agency, FYI.

</comment></comment></comment><comment><USER22> The insistence, in the absence of concrete evidence, that tech companies are using illicit speech recognition is a symptom of tech illiteracy in the broader population. People are basically anthropomorphizing their phones - since humans acquire most of their knowledge by listening to others talk, people irrationally assume that the machines must be operating using the same methods. They don't quite grok that there are other less anthropomorphic, less intuitive, but more sophisticated methods for apps to make inferences about you without having to listen to you talk.

I saw some guy on Reddit once triumphantly declare he had proof that Google (or Facebook?) was listening to his microphone because he started receiving Spanish language ads after he began working at an office with a bunch of Spanish speakers. Come on dude, you share a wifi network with them, your GPS location overlaps with theirs 8 hours a day, and you presumably frequently send/receive emails from them. Facebook/Google doesn't need to listen to your mic to know that you have a bunch of Spanish speakers in your social network.

<comment><USER23> We're long past the days when you had to be paranoid and 'computer illiterate' to believe computing devices couldn't pick up speech.

For a decade or more vendors have been shipping voice recognition technology onto consumer devices, with less and less friction to operate, actively attempting to make it part of very common device interactions.

Now the only questions are:

A. Do these omnipresent hot mics have adequate access control?

B. How many parties have access to them, and

C. Are these parties worthy of being trusted with such responsibility?

<comment><USER22> It's not about whether phones can listen in to you l, it's about whether that's the only possible explanation for accurate as targeting (it is not)

</comment></comment></comment><comment><USER24> Oh yes! "We monitored a few phones for a few minutes and are now so convinced that we write an article telling everyone to relax"

While I am not a fan of conspiracy theories, such half-assed pseudo-proofs actually make me believe there might be something behind the underlying conspiracy theory.

</comment><comment><USER25> It's not a myth though. After GDPR and Google added the new privacy tools to their accounts I was able to hear the audio files it had "accidentally" recorded of me.

Usually I just heard ambiant noise like computer, media, some distant talking. My naive assumption was that it had misfired on "OK google" and recorded for a few seconds to see if I'd give it a command.

I've never opted-in for any OK google type service or device but I did own two vanilla Pixel devices.

I've since opted-out of such recordings and it appears that Google has made no more recordings of me since then.

</comment><comment><USER26> My personal favorite anecdote: I was out driving in the car with my partner. I said "hey, that guy is riding an electric unicycle", and then starting 4 hours later and lasting for 5 months, facebook showed me ads for electric unicycles.

<comment><USER27> Sure, we all have funny little anecdotes about times we talked about something and then saw ads for it later that day.

But as a technologist and software developer myself, I know just how difficult it would be to make a device that would be able to constantly transcribe the audio input stream and surreptitiously upload that data all day - and do so without destroying the battery life or having a noticeable network bandwidth footprint.

Not too mention the fact that if any memos/emails documenting any part of these "black-ops" advertising programs were leaked, it would be the tech story of the decade and probably result in billions of dollars of fines and legal fees for Facebook/Google/Apple/Amazon.

No, in the end, it seems much more likely to me that these occurrences are just the Baader-Meinhof phenomenon in effect.

After all, how many things have you talked about that DIDN'T end up in your advertising stream? I mean, this is pretty easy to test. Why don't you just start talking about adult diapers in front of your phone right now and see what happens?

</comment></comment><comment><USER28> Then how do they explain the third party company with audio recordings from Facebook?

Plus sending full audio recordings to a server would be somewhat odd, isn't collecting key words converted in text and sending that a lot more plausible?

</comment><comment><USER29> I mean, the BBC would say that.

</comment><comment><USER30> I am unable to find anything published by Wandera related to this. What source is BBC quoting and where is the paper?

I am hoping that the datasets used by Wandera will be available for us to look at.

</comment><comment><USER31> In other news, Google isn't biased, which a Google search revealed just yesterday. More news at eleven.

</comment><comment><USER32> This is funny. You can review the audio files Android records online. This feature is completely unknown to my parents, secretly maybe a strong word though.

[https://support.google.com/websearch/answer/6030020?co=GENIE...](https://support.google.com/websearch/answer/6030020?co=GENIE.Platform%3DDesktop&hl=en)

<comment><USER33> Just to be prdantic: that's not Android, but Google Assistant. This would also show requests made through Google Home etc.</comment></comment>


One of my all time favorite videos - Where the Hell is Matt? - <AUTHOR> http://blog.newscred.com/?p=128

<comment><AUTHOR> This is developer Matt Harding's tour around the world. I just love the visuals, the music, his dance. Brings a smile to my face every time. Why can't we all just get along?</comment>


LinkedIn Skills & Expertise Pages - No Longer Supported - <AUTHOR> http://help.linkedin.com/app/answers/detail/a_id/33796

<comment><USER2> I would like them to integrate the skills/expertise set from stackoverflow.com for sw engineers. That probably adds more credibility IMO.</comment>


Big Data Points Humanity to New Minerals, New Deposits - <AUTHOR> https://deepcarbon.net/feature/big-data-points-humanity-new-minerals-new-deposits <comment><AUTHOR> That was the PR release, this is the paper:

[http://ammin.geoscienceworld.org/content/102/8/1588](http://ammin.geoscienceworld.org/content/102/8/1588)</comment>


Feds Bust Dark Web Child Porn Site Thanks to Tremendously Bad Opsec - <AUTHOR> https://www.vice.com/en_us/article/gyzkxb/feds-bust-one-of-the-dark-webs-biggest-child-porn-sites-thanks-to-tremendously-bad-opsec <comment><USER2> _" This is, obviously, a major and important dark web bust, but the indictment raises the question of why the DOJ allowed the site to operate for so long."_

FBI investigations typically take a long time because they want to make sure they can convict with a solid case. With so many suspects (53 beyond the site admin) I am not surprised they took this long.</comment>


Ask HN: I'm building (yet another) RSS Reader – should I quit or give it a try? - <AUTHOR> http://monoclereader.com/ <comment><USER2> Sure why not? I could always use another one for testing purposes. Although I have a feeling that based on the design, it won't work for me.

</comment><comment><USER3> There's an open-source reader called Monocle already, that's an unlucky naming collision (although somewhat expected if using normal language nouns): [https://github.com/aaronpk/Monocle](https://github.com/aaronpk/Monocle)

As a happy user of Inoreader, my main question about "yet another" would be: What can you offer that the established players don't have? Growth is probably fairly slow for traditional feed readers, since people likely either don't care or already have chosen one that works for them. E.g. the open source project linked above is part of an attempt to get social network style features into a reader: [https://aaronparecki.com/2018/03/12/17/building-an- indieweb-...](https://aaronparecki.com/2018/03/12/17/building-an-indieweb- reader), and thus has a clear answer for "why not one of the ones that has been around for years", even if many users of feed readers aren't necessarily interested in that.

<comment><AUTHOR> Thanks for the heads up, hadn't heard of that project before. I just took a look into it and well, I guess there's no problem with the name since mine is actually Monocle Reader (Monocle for short) and I've got the monoclereader.com domain. Might consider a rename down the line if this results in any conflict of course.

About your question, what do I offer that other established players don't have? It's a couple of things actually that I believe will make my reader unique:

1) Modern design

2) Simpler use and minimalistic interface

3) Cheaper paid subscriptions

4) Wide feature availability in the free tier

5) Pocket-like browser extensions for saving urls to a Read Later section

6) Aim at the regular internet user so it's really seamless to get up and running (not like aaronpk's Monocle for which you need to setup your own Microsub server and stuff)

7) Highly customizable: shortcuts, colors, layouts and other options

8) Add a tool to make a feed out of any website (like an integrated [http://fetchrss.com/](http://fetchrss.com/))

9) Save articles as PDF for offline reading or sharing

10) Probably offering an in-app reader but I also want to emphasis the importance of reading content on the actual website to show support and 'live the real experience' just how the creator intended to (and not just my parsed content, stealing traffic from his site)

It's possible that maybe Innoreader already has all this features? But still, I could still make a difference on how I present content, the general feeling you get when using the app and so on.

I'm really looking forward to your response since I'm trying to gather feedback and improve my reader as much as possible. Thanks!

PD: It's also my intention to lately approach and market this reader without making too much emphasis on the RSS thing, since people who don't know what that is instantly turn away, and just focus on the productivity and saving time benefits of using a reader like this (independent of how the tool technically gets the information)

<comment><USER3> _Purely from my perspective_ , 7-9 _might_ be interesting, I feel like the rest is probably not gonna differentiate that much from existing ones (although 4 of course depends on exactly what it amounts to).

</comment></comment></comment><comment><USER4> I hope that material design is temporary...

<comment><AUTHOR> What would you like better?</comment></comment>


The First Time I Used Programming At Work - <AUTHOR> http://justinkan.com/2012/5/15/the-first-time-i-used-programming-at-work <comment><USER3> There are also many people in biology research who do these kinds of repetitive tasks on daily basis for hours without realizing that much of it could easily be automated.

While we're sharing horror stories, a shining example from my experience is a friend who spends days analyzing western blots from experiments: it comes down to identifying a set of 10 rectangular black patches on clear white background and measuring their average brightness. The standard solution is this software package that lets you open each .jpg file one by one (there is a folder with hundreds), and you draw little rectangles and it tells you the average brightness inside. You write the number down inside an excel spreadsheet one by one, alt-tabbing back and forth.

After a brief facepalm moment I opened MATLAB, wrote ~30 lines of code and processed all images in the folder automatically, dumping all average brightnesses into a giant matrix in 1 minute of runtime. Several days of work saved that could go to curing cancer instead of filling spreadsheets.

<comment><USER4> These "wet" labs usually do not have many people who are enthusiastic about computers in the programming, text-only sense. Usually there is one or two people that know some stuff and they do all the technical stuff for everyone else. It's not uncommon to find people who are actually quite computer phobic in these labs. The general interest in computers is prefering Mac's to PC's and that's about as far as it goes. If you start talking up some programming solution to the wrong person you may actually alienate them. The problem extends to much of the software that is written too. There are some command line suites that do batch processing, e.g. for sequences, but much of the software is point and click. And that's the problem. No one expects the typical lab user of these programs to be versed in using the command line and doing batch jobs. Everything is mouse-driven. Even if the back end is powerful, the front end is always a slow GUI.

This just slows everything down.

Another shining example of this sort of thing is audio engineering. You have programs that are designed as GUI's that look like physical pieces of gear. You move the mouse around to adjust turn knobs, switches and sliders and adjust settings. Extremely cumbersome. But this is how they expect users to be comfortable. Then companies started selling physical consoles with real knobs, switches and sliders to control the software. It's comical.

The touch screen trend is not going to help people get comfortable with the command line. And the key to realy fast computing is using a command line and batch processing. There is no way around it. When you are pointing and clicking, you are just sending a signal that began as a text instruction in the code anyway. Why not just start with the text command and skip the point and click?

<comment><USER5> _It's comical._

Only to someone who's never mixed a song.

Professional audio engineers do their work by FEELING the position of the equipment, making adjustments, and LISTENING to the results. And they expect the feedback to be instant -- just like playing a musical instrument.

Now put them in front of a command line and imagine if they had to _type out_ their intents, listen to the results, and then make tweaks to their typed-in textual program to effect some change in the audio output. You'd end up with a bored, pissed, unproductive audio engineer. The sliders and switches made digital audio professionals MORE productive, not less -- even more so when they became controls on a physical console with digital outputs, as it put controlling the digital gear back in the analog realm a musician is already familiar with.

I find it also interesting that lab environments and music are where plenty of programming takes place by people who are not occupational programmers -- and much of it is visual, in environments like LabVIEW and Max/MSP. This is the _future_ of programming. It is why we should listen to those who say we are too bound to a textual representation for our programs and that in order for the craft to evolve, a richer representation must be embraced.

</comment></comment></comment><comment><USER6> Something I find amusing is that the ability the author demonstrates here, to notice that a task is repetitive and has a programmatic solution, is itself a skill that has to be learned, and one that's different than programming.

Once, when I was working on a group project in an operating systems class, we modified our kernel malloc and free routines to print the relevant address every time they were called, to do some simple leak-checking. The result was a file of a few thousand lines like "malloc 0x94583900", "free 0x34739A4", etc. One of my partners looked at the file and said some variant of "oh no - <USER2> is going to take hours to match up". Apparently it hadn't occurred to him, someone who'd been programming professionally for years and who only minutes before had been tweaking the internals of a process management function in a multithreaded x86 operating system, that there was an easier avenue available to him.

<comment><USER7> It's a mindset that's rarely taught: if you have a problem, and you're not writing code to find the solution, you're probably wasting effort.

<comment><USER8> Yes. But you can also waste lots of time creating code, when just doing it quick, dirty and manual is faster.

<comment><USER9> I agree.

But I'd add that even tasks that only require a quick-and-dirty solution can frequently benefit from quick-and-dirty automation. A few throw-away lines in an interpreter environment like irb, for example can often shave several minutes of certain tasks, and throwing a lit bit of code into such tasks can certainly make some otherwise dull processes more interesting.

</comment></comment></comment></comment><comment><USER10> I remember the first time I used programming at work too. I worked in a marketing agency as a PC operator. The two tasks I had were:

* print enormous text files with tables showing survey data.

* put a long list of .gif files each one on its own .ppt file (yes, as ridiculous as it sounds).

For the first problem I did two little Pascal programs. The first measured the files and calculated the best settings for the printer (it used to be a try and error process). The second, I did another little Pascal program that added page numbers to the files, so when the printer got stuck (or the pages got mixed) it was easy to resume instead of starting over.

For the second problem, it did an MS Office Macro.

I stopped programming at the office because one of my coworkers asked me to stop because "we are all going to be out of work" (sic).

That is when I learnt the importance of knowing how to code.

<comment><USER11> " _one of my coworkers asked me to stop because 'we are all going to be out of work'."_

This is the argument of the anti-technologist, sometimes called the "Luddite fallacy" [1].

Somehow these people think that the time freed by automating menial and repetitive tasks can't be used to do more productive or additional tasks, or heighten the ability and/or education of the workers.

[1] <http://en.wikipedia.org/wiki/Luddite_fallacy>

<comment><USER10> While that's true; If you look at how the surplus of productivity generated by technology was distributed among the members of the society in the last 30 years, you will notice that, maybe, she was right.

Who got benefited from the increase in productivity I generated with the automation I made? The owners of the company.

<comment><USER12> I automated an entire department down to a single person (me) -- this was during a severe downturn for the business with layoffs left at right. I didn't quite feel I was putting others out of work, as two jumped ship and one was retired out (comfortably).

I even significantly improved timeliness and accuracy. And I absorbed a major change in inputs that likely would not have been accommodate-able under the old system. Turned one of the high-priced consultants onto a reporting package that greatly improved their life feeding upper Management endless varieties of ad hoc reports. Etc.

And... this senior management, brought in from outside due to the share price, set up separate reporting channels using their people, and eventually laid me off.

These days, with so much top-down control, it's often a matter of who you know more that what, and how you are perceived. I was very good at what I did, but from their perspective, I was a "grunt" and a replaceable cog.

In the mainstream work environment, you can indeed work yourself right out of a job, especially if perceptions of you don't match a profile that Management is used to respecting and promoting.

I know I often come across as cynical, here. And certainly, I bring my own faults to the table. But... experience has shown me that these days, _still_ , image often outshines talent in the job market.

</comment></comment></comment></comment><comment><USER13> When I envision "Everyone can program." this sort of thing is what comes to mind. Not people building cathedrals out of notepad.exe and a python REPL.

<comment><USER14> In other words, "Everyone can script", rather than "Everyone can program". Where by "script" I mean to write a program, but for ad hoc tasks, and on a relatively small scale.

<comment><USER13> I use the word "program" pretty loosely. To me, a script is a program.

But for what it's worth, I think a working knowledge of the gnu utilities is worth more than most programming languages to most people. (If you're on a system that supports *nix utilities.)

Ideally it's not an either or thing.

<comment><USER15> How would you recommend gaining that knowledge?

<comment><USER13> This free book:

<http://linuxcommand.org/tlcl.php>

The best way I know short of immersing yourself in a community of *nix geeks for a long period of time.

If you were looking for something more grandmother or time friendly, I'm afraid I can't help you.

</comment></comment></comment></comment></comment><comment><USER16> My mother was a computer science professor and was active with ACM's SIGGRAPH for most of my early childhood, but somehow I got through high school never learning how to write code. I did, however, learn that laziness can sometimes be a virtue.

In my second job out of college, I was running the sound board for a national talk radio show. We broadcasted three hours a day, and I was tasked with getting recordings of each day's shows uploaded as a podcast with the commercial breaks edited out. Being far too lazy to want to do this by hand, and recognizing that we broadcasted on a "hard clock" (a commercial break would run every day from 2:06:55 to 2:10:25 for example), I decided it was time to learn how to let a computer do the work for me.

Fortunately, a friendly coworker who was a hybrid of broadcast engineer and sysadmin pointed me toward bash scripts and cron jobs. Figuring it all out was a frustrating couple of weeks, but I certainly didn't regret it.

A few years later, among other duties, I was responsible for the audio archives of a very old musical organization. Mostly this meant knowing how to make decent transfers of DATs, LPs, 1/4" reel to reels, etc. We found ourselves with over 20,000 tracks of original recordings in .wav format and the metadata about those tracks in an unconnected database.

After far too many searches in the library database, followed by finding the desired file by hand, I got busy scripting (and perusing posts on HN that have helped immensely along the way). That project has evolved into a searchable website containing audio, video, photos and print materials of everything in our catalog.

Next week, I'm on my way to Silicon Valley to interview for a programming gig at a company doing very exciting things. I definitely didn't see it happening this way, and maybe it's not the best career move for me, but my life wouldn't be the same if I hadn't realized my laziness could be rewarded by learning how to code.

<comment><USER17> Good luck next week.

<comment><USER16> Thanks!

</comment></comment></comment><comment><USER18> When I was 16 I had a summer job as an intern for my Physics professor. Our main task was to stir these huge vials of very viscous liquid for hours on end.

So I went to the Chemistry lab, got some mechanical stirrers, plopped them in, and played video games all summer. Good times.

</comment><comment><USER19> Recognizing that automation is possible is very hard. A quick tip, if you can find a pattern in a task, you can probably automate some or all of that task.

My first internship after college had me taking technical tables in old Word documents, and rewriting them as old-style HTML tables. These documents were often thousands of pages of tables, and there were hundreds of such documents.

It took me a while to realize that, over a few dozen tables, I had come up with a manual system that was just a series of copy/pastes of various HTML snippets in and around the copy/paste of the table contents in a text editor. At the rate I was going using my copy/paste method I was processing tables at about 2-3x the rate of the other interns who were hand typing the markup in the table (the average was about 20 tables per day, I was doing a solid 50-60).

Thinking for a minute I whipped up a script to do the work and my metrics went through the roof. All I had to do was copy/paste the table contents (I couldn't come up with a way at the time to automate that bit of tedium), then run the script. In a day I went to doing 6000-8000 tables a day. My boss, who fortunately thought it was pretty cool, had me train the other interns and we finished the project months ahead of schedule.

So even if a complete solution wasn't to be found, automating even part of it was a massive boost to productivity.

</comment><comment><USER20> I am trying to use programming at work, but I feel the thing is seen as mere "IT" and that it has no consideration at all. Which is weird, as we are sharing horrors, now at my company (a multinational) is budget time. There is an excel file with 500 tabs (500) going around to collect the budget for each cost center.

I mean, when you start clearly overusing a tool, if you call yourself a manager you must realize it. You can't keep saying that your job is not IT because at the end your and others productivity depends on it. Somehow I start thinking that the way you do things is as important as the things you are doing.

Bottom line is, we will come up with a budget, everybody will work a lot over the 40 hrs per week, there will be lots of input and logic mistakes (most of them will be corrected, others not). But they will be able to check the 'budget done' radio button. And they will be proud of this achievement. I won't, because I will be feeling stupid spending time doing this kind of things in 2012.

</comment><comment><USER21> I've been interning at a power company since high school (I'm a junior in college now -- EE/CpE major) and one of the things I did was taking log data for a bunch of events in csv format, filtering the irrelevant data, and formatting it to make it more readable by humans. Since I had to repeat the process a bunch of times, I spent an hour or so writing an Excel macro that let me select a few examples of what was relevant and then proceeded to do all the hard work. I told my supervisor about it and she had me write up documentation for using the macro and then shared it with everyone in the department to save everyone's time.

<comment><USER8> That was one good manager!

</comment></comment><comment><USER22> Excel can be an amazing tool when automated. I am amazed that more companies don't train their employees on task automation (Excel or not).

A few years ago I had a situation while working on a circuit board design. The board used huge FPGA's with over a thousand pins each. Creating and managing the schematic and PCB symbols for these chips took days of agonizing work. I finally had enough and wrote a tool that used Excel for data entry and editing. After about a month of work we could take an FPGA from data sheet to EDA files inside of 30 minutes. The tool most-definitely paid for itself many times over as it saw use over several years.

That's the one thing that is outstanding about MS Office: The ability to automate across tools is fantastic.

</comment><comment><USER23> At this place where I'm currently working they have this legacy system which logs transactions, and sometimes there are lines missing from the logs.

Early on these guys figured out they could recreate the missing lines using the data they did have.. the problem is that it's the most horribly tedious process imaginable, involving lots of calculations and comparisons.

One day one of the guys runs into a problem where 3-4 of these transactions hadn't been logged properly. I didn't have much to do that day so I opened up Visual Studio and got to work. Field by field I figured out what the calculations were supposed to be and then put them down in code.

Saved this guy hours of work that day, and we save ourselves time every time the problem has cropped up since.

Oh to be a programmer.

</comment><comment><USER24> The IT manager that didn't believe in him are the types of people that can't embrace technology, and when you don't embrace technology or at least try, you end up getting fucked. She could have told him to prove it, show an example, etc, but instead she automatically dismissed it and he got the better end of the deal by getting to be lazy for a few days. If she would have looked, commended, and appreciated what he spent effort trying to improve it would've had a better outcome for both parties, but instead he was the only one that got to take advantage of it.

</comment><comment><USER25> Yep. I got sick of entering the same data six times in spreadsheets at work. I made an Access database. Added VBA to make it better. Ended up hitting the limits of VBA, made a full client app in VB.NET. Ended up importing data from a remote system using an AutoIT script. I was still an admin assistant- no-one would promote me.

That was the beginning, I never looked back.

</comment><comment><USER26> Wonderful post. I love that he solved the problem, and since he received no reinforcement, kept the success to himself and did what ever he wanted while the program did his work. Win X 5

</comment><comment><USER27> I had a very similar experience! I used to work holidays for the family mail order business.

While we could easily process upto 400 small orders a day (because the warehouse was well organised) we maxed out at shipping around 6 large trade orders per day. Each one took an hour and the courier collection was at 3pm.

After getting through a particularly manic afternoon, I decided to have a look at why they took so long.

I realised it took 25mins just to turn the mailorder spreadsheet into an invoice to mail out with the parcel.

I was dying to automate it! I made a macro to remove the empty product lines, format the layout, background & colour, position the delivery address, insert the company logo and information and generate an invoice number and mark it as paid.

I even future-proofed it so that it would still work even if we added new products to our order form.

I stuck a big smiley face button on the tool bar. I tested it out, invoices now took around 4 seconds. The screen sat there with a beautifully formatted invoice, cursor flashing patiently for me to enter a payment reference and hit print.

Basically that one small change doubled the trade order capacity of the business, which was a LOT of income. It's an awesome feeling :D

That was about 8 years ago, I think they still use it now!

</comment><comment><USER28> The sad part is that they probably went back to doing it by hand once he left the company.

Also amazing that "normal users" consistently don't even want help with automating their IT tasks. Maybe they are worried about losing their jobs?

<comment><USER8> Or they just don't know what is automatable? Or they are worried they won't understand the new system?

<comment><USER28> In that story the boss didn't want the automation, and I made the same experience whenever I offered to automate the tasks for somebody else.

</comment></comment></comment><comment><USER29> Really I think we just need to use the term Automation for this, rather than "coding" or "programming".

My take on this latest Jeff Atwood fiasco[1] is that yes, I certainly agree with him that "coding" should not be lumped in with the same concepts as general literacy and numeracy.

But certainly _automation_ using a computer should be, and there are a range of ways available to teach people how to do that.

[1][http://www.workingsoftware.com.au/page/When_does_automation_...](http://www.workingsoftware.com.au/page/When_does_automation_become_coding)

<comment><USER30> That's really a distinction of degree, though. It seems like the only real difference is whether you're programming Python, Photoshop, Excel, bash or whatever, and the power of the tools available to you. I'm not sure that whether it actually involves writing text to be interpreted is important.

<comment><USER29> I think it's not a distinction of degree any more than the difference between algebra (which everyone is expected to know) is different from calculus (which most high school graduates are exposed to) and then fourier transforms and vector calculus (which 2nd year university students are expected to know).

I think something like "basic automation" provides the same type of introductory computer literacy as algebra does for maths, and "learning how to program" is similar to a continued education in maths.

<comment><USER30> Different syntax/symbols, different levels of power, yep, pretty much the same deal. Different levels of math are, in a practical sense, different degrees of the same thing. There are different kinds of math and there are different kinds of programming/automation, sure, but it mostly feels like "this is harder/easier/more intense".

"Automation" may well be a good bait word for luring in unsuspecting non- nerds, but it's still lightweight programming, a subset. I think at this point we're in agreement.

</comment></comment></comment></comment><comment><USER31> Before I started studying computer science in university, I was working as a laboratory assistant in a paint coloring laboratory. One of my tasks was to do colorant characterizations to our custom-built (the company had one programmer) application so the software can approximate formulae for hardware stores.

This was really manual work. You needed to do different mixtures with the colorant and the paint to get an even spectrum of colors from light to dark. Everybody just started to do them randomly, calculating approximate mixtures, doing draw-downs, measuring and repeating until the result was good enough. I started playing around with excel macros and studied how Kubelka-Munk formulae worked. I even realized that there are lots of colorants using the same pigment, but with different strength.

When getting these numbers and some old measurements I built a big excel file with macros, where you just type the pigment data and it would print you instructions what to mix and how much. This saved several days in the process and finally they let me learn Delphi and convert the excel file into a real program.

I wouldn't want to read that code nowadays, but I think they're still using it in the laboratory.

</comment><comment><USER32> Yeah, it's really useful to be able to automate tasks, especially when you're the intern that gets handed the boring task the higher ups don't want to do. An intern's job is basically to do the lowest level tasks that computers can't do. And yeah, sometimes (ok a bunch of times) your superiors have no idea how simple that task is on a computer. However, other times the task is just complex enough and happening rarely enough and your skills are not good enough to justify an automation scenario. Then it's really boring. I would say learn programming to raise your complexity bar on repetitive tasks.

</comment><comment><USER33> know your tools. i had a colleague who needed to xy plot some data from an instrument dump into excel. problem was, the data and timestamps weren't one- to-one. he was looking at a few thousand rows of timestamps with almost random gaps between instrument readings. he'd psyched himself up to spend the next day right-clicking 'delete row' to get his data set. select-column, goto special, blanks, right click delete rows. seriously, the guy almost had tears in his eyes. i'm with the comment "learn how to automate using scripts" - should be a core skill

</comment><comment><USER34> I liked this post a lot. I notice that most everyone mentions the time saved by automating tedious tasks, but not how such automation eliminates user- error. For me, that's at least as important for maintaining peace of mind.

</comment><comment><USER35> there was an assignment to print 10000 random number.

a guy used excel, put a formula into cell A1, and drag it to A10000. file- print ... presto ... wait for 200 pages or so

i wrote a one liner matlab and printed the result in about 20 pages

i just happened to know little bit of matlab -- and that's all needed to 'beat' the competition by 1000%

<comment><USER36> The way I see it you came out about the same, both getting things done in under five minutes. Excel guy could have dragged from A1 to J1000 if he cared about page count.

</comment></comment><comment><USER37> This is by far the best entry in the "learn/don't learn to program" series!

</comment><comment><USER38> ice t would be proud of you <http://www.youtube.com/watch?v=mAlzPgXb6rE>

</comment><comment><USER39> This is a pervasive problem, one of the prime examples of why knowing the basics about algorithms and programming is a great skill to have.

Many people do not think about whether they can automate certain things. Even if you tell them too, they will often not have the right mindset for it and be unable to recognize that something is automatable.

This is not the fault of those people, it's a fault of education. Automation is not intuitive, it's not something humans understand instinctively. To know what works and what doesn't, to know what's possible, people need to learn the basics about how to code. (Maybe some will even be able to do some of it themselves, while others can at least ask around for an actual implementation.)

No, it's not the solution to the failing education system in the US. No, it's not the best thing since sliced bread. It's not the savior. What it is, though, is just a good idea.

<comment><USER40> > This is not the fault of those people, it's a fault of education. Automation > is not intuitive, it's not something humans understand instinctively.

Actually, I think a large number of people who are good programmers figured out automation on their own, meaning the education system is almost entirely useless when it comes to this. We all have our own stories for this moment of enlightenment - mine was writing TI calculator programs to "automate" problem sovling in middle school algebra.

<comment><USER41> Well, yes, "people who are good programmers". The point here is though that the others should get an idea about this. There will always be people who don;t understand the easiest of procedures, but I am sure a large number of people would understand automation if the idea was given to them (at an early age). Probably would they then think about it when the need arrises, and maybe come up with some creative solutions. If we just leave it to the natural good programmers, the solutions may be better, but the number will be lower.

</comment></comment></comment><comment><USER42> I think the most important part of this post is "Well, fuck that."

Update: Actually it's this part: "Well, double fuck that."

<comment><USER43> I had this same moment at work not long ago. My boss gave me a task to compare user permissions exported from a directory (not a database). The output was ridiculous and it could easily take 10 hours to work through lining it up and correlating just 5 users. They had been doing this by hand for 6 years before I got there. Within a few minutes of her explaining it to me, I flat-out said "I don't want to do this, so here's my plan". 30 hours of Bash scripting later, I had the task completely automated, and we could run through 20 users in less than 2 seconds, with the output going straight to Excel.

You can't be afraid to tell your boss you think a task is ridiculous, but you'd better be prepared with a better solution.

<comment><USER44> I once worked in an office where at the start of every month we'd get a cell phone invoice for 30-some phones listing every call made in the last month in an Excel document. One of the secretaries would then print out the whole thing spend the next two weeks going through it with multiple colored markers annotating it with who had which phone one which days and checking for different classes of violations of the cell phone usage policies.

Another secretary realized that he could make it a lot faster by just copying and pasting the data into different sheets for each person and then highlighting it in Excel. Then he wrote a macro program that would automate the highlighting once you'd done all of the copy and pasting. This cut the total work load down to a few days.

I was very derogatory of the whole thing and insisted that it was absolutely appalling that humans were spending so much time doing obviously machines' work, so I got assigned to fixing it. I ended up with an HTML+JavaScript application that would read the invoice file and monthly phone directories to automatically match up names with numbers for every day, calculate policy violations, and write the results to a new Excel spreadsheet in the established format. It cut the total human time involvement down to 5 minutes per month and time to wait for results down to about an hour.

Once we had the phone records available on the first day of the month, demand suddenly rose for even more detailed and different kinds of analyses. So it turns out we still spent more than two weeks on making reports, but only because new ones kept being demanded since they could actually produced on demand.

</comment></comment></comment><comment><USER45> two words to explain the law firms reaction:

billable hours

*credentials/disclaimer: I'm a programmer and I worked as a clerk for law firms back in college too.

<comment><USER46> The hours for this task weren't billed.

<comment><USER47> He means that the law firm makes money by billing hours; hence if you take less time to do a task, the firm earns less money.

<comment><USER8> Although in theory they could just up their hourly rate, if their hours are more productive, now.</comment></comment></comment></comment>