The threat of technological unemployment - <AUTHOR> http://lemire.me/blog/2016/12/26/the-threat-of-technological-unemployment/ <comment><USER2> It's not just the existence of technological unemployment, it's the pace of it. Over the past two centuries, we've shifted from an economy where virtually everyone is a farmer to an economy where hardly anyone is a farmer. We've mostly automated food production and distribution. But it took generations to do it.

Today, people embark on careers in their 20s that disappear in their 40s and 50s. Such a pace of change is distressing. Consider how many people drive trucks for a living. In another generation, that job will be 90% wiped out. It'll be local short hauls only, places where there's a lot of human interaction and judgment calls. Long haul semis will be entirely automated. It'll save money and it will save lives, and it will be a huge boon to everyone in society - except professional long haul truckers, who will lose their livelihoods. They won't take it well.

Given an opportunity and a mechanism to retrain or switch careers, people can do that. But what if it takes a year or more to retrain? How will they pay the mortgage? How will they feed the kids? People who can't care for themselves and their families feel weak, vulnerable, and ashamed. They'll be angry and act out. They'll listen to political demagogues who tell them what they want to hear and give them scapegoats to blame. Such a sudden social change is politically dangerous.

<comment><USER3> A lot of the fear that comes from tech unemployment is based on predictions of the future, which are notoriously horrible.

You give the example of trucking jobs which will be 90% wiped out. This hews to the conventional wisdom you find on HN, but you'd be hard pressed to put out real evidence that this will be the case on any identifiable near-term timeline. I'm fairly sure there are exactly zero US states where bot trucking is presently common. Going from "this technology is getting good" to "soon it will own 90% of the market" is a pretty huge leap and bold prediction. Tech readiness is only one small factor in whether or not this even could happen.

Bold predictions are, with hindsight, usually wrong.

A lot of these fearful arguments boil down to a simple syllogism; "if present trends continue, then (bad outcome here)". The trouble with that is that trends in general don't continue. Look back on the future predictions of 50 years ago, from any moment in time you choose as your reference point, and you'll see uniformly that the future predictors expected certain trends to continue which didn't, and missed the existence of other big trends that ended up as primary drivers.

People just don't know what's going to happen in the future. I suppose guessing is better than doing nothing, but we shouldn't think that the future guessing we do is actually good or accurate, past experience teaches it's actually really far off.

<comment><USER4> The fear comes from the unknown. Appalachia in the United States is what people dread. High unemployment, jobs never coming back, addiction to pain killers killing you in your 30s and 40s. Really, I suggest anyone on Hacker News to take 2-3 days and drive through parts of West Virginia, during the daylight only, to see, physically see, what the commotion is about. [1]

If we had solid safety nets, this wouldn't happen. People wouldn't be worried about being homeless, hungry, and without a support system. People would embrace automation.

But that's the problem. The Technologist's Libertarian Outlook. Disruption is okay, just help yourself to some bootstraps, and if you die in the street you weren't valuable enough.

Protect against the unknown, and people will be less anxious about rapid change. Don't protect them, and risk social instability. This is what government, social programs, and taxes are for. Or we can wait for the 21st century equivalent of the guillotines to come out.

[1] [http://www.post- gazette.com/local/region/2015/05/31/Littleto...](http://www.post- gazette.com/local/region/2015/05/31/Littleton-W-Va-is-a-town-decimated-by- poverty-drugs/stories/201504280190)

<comment><USER5> _If we had solid safety nets, this wouldn 't happen_

It depends on their form. People aren't on drugs because they don't have enough money. People are on drugs because their lives lack meaning, and simply getting a government check won't change that. A man cannot provide for his family by simply being alive to cash a check.

There are things that look a lot like welfare to the government that don't look like it to the recipients, though. Wage subsidies could keep a lot of the people who are below the threshold of self-sustainment actually engaged with their communities.

And, for better or for worse, most people engage through their communities with employment. Unemployed people, especially unemployed men, do not suddenly become poets or volunteers for Meals on Wheels.

<comment><USER4> I appreciate you poking holes in my assumptions and suggestions, seriously, because I don't know how to solve this.

I know I don't want parents/grandparents ODing on heroin in a parking lot with a child in the back of the car. [1] So where does society start to fix this? How do you give people purpose and the means to embrace that purpose?

What good are self driving cars/trucks, machine learning/AI, and all that comes with it if you've hollowed out any substance your species had? EDIT: Maybe the Amish had it right the whole time. Social fabric above "progress".

[1] [http://ktla.com/2016/09/09/ohio-police-post-photos-of- adults...](http://ktla.com/2016/09/09/ohio-police-post-photos-of-adults-who- overdosed-with-4-year-old-in-backseat-of-car/)

<comment><USER5> Make labor cheaper. Not for the worker, but for the employer.

1\. Wage subsidy. Tax people like me more to increase the EITC. At the extreme this is make-work jobs, but there is a lot of ground to cover between here and there.

2\. Make employment cheaper. Cut down on employment taxes and other expenses associated with employing people.

3\. Make employment easier. This won't happen overnight, but, after coming to wage and term agreement with someone, I should be able to hire someone by clicking a few links on a government webpage. And I say "I should" as a person, not as a corporation. Make everyone in the country able to hire another person quickly and easily. If I see too much trash on a street in my neighborhood, let me hire people to clean it up a few times a week. I shouldn't need any kind of HR department.

<comment><USER4> I disagree with you here: labor should be more expensive so make work jobs (telemarketing anyone?) are driven out of the labor marketplace. I'd rather the government provide jobs in a transparent manner that build, improve, maintain infrastructure, national parks, etc. There's no reason we can't transition from consumers to stewards of our local communities and national resources.

I have yet to see an employer who is friendly to labor, so that's where I'm coming from why I don't trust employers to be the solution (removing minimum wage, wage subsidies, etc). Labor regulation should walk softly but carry a big stick.

Example: We subsidize Walmart employees pretty heavily with government resources. We should not be doing that, and I highly doubt most people in those jobs feel as useful as they could be.

[http://www.msnbc.com/msnbc/walmart-government-subsidies- stud...](http://www.msnbc.com/msnbc/walmart-government-subsidies-study)

[http://americansfortaxfairness.org/files/Walmart-on-Tax- Day-...](http://americansfortaxfairness.org/files/Walmart-on-Tax-Day- Americans-for-Tax-Fairness-1.pdf)

<comment><USER5> Telemarketing isn't "make work." They are providing value to their employer.

You are hitting on one of my concerns, though: make labor cheap, and people could be hired to basically annoy other people. I want to keep it simple, but there are certain categories of jobs that need to be ineligible for wage subsidy: anything that involves attempting to grab someone else's attention or time. So no sandwich board walkers, no demonstrators, no telemarketers, no door-to-door salesman, no picket walkers. Those can still be employed by businesses if they want, but without subsidy.

If you don't trust employers, you can bid up the wages of labor yourself by hiring people. Think what a non-profit interested in increasing recycling rates could do with access to cheap labor. In fact, all sorts of environmental issues are reduced by cheap labor. We can stop dumping old VCRs on China to disassemble. A local environmental group can collect them, and then use a disassembly line to teach workers the basics of technology repair while breaking out all the components for responsible disposal.

If you want to keep people employed but not willing to risk employers possibly benefiting in any way, well, it indicates that improving the standards of living of the workers wasn't your real concern. Keep in mind that "the government should take care of paying for health care" would fit in with my recommendations but run afoul of your objections.

<comment><USER6> In China, when [say] 100 people are about to get fired government will negotiate a deal. That might be to have them hire 300 more for at least 2 years, get 25% of the salary in advance and have them provide training for 200.

I don't remember the exact numbers but it was some kind of custom combination like that designed specifically for the type of business.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> As I understand it (and I say this as a fellow "techno-optimist"), one of the most significant challenges is that cultural and political change occur much more slowly than technological change. So it's not that we _can 't_ adapt, but that adaptation is exceedingly painful to various segments of the population. We tend to accept certain facts as "permanently true", which ends up tearing holes in society and in our lives when these facts inevitably change. For example:

        * Building a long-term career at a single company is safe.
        * Traditional college degrees are worth the monetary cost.
        * Society can only function if everyone works a job.

I'm a bit skeptical that "we'll always find a way to quickly replace jobs" will stand the test of time, especially when the primary evidence given is, "Well that's how it's worked out so far."

<comment><USER8>
       I'm a bit skeptical that "we'll always find a way to
       quickly replace jobs" will stand the test of time,
       especially when the primary evidence given is, "Well
       that's how it's worked out so far."

Perhaps your skepticism can be allayed by looking at the mechanism. Money, in the form of capital, flows through an economy from providers to consumers. With a functional banking system capital doesn't accumulate. Even when a company like Apple has $200B in "the bank", that money is being lent out to others to build things and start businesses etc. Building stuff consumes labor and that transfers money back to the labor force which then uses it to purchase goods and services. All basic economics but the key is that the existence of money (capital) creates an economic force. Banks cannot simply sit on a pile of dollar bills and pay out 1.2% interest on that, they would go broke. They need to make a return on it so they find people (economic force) who will use the money to create additional value.

In many ways its like "running out of water" which on its face is preposterous. The planet is literally covered in the stuff. The mechanism (evaporation and condensation) which move it around the planet can put too much or too little somewhere but we'll never be permanently "out" of water. It will rain somewhere at some point (or snow) because water is evaporating from the oceans every day.

Economies are very much like that as well, they may have capital dry up in one area (like Appalachia) but it is probably raining somewhere else (like San Francisco). But the mechanism of buying and selling products or services is a constant force to move the money around.

So that money will always find a way to be spent. And that means jobs. They may be different than the ones before but they will get created.

<comment><USER9> I don't disagree with what you've stated, but I think the disconnect with what you've stated and what the GP stated is that a specific unit of money doesn't buy/pay for the same amout of jobs when applied in different areas (whether "area" be geographic or sector). For example, taking money that previously was applied to 1000 coal miners and applying it towards a software company yields far fewer software engineers.

Additionally, people are a resource, but a very finicky one. You would expect them to move to where there is a need for them, and to do what it takes to make themselves usable, but there are many external factors that make that far less likely than it could be, such as family, property, and various features of human nature. So, as money flows from labor-intensive industries to non- labor intensive industries, or transforms a previously high-labor industry into a low-labor one, what happens to labor as it increasingly loses value? What happens when the value of labor falls below the threshold required for human survival for increasingly more segments of the economy?

<comment><USER8> Fair enough, but this "what the GP stated is that a specific unit of money doesn't buy/pay for the same amout of jobs when applied in different areas" is the basic fallacy. It is misses the point.

At a microeconomic scale, people are displaced. We saw last week an article on one of the few book binders left in Manhattan, there used to be hundreds of book binders. They stopped binding books. But the number of jobs in Manhattan didn't go down to n - (# of book binders) because in a _macro_ sense what used to be spent on book binding is being spent on something else. Maybe it created a thousand jobs at print shops (for example).

We completely agree that people aren't fungible. Just like cattle die in a drought, people without marketable skills go hungry and homeless. I, and many folks I have talked to agree that one of the reasons you tax business is to fund transition plans for people being left behind by changes in the job market. Whether that is retraining programs, temporary shelters, or tuition assistance for additional education. In my mind I consider that social justice matter to be independent of the macroeconomic mechanism of job creation.

<comment><USER9> > But the number of jobs in Manhattan didn't go down to n - (# of book > binders) because in a macro sense what used to be spent on book binding is > being spent on something else. Maybe it created a thousand jobs at print > shops (for example).

But the assumption here is that there are still lots of jobs that need people to shift people into. As a resource, we're consistently reducing the areas in which it's appropriate and cost effective to use people instead of some other resource.

This happens. Resources that were previously used for many things are displaced by other resources which are fundamentally better in those categories. Think stone age. Eventually stone as a resource was completely surpassed in almost every area it was previously useful in. It's value has been reduced to the point that if you have a large quantity of unsorted stone material, it probably costs more to sort the useful bits than you can make selling them, and without a _local_ market that wants to use it, you're stuck trying to figure out the least costly way to either store or dispose of it.

I hope we aren't transforming ourselves into the equivalent of stone, where we end up that figuring out which select few of us is capable of contributing in some way is more costly than producing a product slightly worse quality but much cheaper using automation the majority of the time, for the majority of fields.

Hopefully I'm dead wrong and we transition to something far more uplifting, because it's depressing to think this way.

<comment><USER8> I don't think you have to assume that _at the time of the shift_ there are lots of jobs to shift into, only that as it shifts the economy will create new jobs. Further I think those shifts are on the order of years not days, as things shift over its going to be useful to cross train. For example, right now there is a possibility that trucks can become self driving so if you drive a truck you need to think of three things; 1) How much longer might this be a viable way to earn a living, 2) what sort of new jobs are currently being advertised as unfilled, 3) what resources can I apply to cross train into something different than truck driving.

</comment></comment></comment></comment></comment></comment><comment><USER10> The idea of transemployment (work that is not needed, but is created to prevent unemployment) is a terrible idea. Have we really become so brainwashed that we have forgotten that the entire purpose of work is to create value?

Simply letting people not work doesn't remove incentives to perform: the incentives that we have are already broken. We don't live in a meritocratic economy: the Monsantos and Blackwaters of the world are rewarded for destructive behaviors, while advertising allows inferior products to drive superior products out of business. If we have enough surplus to support these massive inefficiencies, surely we can spare enough to support the unemployed.

Further, transemployment actually costs us. It requires infrastructure to create fake jobs.

<comment><USER11> > If we have enough surplus to support these massive inefficiencies, surely we > can spare enough to support the unemployed.

This is demonstrably not true. I mean, it's tautological. We have the economic surplus to support the unemployed, yet we do not. Therefore, the problem is not the lack of economic surplus.

People have noted this apparent perversity for well over a century, but it seems unlikely to change anytime soon. So the question isn't whether transemployment is preferable to some lower-friction redistribution scheme, but _whether transemployment is better than letting those same people starve_ , or die from opiate addition, or alcoholism, or diabetes, or any of the other methods of benign neglect by which society has found to execute the economically redundant.

Also, I'm not sure that "the entire purpose of work is to create value" really encompasses the role of "work" in American society. The creation of value is, to the worker, often a side-effect, since that value in the capitalist model generally accrues to someone else besides the worker. The purpose of work for most workers is to bring in _wages_ , which are necessary for personal comfort, but more importantly (above a certain level) as chips in a zero-sum game of social hierarchy. Extracting value from labor is the province of the employer; extracting wages from time spent, regardless of value created, is the province of the worker. The capitalist market's labor economy is the resulting intersection.

<comment><USER10> This is just semantics. Typically wages are valuable--I don't know anyone who is being paid in smiles. Ostensibly when work is done and paid for, value has been created.

</comment></comment></comment><comment><USER12> Unemployment is not the problem. The problem is that the surplus / wealth created by technology is controlled by the 1%. Redistribute that wealth and the majority of the workforce will no longer need to be truck drivers, or farmers, or laborers. Then, we can start paying people to learn.

<comment><USER13> I make a good standard of living for my location and I don't really ever spend money so I'm saving it for retirement. If I invest it through standard channels then that mainly benefits the 1% because they control investment banking.

I'm thinking that one of the most disruptive things we could do is to invest our money in nonstandard channels. For example, promote systems that solve problems once and for all through research and then place the solutions into the public domain. A handful of engineers working over the span of months or a few short years could automate everything from wifi mesh networks to drone delivery to robotic home gardens. In a very short time we could remove basic need dependency from the system. Kind of like "a penny saved is a penny earned" on steroids.

Once people are not dependent on a system that tends towards wealth inequality, it would give them the leverage to demand a basic income because they could choose to no longer buy the goods and services from which the 1% derives its income. I'm not saying that members of the 1% are bad, just that whatever benefits they bring society will likely be commoditized like everything else when full automation provides cheaper (or free) alternatives.

<comment><USER14> The 1% have to much power to slow down and to capture (as soon as possible, so when states comes in) that kind of initiatives.

Look at the stories of alternatives energies.

The trick is here, I think too, but really hard with no free time.

edit: Maybe the automatization of the expropriation of the 1% won't be captured..

</comment></comment></comment><comment><USER15> This is a _horrible_ assessment of whether we are facing technological unemployment.

 _There are more bank tellers in the US than ever before_

But.. there are more people than ever before. And the number of bank tellers is leveling off even as the population grows.

 _Overall, there is scant evidence that we are undergoing a technological- unemployment crisis, if only because unemployment rates are low._

Are you _kidding me_? (First, you have to always be careful with any metric where the collecting agent has even the slightest bias to the outcome. Unemployment is not the obvious number you might think it is for a variety of reasons.) Jobless and underemployment are the numbers you want to be looking at -- and good luck getting good numbers on those but when you do, I think they'll tell a very different story.

<comment><USER16> What used to be a "bank teller" is now a sales rep, trying to sell you on loans and steer you to investments. Bank of America is rebuilding their branches into a new design. There are maybe two teller windows in the back for the occasional non-ATM cash transaction, while the front of the branch has little glassed-in sales offices like a car dealership.

A lot of employment is in the marketing overhead of capitalism. There are many products, from movies to Internet access, where the marketing cost exceeds the product cost. This is zero-sum; most marketing in the US just moves demand from one product to another. (Spending is income-limited in the US; the savings rate is very low.) It's also why people are still working so hard. Marketing is inherently competitive, which means it can chew up resources out of proportion to the value produced.

<comment><USER11> One of the issues with the "new economy" / "service economy" model that has been pushed heavily over the past few decades is that it doesn't distinguish between zero-sum jobs that have as their end goal merely a reallocation of a fixed amount of resources, and jobs which create value and are basically industrial in nature but where the end product is intangible (e.g. movies, television, writing, software development, etc.).

State and local governments have spent vast sums trying to attract "new economy" jobs to replace decimated primary industries, but I question how many of those jobs are marketing or "economic overhead" related. That's not to say that they're bad jobs at an individual level, or there's anything wrong with anyone working in them, but it doesn't seem like it's a sustainable basis for a national economy.

</comment></comment></comment><comment><USER17> > Some authors fear that technology will result in a radical concentration of > wealth, the like of which we have never seen… while a few people will be > super wealthy, all of us will slowly starve. Except that fewer people than > ever in history are starving!

Starving is how revolutions happen, so probably not literally starving. But poor people in the US are already wholly politically disenfranchised, and children of poor parents likely to be poor.

The two questions which are important to consider in the light of extreme automation: "how large and diverse is our oligarchy?" and "how much social and economic mobility is there between generations?".

We are well on the way to having a small oligarchy like Russia and mostly inherited wealth, at which point our society will simply cease to exist.

</comment><comment><USER18> I would like to see a little more serious attention to the "participation" rates in the workforce. Author is a little too reliant, for this particular topic, on the 5% unemployment rate.

<comment><USER19> The recession certainly did a number on the participation rate, but the current rate is more or less organic (as evidenced by recent gains in wages once unemployment stabilized at ~5%).

A lot of baby-boomers aged out around during the recession, and will continue to do so (people born between 1943-1951 all turned 65 during the recession), and a lot of people went back into school to "hide" from it.

_Under_employment is still a serious issue, but that's only reflected in labor statistics by median wages. On the bright side those have been going up too, but aggregates only tell us so much.

<comment><USER20> I don't know what "more or less organic" means here. Using the prime age (25-54) employment rate filters out that noise, and it shows that we've retraced about halfway from the worst of the housing crash.

[https://data.bls.gov/timeseries/LNS12300060](https://data.bls.gov/timeseries/LNS12300060)

edit:

...which is really terrible if you look at it in the longer term:

[http://www.epi.org/blog/prime-age-employment-to- population-r...](http://www.epi.org/blog/prime-age-employment-to-population- ratio-remains-terribly-depressed/)

</comment></comment></comment><comment> awinter-py hmm -- his 'transemployment' concept where complex processes are created and assigned to people as 'jobs' is brutal. We've all worked at process-heavy orgs and most of us have left them.

Process-heavy orgs will get out-competed by orgs managed almost any other way. If you're optimized to create useless work for people, you'll probably lose to an org that's optimized for something almost as silly like optimizing for high cost of labor. My consultancy is optimized for high cost of labor and I do okay.

Some people argue this is what's happening to France now. 40 years of shaping labor law to improve retirement & reduce unemployment has taken its toll. Paris is a smart tech-savvy and beautiful city that could have been Europe's silicon valley, but you're not allowed to work more than 40 hours a week and you can't fire people. Ah, 20-20 hindsight.

</comment><comment><USER21> "If human labor were to become wholly unnecessary, we are going to substitute for it through transemployment. That is, we will create work that is not strictly needed."

In other words, bullshit jobs: [http://web.archive.org/web/20161017003907/http://strikemag.o...](http://web.archive.org/web/20161017003907/http://strikemag.org/bullshit- jobs/)

That's fine if employment becomes fully optional, but I fear we'll be forced into a life without the meaning of useful jobs, and without the freedom of technological unemployment.

<comment><USER2> Bullshit jobs are about the social order, not about productivity. People can find productive things to do on their own.

"Bullshit jobs" depend on economic surplus to create work that is not immediately productive in terms of food, shelter, or safety. Most entertainers and craft artists fall under the definition of "bullshit jobs". Fine dining chefs are doing something entirely unnecessary. So are rock stars.

<comment><USER22> It's a couple of years since I read that article but I'm pretty certain top chefs -- who work directly on products that people clearly want a lot -- are the exact opposite of the kind of alienated work Graeber had in mind.

<comment><USER2> Sure they are. But those jobs exist because of economic surplus.

<comment><USER23> Well, all Apple jobs exist for the same reason.

</comment></comment></comment></comment></comment><comment><USER24> I have often wondered what happens when large scale technological unemployment occurs, or even the threat of it. I think we have seen the result in the 2016 Presidential election with the election of Donald Trump, probably the least qualified president since Harding. He's a psychopath, for god's sake. And his cabinet is beyond comprehension.

The possible result is an attack on all branches of the executive except the military. Replacement of Social Security and Medicare by virtually worthless vouchers, elimination of the departments of Education, Housing, and Energy. emasculation of the EPA. And on and on.

This could be as self-destructive as an armed uprising with fewer deaths (maybe).

Many of the issues are easy to fix. Remove the wage cap on Social Security taxes. Convert to health insurance to Medicare. Some of the unemployment issues can be mitigated by making the workweek 30 or 32 hours. Turning truck drivers into nurses or medical technicians takes a few years of training. Many of them are smart enough to do it. The current stumbling block is that they must have income support during the education period. Giving someone $100K to manage the transition doesnt seem like much to me.

Fundamentally increased productivity and wealth should lead to more leisure and a better standard of living for everyone. Eventually everyone should get a minimum guaranteed income so they have time to pursue their education, raise families, be artists or novelists.

See [https://en.wikipedia.org/wiki/Mincome](https://en.wikipedia.org/wiki/Mincome) for an experiment on a guaranteed income.

</comment><comment><USER25> Global economy & services may be fully automated pretty soon but local economies? No chance, we are still going to have humans doing local trades and services to other humans for the foreseeable future, just go back to your roots as a member of a family, village, community.

<comment><USER26> > just go back to your roots as a member of a family, village, community.

I think unemployed people will start teaming up to solve their needs directly - farms, small fab shops, schools, construction - we can make all these with local workforce, "self employed for self reliance". We'd only need access to raw materials and some land.

</comment></comment><comment><USER27> AI is a fundamentally different technological innovation than the ones he compares against in the post. It's not an incremental improvement on methods, it's a complete replacement of the labor force for many industries.

But the problem to solve isn't how to blunt the impact of AI, rather how to handle the newly freed labor force. I don't think this should be seen as a bad thing. Tasks a machine can take care of should be left to a machine, and other work should be handled by a human. We'll have a lot of open human capacity to accomplish something, but if the transition isn't handled well it could be really painful.

</comment><comment><USER28> I used to think techno-optimists were delusional but history is on their side. You could argue there is a phase shift happening and this one will be different but I doubt it. Like the author says we'll make up new jobs.

</comment><comment><USER29> The main thesis of this article is true but honestly the author has done a very unconvincing job to back it up. For example my problem is with his references: "Firm investments in high-tech equipment and software is falling"...whereas the liked article is about "The recent slowdown in high- tech equipment price declines and some implications for business investment and labor productivity"

</comment><comment><USER16> The real deal with technological unemployment is that the list of things humans can do and machines can't keeps shrinking.

<comment><USER3> That has been true for at least 400 years. Humans have generally adapted by inventing new things to be done that didn't exist previously. (The app industry was very small 20 years ago. ;)

The interesting question is why people think this time around is different. Answers seem to vary from just "computers" to, "the same change is happening but now it's happening faster". The compensatory mechanisms still work though, yet it seems no one things they will.

<comment><USER30> People think it's different this time because the new machines are more generally capable than the specific purpose machines of the past.

It's not "the new machine can gin cotton" or "the new machine can weave cloth", it's "the new machine can handle an extremely wide range of problems, and we're moving from having to teach it each problem individually to having it teach itself."

This time _really is_ different and it's crucial everyone realize why as soon as possible.

<comment><USER3> > It's not "the new machine can gin cotton" or "the new machine can weave > cloth", it's "the new machine can handle an extremely wide range of > problems, and we're moving from having to teach it each problem individually > to having it teach itself."

It's funny you mention this because in the past few days on HN there have been a number of articles whose general point was that the machine learning stuff presently in use is well tailored to very narrow, specific problems, and that the AI tech that would tackle a broad range of problems is still very elusive.

This new machine that tackles a very wide range of problems without massive programming investment is something I've been wanting for 20 years, and I still don't see anyone with plans to bring it to market.

<comment><USER30> > the machine learning stuff presently in use is well tailored to very narrow, > specific problems

It's more accurate to say the machine learning in use now works on narrow _classes_ of problems. But even that is a giant leap forward from teaching the machine to solve each problem individually.

Teaching a car to drive very specific routes is one thing (and has been effectively done for decades now). Teaching a car to drive entire classes of routes is something else, and where we are now. In a decade or so cars will drive all classes of routes better than humans, and that will be something else again.

</comment></comment></comment></comment></comment><comment><USER31> > _[...] the unemployment rate worldwide is low. It is under 5% in the US > [...]_

I'm not sure how this is calculated. Do people employed part-time count as employed? How has that changed over time? The problems of the _precariat_ include under-employment and being forced to hold undesirable jobs.

<comment><USER32> In the US, the official unemployment rate cited is the U-3 rate, which is exclusively limited to those currently seeking a job, with no employment status at all.

Partial employment (such as part-time) due to economic circumstances, as well as those who are within the employable age range (18-60something) but are not employed nor seeking a new job due to economic circumstances (i.e. gave up) are covered by the U-6 measurement, which is substantially higher at 9.8%. See here: [https://www.bls.gov/lau/stalt.htm](https://www.bls.gov/lau/stalt.htm)

That page also has links to historical measurements back to 2003, and you can probably find further back (though I'm not sure if there are older resources that also include the state-by-state breakdown).

</comment></comment><comment><USER33> I'm getting an internal server error when I click on the link.

<comment><USER34> For now you can find it on the front page of the blog: [http://lemire.me/blog/](http://lemire.me/blog/)</comment></comment>


Ask HN: What's the best way to find a product co-founder? - <AUTHOR> <comment><AUTHOR> I'm looking for ways, places, online platforms, any recommendations...

<comment><USER2> what's wrong with the one your working with on Finimize.</comment></comment>


WakeMate Delays Initial Shipment, Makes Moves To Appease Upset Customers - <AUTHOR> http://www.techcrunch.com/2010/01/23/wakemate-delay/

<comment><USER2> I could not be happier with the outcome of this situation. Big props to the WakeMate team for straightening things out.

<comment><USER3> Agree. I wasn't even upset, and now I get "all future premium software features" for free. I will be downright _happy_ to part with the remaining 90% of the purchase price.

</comment></comment><comment><USER4> I am really excited about this because I have trouble sleeping. It would be helpful for me to actually use this.

<comment><USER5> There's something called Smart Alarm Clock for Android (if you have a supported phone). I tried it as a way to judge if the idea had any merit, and it worked really well. Obviously it doesn't help with sleeping, just waking you up. But for the last week or so, I've woken up fresh every time, even when sleeping around 5 hours or so. Obviously, this works best if you get into a good sleep/wake cycle. There's something similar for the iPhone too, might be a good idea to do a trial run with a cheap app.

<comment><USER6> Smart Alarm used to work, but the latest edition simply wakes you up at the desired time every time and it runs on your phone all day.

I was happy till the latest update and now its unusable.

</comment></comment></comment><comment><USER7> Kudos guys. Appreciate you taking all the feedback and criticism to heart and righting the ship.

</comment><comment><USER8> Sweet, that worked out well. Thanks, WakeMate!

</comment><comment><USER9> I don't think many customers are "upset", per se. This is a fun-to-try, completely non-essential product at this point, and hardware is hard to make, especially for a startup. Delays are forgivable. So while I'll certainly give it a shot when it finally ships, I wonder if wearing a WakeMate may just train me to keep very still towards the morning hours of my sleep?

<comment><USER10> They actually were upset. I wrote a comment describing the points of issue, but there is already a concise description here: [http://danieltenner.com/posts/0014-learning-from- wakemate.ht...](http://danieltenner.com/posts/0014-learning-from- wakemate.html)

They were willing to give a refund for the $5 down-payment, but there was still an overall bitterness. This was a wise move for wakemate, even if there is nothing wrong with providing more than what was promised/planned for an additional fee.

The customers they have now are very early adopters who will be spreading the word and writing reviews. It's essential that they do whatever they can to make positive first impressions.

</comment></comment><comment><USER11> I'm glad to see WakeMate doing right by those who have pre-ordered. Are those who say pre-ordered today going to see the same sort of deal?

</comment><comment><USER12> This is great. I was one of the disgruntled customers but this mail has turned me into a happy one again.</comment>


The Extraordinary Nazca Prehistoric Balloon - <AUTHOR> http://nott.com/nazca/ <comment><USER2> Title (disappointingly) suggests they have evidence that the Nazca _did_ use hot air balloons. No such evidence is presented. This describes the creation of a proof-of-concept hot air balloon using period technology. That's neat, but it's pretty different from finding historical evidence.

</comment><comment><USER3> This concept was also featured in the adventure game "The Journeyman Project 3: Legacy of Time", where the player travels back in time to an Incan settlement that was meant to be the inspiration for El Dorado, and discovers that simple hot-air balloons were used for transport.

[https://en.wikipedia.org/wiki/The_Journeyman_Project_3:_Lega...](https://en.wikipedia.org/wiki/The_Journeyman_Project_3:_Legacy_of_Time)

[http://www.mobygames.com/images/shots/l/167030-the- journeyma...](http://www.mobygames.com/images/shots/l/167030-the-journeyman- project-3-legacy-of-time-windows-screenshot.jpg)

[http://www.mobygames.com/images/shots/l/167031-the- journeyma...](http://www.mobygames.com/images/shots/l/167031-the-journeyman- project-3-legacy-of-time-windows-screenshot.jpg)

</comment><comment><USER4> They _could_ have also built man-lifting kites:

[https://en.wikipedia.org/wiki/Samuel_Franklin_Cody#Kites](https://en.wikipedia.org/wiki/Samuel_Franklin_Cody#Kites)

</comment><comment><USER5> I am pretty familiar with the Nazca geoglyphs. (flown over several times) I favour the theory of tethered kites as the are steady 25mph winds over the area, pretty much every afternoon.

While flying a local "expert" (Eddy Herran) on the area, he showed me a geoglyph resembling a basic hang glider (rogallo wing).

Flying over the Nazca lines, stands as one of the most mind-bending experiences of my life. The whole area is full of mysticism and unexplained situations.

</comment><comment><USER6> > It is incorrect to say that the lines can not be seen from the ground. They > are visible from atop the surrounding foothills

cool technology but the conclusion where made from a false assumption. it still may or may not be possible that lines where drawn in such fashion, but the theory itself as exposed sits on very shaky ground.

<comment><USER7> IMO much more likely that the lines were drawn as some kind of worship to gods believed to live in the sky, rather than for viewing by humans floating overhead in balloons.

<comment><USER5> I think the idea of flying to see them is not the final objective.. just a mean to see the results of your works.

Maria Reich (Principal investigator) found out a lot of correlation between some drawings and star constellations. She also found several straight lines aligned with key calendar days like harvest , summer & winter solstice and so on.

</comment></comment></comment><comment><USER8> Inspect -> delete <head>, delete #main-nav.

There, now I can read it.

<comment><USER9> Applies to most web sites these days, sadly.

</comment></comment><comment><USER10> > to dare to invade the territory of the Angles.

They travelled further than I thought.

Hell of a balloon ride all the way to northern Europe.

</comment><comment><USER11> It's worth noting that the ancient Chinese used hot air balloons, as toys and signalling devices, etc.</comment>


Create objects 2x faster in Python - <AUTHOR> http://karamatli.com/pages/lang/Pools_in_Python/

<comment><USER2> This is a pretty bad benchmark. Of course the harbor with pooling is faster. It's creating all 100000 objects outside the timer, while the harbor without pooling is creating them within the timer. If I change the code so that it creates the objects inside the timer, I get the following results which are nowhere near as good:

        Total With Object Pooling: 633.393 ms
        Total Without Object Pooling: 714.89 ms

This is hardly 2x faster and the speed increase that is there probably comes more from allocating the objects in a list comprehension in the pooled harbor.

<comment><AUTHOR> Of course it is creating them outside the timer, the point is _pre-allocating_ objects!

<comment><USER3> Why not just run your entire program outside the timer? Then it is infinitely fast!</comment></comment></comment>


Ask HN: Web app pricing... to charge or not to charge - <AUTHOR>

I'm avoiding the temptation of delaying my web app's release due to lack of features and polish.  As I've read on HN over the past 6 months: Release early! then iterate... (I've drank the kool-aid) I will be releasing my application sometime in June. (yikes)
My next concern regards whether or not to charge users to use my application.  I've read articles and have had debates over the past few months.  Now I'd like to pose this question to the HN population:
Assuming the founders' costs are are trivial (excluding time), what are the scenarios/reasons to charge users to use a product vs. providing free access to all users? <comment><USER2> Recently I needed a good diagram program for my assignment to use on my non- windows box. I found this web app: <http://www.gliffy.com/> They had free and paid accounts. If it didn't have free accounts, I wouldn't have been impressed by how well it worked, and I wouldn't have told all my friends and my parents about it. Hmm but I'm not a founder so this is just from a particular free product consumer's point of view.

<comment><USER3> I agree; I think the freemium model works well. I have a client-side application that I have in a freemium model. The Free version provides just the basic core functionality. The Pro version adds all kinds of other extras that aren't necessary but add value. I think it's a great way to get market penetration with free word of mouth advertising.

</comment></comment><comment><USER4> It depends on how much 'tilt' there is to the category of the industry you're playing in. If there are strong viral and network effects, don't charge (for now and a long time).</comment>


The Internet Locusts Descend on Ristretto Roasters - <AUTHOR> https://quillette.com/2019/02/18/the-internet-locusts-descend-on-ristretto-roasters/ <comment><AUTHOR> It sounds melodramatic, but I'm starting to wonder whether our society can survive this sort of thing.</comment>


Build Node-JS expert systems with node-CLIPS - <AUTHOR> https://github.com/atrniv/node-clips CLIPS, an acronym for "C Language Integrated Production System" is a fast, efficient and absolutely free tool for building expert systems. Now, we bring that goodness to Node-js. <comment><AUTHOR> A Nodejs native extension which allows developers to use this popular CLIPS system to build expert systems.

CLIPS is a productive development and delivery expert system tool which provides a complete environment for the construction of rule and/or object based expert systems. Developed at NASA in 1985, CLIPS is now widely used throughout the government, industry, and academia.

Check out the online CLIPS documentation: <http://clipsrules.sourceforge.net/OnlineDocs.html></comment>


My 12-Week 30-Pound Challenge - <AUTHOR> http://ste.vn/12-week-30-pound-challenge/

<comment><USER2> First off -- Congrats on making the huge step to "do something about it"! Since it looks like you posted your own link I'll add comments here so people of HN can see them too. Personally, I've tried many diets, workouts, and other combinations at the same time and find it really hard to keep focus.

\- Building a workout takes time

\- Building a cardio plan takes times

\- Grocery shopping for a new diet takes time

\- Finding that you're starving and can only eat XX food but don't have it at home is frustrating!

\- Looking at labels to see how much sugar is in XX takes time when you're starving!!

\- Making fitness / diet changes at the same time is painful!!

I would focus on fitness first and once you the routine down fiddle with your diet. I just focus on not eating fast food and cooking at home. Here is a awesome workout by the RCMP that includes cardio / weights in a 12 week program. This program was designed to get people who are out of shape and into shape before their program starts.

I am 5 weeks in and have a spreadsheet with my status if you want to copy it. Website with benchmarks and other info:

[http://www.rcmp-grc.gc.ca/recruiting-recrutement/htm- form/pa...](http://www.rcmp-grc.gc.ca/recruiting-recrutement/htm-form/pare- tape-expectation-attente-eng.htm)

<http://www.scribd.com/doc/38653230/Pare-Tape-12W-Eng> is a PDF with the 12 week program broken down into warp-up, cardio, weights, cool-down and stretching.

p9 - Warm-up p15 - 12-Week Cardiovascular Training Program p19 - 12-Week Resistance Training Program p24 - Cool-down and Stretching

Good Luck!!</comment>


Will android take over iphone? - <AUTHOR>

Will Android eventually take over iphone and Blackberry? If you do think so how long do you think it would take for Android to make that happen?<p>I think Google would be able to make Android as good as the iphone OS and Android been open source and giving more control for developers would be able to close in on the huge number of iphone apps already available. The problem will always be from the side of Device manufacturers. some one from the Open Handset Alliance will have to deliver a product that can match the iphone. <comment><USER2> I wonder if the story Mac vs. PC will be all over again but in different market. We have one company with new, sexy and revolutionary device, we have another company—the market leader—that missed the moment and we have the third company that started by writing software for the sexy device and eventually released their own operating system: definitely not as polished but open so any number of hardware-oriented companies can release their devices with this new OS.

<comment><USER3> I'm sorry, how is that the story of Mac vs. PC?

<comment><USER2> IBM as a market leader, that missed the moment, Apple as, well, Apple—company that released cool Macintosh computer—and Microsoft, as a company who started by writing software for Macs (i.e. BASIC version for Mac) but eventually released their own OS.

And Microsoft was the winner at the time.

</comment></comment></comment><comment><USER4> It won't happen in the near future because there is very little interest in Android even among the early adopters and hackers. Android sales have been dismal.

What makes this round of platform struggles much more interesting is that Apple has learned from its mistakes in the 1980s. Apple listened to its developers and released the iPhone SDK, and lowered the iPhone 3G price to appeal to a wider audience. Sure, there are still many rough patches such as the App Store, and the much touted security vulnerabilities. But the latter are only going further to perpetuate it in the consciousness of the general public. Bad PR is still PR.

On the whole they've been doing a very good job and should remain the benchmark for at least the next 18 months.

</comment><comment><USER5> No, because Android have a fundemental business model problem. OS supplies by one company, product designed by the manufacturer goes only so far. One size does not fit all.

Apple has demonstrated the value in having top down owneship of a platform quite clearly. You have one product to worry about at any given time and you dedicate all your resources to make it perfect. Google at the same have a distracted focus, they are trying to make Motorla, HTC and what not...

</comment><comment><USER6> No.

<comment><USER7> I concur, it seems as simple as that. Android doesn't have the momentum, the sexiness, the tight hold on quality that the iPhone has. All of these contribute to its success, and, well iPhone got there first, that counts for a lot.

</comment></comment><comment><USER8> I think everyone is missing the point here that Android as devices so far haven't been great. I have a G1 and compared to the iPhone it certainly isn't sexy.

But as a platform, in the next 18 months you'll see an explosion of devices for different handset manufacturers that are all compatible with each other to to some extent - so as a developer you can write once and deploy to a range of devices, most of them phones, but some of them devices like the Zii Egg.

Don't write off Android yet. Weren't people saying the same thing about Apple years ago?

<comment><USER9> This seems to be the main sticking point for people I know - Android Phones are nice, but thay aren't iPhones. It's the hardware that is holding up Android rather than the OS, as far as I can see.

If there was a phone as well made as the iPhone that runs Android, I'm sure many developers would switch to an open platform rather than continue to jump through obfuscated hoops for Apple.

</comment></comment><comment> onreact-com Until two weeks ago I'd say no but at the current pace Apple is scaring off customers Android may catch up sooner that we'd expect. People tired of Apple will embrace the Android platform so that their critical mass will propel the developer community and thus better interfaces and the large number of apps will convince even more people to use Android phones.

<comment><USER10> I completely agree. I look forward to my AT&T contract ending with the hope that the Android phones will have caught up with the iPhone's sexiness. If so, I'll definitely consider an Android handset.</comment></comment>


What's the most optimal numeric base? - <AUTHOR> http://www.burtonmackenzie.com/2007/12/whats-most-optimal-numeric-base.html

<comment> daniel-cussen Cool article. I'm still not sure if his definition of what the best numeric base would be is valid. My hunch is that ternary will probably work great in reality, but a larger base should be used for humans. Maybe senary (base 6), because that's about as many objects as a person can eyeball.

Thankfully, it's things like these that promise economic growth for centuries to come. After all, if your growth rate is 0, you can switch everyone over to a different number system, and if it really is better, you'll get a return on investment for society no matter what the switching cost is (mostly because the divisor in the ROI equation would be 0).</comment>


Microsoft Word Tips and Tricks to Get Things Done Faster - <AUTHOR> http://www.labnol.org/software/word-tips-tricks/28267/ <comment><USER2> Some useful tips, yes, and some not-so-useful. The Unicode trick isn't exactly a Microsoft Word trick--it's more a Windows thing. The Hidden Calculator left me flat; I'd rather use the Windows Calculator, but YMMV. My only issue with these tips is that the problems they solve tend to arise so infrequently that I don't repeat the solutions often enough to remember them.

</comment><comment><USER3> I fail to understand how this belongs on _Hacker_ News.

</comment><comment><USER4> I hear Microsoft Word has a feature that instantly uploads what you type to the cloud. It's called Numbers Symbols Alphabet (NSA for short).

I'm not even joking.</comment>


Search results beat FDA in finding drug combo side effects - <AUTHOR> http://news.cnet.com/8301-11386_3-57572948-76/search-results-beat-fda-in-finding-drug-combo-side-effects/

<comment><USER2> The New York Times article "Unreported Side Effects of Drugs Are Found Using Internet Search Data, Study Finds" by John Markoff

[http://www.nytimes.com/2013/03/07/science/unreported-side- ef...](http://www.nytimes.com/2013/03/07/science/unreported-side-effects-of- drugs-found-using-internet-data-study-finds.html)

is better reporting on this issue, and the title isn't linkbait.</comment>


High Quality Video Encoding at Scale - <AUTHOR> http://techblog.netflix.com/2015/12/high-quality-video-encoding-at-scale.html <comment><USER2> It's a fascinating look into how they produce multiple encoded formats from source video, but something stuck out at me... One feature in their ingestion system is meant to reject source inputs that would lead to a poor viewer experience. This doesn't seem to include interlaced NTSC video as a "poor experience" metric. I'm willing to bet that the majority of the screens on which Star Trek: Deep Space Nine is played are progressive scan, yet that show is interlaced at NTSC resolution.

In cases like this, where the source material is analog and, I would have to assume, not available in progressive scan, is there a technical reason why Netflix doesn't de-interlace the source before encoding? DS9 seems big enough of a catalog to be worth encoding in a less jarring scan rate.

Not complaining, mind you. The DVDs are interlaced too. Seems that the original recordings were NTSC only. Honestly curious if anyone has any insight.

<comment><USER3> TNG was like this (sourced from a video version and with a very nasty interlace look) on Netflix up to a couple of months ago. Now they stream the magnificent remastered blu-ray HD version which is sourced from the original film with fresh CGI since the original CGI was tied to the video.

It's not clear whether DS9 will get the same treatment.

<comment><USER4> I prefer to watch SD rips from the TNG blu-rays. The merciless clarity of 1080p just kills the magic.

<comment><USER5> I've wondered if this could be fairly easily emulated with some playback filters locally if a viewer has a preference of experiencing material with lesser fidelity (CD v. vinyl comes to mind where it's almost more about ceremony and romanticism than technical merits 95%+ of the time to reject the hi-fi version, "remastering" insanity notwithstanding).

<comment><USER6> CD vs Vinyl is not a fidelity issue but actually a compression that is a part of analog which is what people call "warm" sounding. A good analog recording has just as much or more fidelity then digital. An analog recording can actually reproduce a accurate sound wave while a digital can technically never be able to do that. (Former sound engineer)

<comment><USER7> Actually digital recordings can reproduce signals up to Nyquist perfectly. Watch this surprising demo: [http://xiph.org/video/vid2.shtml](http://xiph.org/video/vid2.shtml)

<comment><USER5> In some defense of vinyl purists, there's a lot of signal intensity above audible human hearing range in sibilants and especially percussions. [http://www.cco.caltech.edu/~boyk/spectra/spectra.htm](http://www.cco.caltech.edu/~boyk/spectra/spectra.htm)

However, similar to how we perceive ultra low frequency more as a physical vibration than as sound, these harmonics above 20 kHz are probably merely annoying and subtly felt and impact one major issue of enjoyability - listening fatigue. So maybe we don't want those frequencies from a musical perspective anyway.

Regardless, there's hardly any equipment in use by even "audiophiles" on full- blown analog setups that can faithfully reproduce sounds beyond 30 kHz because of electronics design limitations in themselves rather than an analog-digital end user recording format distinction. In fact, a lot of vinyl historically had to be mastered with a low-pass filter cutting off a lot of the high frequencies because historically with sufficiently high enough energy in high frequencies, the needle would be tougher to control and fly right off the track sometimes (one explanation I read from an audio engineer - really not sure about that logic, but there's definitely low AND high pass filtering on vinyl that makes it lower fidelity in many respects than the master).

Heck, most vinyl produced since the 70s comes from digital masters in the first place. [http://wiki.hydrogenaud.io/index.php?title=Myths_(Vinyl)#Myt...](http://wiki.hydrogenaud.io/index.php?title=Myths_\(Vinyl\)#Myth:_Vinyl_is_better_than_CD_because_it_reproduces_higher_frequencies_than_CD_and_avoids_anti- aliasing_filter_issues_at_the_frequencies_CDs_can_reproduce)

Bugs the crap out of me to see people claim vinyl is superior on technical merits rather than aesthetic ones (sound preference / taste is real). You'd think they're climate change deniers with their insistence and rhetoric. But this is what I meant by purists about the "original" \- higher fidelity and clarity is oftentimes not what people desire.

<comment><USER7> Yeah, I love vinyl, not because it's good, but because it's so bad. I'll still listen to digital recordings a majority of the time.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I'm envious of these capabilities. I wrote the backend for a streaming service that streams about forty channels to a quarter of a million mobile users in Africa. We get source video from our content providers that has given me grey hair. The problem is that I simply can't spin off the encoding and source quality checks to the cloud because of bandwidth costs here. So I do the quality checking and compression on local servers and then upload the compressed output to the servers.

I'd kill for a 100 megabit line at a decent price.

<comment><USER9> That sounds fascinating, what is the video used for?

<comment><USER8> It's literally a bunch of TV channels: sport, soaps, educational, news and music videos.

</comment></comment></comment><comment><USER10> As someone who encodes video for work with a ton of ffmpeg-based batch files, this article makes me feel like a child.

<comment><USER11> I've done a bunch of playing with ffmpeg at home, and I imagine the tech stack is probably similar at Netflix, at least for the Source-->Chunk, Chunk-->Assemble, and Assembled-->Encode steps.

The validation done during all of these is interesting. Netflix's early years are probably exactly like what you're doing - single file in, transcode, single file out and deploy.

Chunking the pieces up is clever. Getting it right must have been challenging. How do you write an oracle for something that complex?

<comment><USER12> _How do you write an oracle for something that complex?_

You start with a much smaller problem and incrementally build up.

</comment></comment></comment><comment><USER13> Fascinating article. Forgive me for being ignorant about this, but how often do they need to encode video?

I would think that it's only done when new titles are added to streaming, and once the video has been encoded into all the required formats they would be done with it. Sure, there is a lot of video content out there to be encoded but it isn't unlimited. Is serving the content and providing the recommendation engine to users at scale not a greater challenge than encoding the video?

<comment><USER14> Yes, encoding needs to be done only once.

Generally it would take 2-3x of original duration of video to encode a source into a 1080p, so I am not sure why they take full 1 day? unless they do each bitrate serially which I think is not as hard to parallelize as it is to parallalize single bit rate by chunking.

Yes, I believe serving is lot harder, but serving is almost a solved problem since people are dealing with for long time.

<comment><USER15> For x264 that is true, HEVC which is also mentioned is much slower. For a 4k source transcoding can take more than a second per frame. For a normal movie this can quickly result in encoding times of more than a day.

Another problem is that you have to encode the movie for each codec profile times the number of different bitrates per profile. The article mentions four profiles (VC1, H.264/AVC Baseline, H.264/AVC Main and HEVC) and bitrates ranging from 100 kbps to 16 Mbps. Assuming now there are 20 different bitrates per code you already get 4*20 => 80 encoded copies per source. But of course this can be solved by parallelism.

<comment><USER16> Are there any codecs that can output multiple versions of an input at the same time? Seems like a lot of the encoding process (like motion estimation) is the same every time, so why do it once for every output instead of reusing it?

<comment><USER17> That would be interesting to know. A lot of transcoders can make multiple passes over the source, so being able to reuse the meta data generated for subsequent passes at different output qualities might help speed up the process. I dunno, not my forte, just thinking out loud.

</comment></comment></comment></comment></comment><comment><USER14> Interestingly, not even a single mention of ffmpeg.

<comment><USER18> It's possible their AVC encoder is not x264, and their VC1 encoder might be from Microsoft, in which case it would be a pretty modified ffmpeg. And some of the input formats might not be using it either.

But you can be sure it's involved somewhere, since there's no other ProRes decoder that works on Linux!

<comment><USER19> FFmpeg supports Prores decode and encode? FFMBC for sure last time I checked

<comment><USER20> FFmpeg does, it has different decoder implementations with their own quirks (btw, moot AFAIK because I think they use EyeIO):

      # ffmpeg -codecs | grep prores
      ffmpeg version 2.8.1 Copyright (c) 2000-2015 the FFmpeg developers
        built with Apple LLVM version 7.0.0 (clang-700.1.76)
        configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-indev=jack --disable-outdev=xv --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-vda --enable-videotoolbox --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libsmbclient --enable-nonfree --enable-libfdk-aac --enable-libfaac
        libavutil      54. 31.100 / 54. 31.100
        libavcodec     56. 60.100 / 56. 60.100
        libavformat    56. 40.101 / 56. 40.101
        libavdevice    56.  4.100 / 56.  4.100
        libavfilter     5. 40.101 /  5. 40.101
        libavresample   2.  1.  0 /  2.  1.  0
        libswscale      3.  1.101 /  3.  1.101
        libswresample   1.  2.101 /  1.  2.101
        libpostproc    53.  3.100 / 53.  3.100
       DEVIL. prores               Apple ProRes (iCodec Pro) (decoders: prores prores_lgpl ) (encoders: prores prores_aw prores_ks )

</comment></comment></comment></comment><comment><USER17> Interesting post. Netflix's posts usually are.

And then you scroll to the comments on the page...

</comment><comment><USER21> I personally worked on a similar system for a major mobile video solutions provider six years ago: none of this is particularly new.

</comment><comment><USER22> Has Netflix started with HEVC h.265 support? I see they mention HEVC briefly.

<comment><USER23> I think their 4K streaming to TVs uses HEVC.</comment></comment>