Distances you can travel on a European train in less than a day - <AUTHOR> http://www.washingtonpost.com/blogs/worldviews/wp/2015/06/05/map-the-remarkable-distances-you-can-travel-on-a-european-train-in-less-than-a-day/ <comment><USER2> You commonly hear that it's ridiculous for California to try to build a high- speed rail network, because people are "so far apart". But France -- a nation called out in this article for being well-connected by high-speed trains, is about the same vertical height as California. It's a bit wider.

If you place the northern tip of France on the northern border of California, the southern tip of France lands just north of San Diego. We have about 40 million people; France has about 60 million.

Japan (another country with great trains), is about the same vertical distance north/south as the west coast of the US. And while Japan's population is crammed into the coastline of an island nation where it's expensive to build, we have gobs of open space where it's comparatively cheap to run rail.

There's simply no excuse. Sure, it doesn't make sense to try to make a high- speed train across the country, but we should have _great_ regional rail systems in the US. The east/west coasts, in particular, should be corridors for rail -- if you can get from Kagoshima to Tokyo in 6 hours by Shinkansen, there's no reason you couldn't get from New York to Miami in the same amount of time.

<comment><USER3> The question is why? Why should the U.S. spend billions on a great passenger rail system? High speed rail is almost always a serious money loser. What are the benefits?

Japan and Europe's high speed rail is nice, but it hasn't necessarily resulted in a greater share of passengers taking rail. "Since Japan introduced high- speed bullet trains, passenger rail has lost more than half its market share to the automobile. Since Italy, France, and other European countries opened their high-speed rail lines, rail's market share in Europe has dwindled from 8.2 to 5.8 percent of travel. If high-speed rail doesn't work in Japan and Europe, how can it work in the United States?" [http://www.cato.org/publications/policy- analysis/highspeed-r...](http://www.cato.org/publications/policy- analysis/highspeed-rail-wrong-road-america)

And some environmentalist don't like it either. [http://www.motherjones.com/environment/2015/01/california- hi...](http://www.motherjones.com/environment/2015/01/california-high-speed- rail-waste-time-and-money)

Most people don't know that the U.S. has a great freight rail network. U.S. passenger rail obviously isn't great, but U.S. freight rail is the best in the world. [http://business.time.com/2012/07/09/us-freight- railroads/](http://business.time.com/2012/07/09/us-freight-railroads/)

I just don't know why the U.S. should build high speed rail, other than the fact that it is fun to ride.

<comment><USER4> Some benefits of a functioning railway system:

\- increased throughput compared to highways due to lower volume per passenger (something the US is trying to achieve with carpooling lines),

\- increased reliability (there are no train-jams),

\- increased passenger convenience (no need to look for parking or do maintenance on the train you ride),

\- increased passenger productivity (you can work or read on a train),

\- ease of transition to green source of energy.

Advantages of the car over the train:

\- greater flexibility (generally roads offer denser network than railways, even in Europe or Japan),

\- car is much more convenient for families with children.

I love trains. I love driving. I find that the latter is much more fun and convenience in places where the former works well. San Francisco's 101 could be one of such places if Caltrain was more like SBB.

EDIT: one more important benefit of the train:

\- increased safety (much fewer fatalities per passenger; also no need for a designated driver when you go to a party).

<comment><USER5> Some clear issues with long distance railway system: \- it costs TONS of money to built. \- you need to buy tons of land. \- to keep straight lines in a place like Europe, it often means you have to go through places where people live, and you need to have villages partly destroyed by it. Yay, democracy. You'll notice that high speed train systems are usually very much correlated with socialism (France and Japan are both well known socialistic countries - no surprise here at all). \- Above certain distances it makes no sense to take the train over the plane. I take the train in Japan every week for work between Kansai and Kantou, and it takes about 3 hours. A plane takes one hour, and it is 40% cheaper. Remind me why we need trains there? \- The maintenance of a railway is a huge ongoing cost. \- if there is one incident on the main railways, your subsequent trains are delayed or blocked. Yay for flexibility and "reliability". \- Try working in economy class in TGV in France. It's about as narrow as plane economy, and it sucks for working. In Japan the situation is much better but don't make a generalization there. \- green energy ? Oh, like in Japan where most of the electricity is produced by gas, petrol, imported at high costs from overseas since they have no such natural resources? Yay for efficiency! (truth is, Fukushima did not help, and electricity bills have been doubling since then (even home bills)). \- Anything that does not depend on railways can be upgraded, transformed, improved. We will have self driving cars at some point, and we will be stuck with high cost trains running on linear tracks forever, because there's no improvement to be made there. It's a 18th century concept pushed to the max.

There are cases for high speed trains, but even in Japan what they are doing is stretching it - nobody takes a train from north Japan to south Japan even if it's possible. It is just too expensive and takes too much time to be actually worth it.

<comment><USER6> I agree that planes are faster over longer distances, and cheaper too. The amount of hassle involved in planes is definitely more though, more waiting, you're supposed to be there earlier, more transit time to and from the airport. I think a 3h train and 1h flight are pretty comparable in the end.

<comment><USER7> Couldn't agree more. Short haul airlines are only profitable because they treat passengers' wasted time as free (and for some reason passengers agree).

</comment></comment></comment></comment></comment></comment><comment><USER8> Some of the travel times quoted don't even do justice to reality !

 _"I was really surprised how well connected Paris is. Both London and the Mediterranean coast are less than four hours away,"_

The Mediterranean coast (Marseille) is actually just 3 hours away, and London to Paris a mere 2h15 on the Eurostar train, with stations conveniently located in the heart of each city.

Almost nobody is masochistic enough to fly the London-Paris route, apart from people on connections or some American tourists who didn't do their research and fly by default.

Also "Less than 8 hours" from Stockholm to Copenhagen is actually 5 hours IIRC, on comfy, clean trains with free Wi-Fi and AC power.

</comment><comment><USER9> A great benefit is you end up in the heart of a city, not at airfield 80 miles away. Lock your bag up in the station and explore.

If you like your long railway journeys, Olso to Bergen is beautiful 7 hour trip.

<comment><USER10> In Europe, the convenience/inconvenience of airport locations varies a ton (which does influence purchase choices somewhat). For example Frankfurt's airport is only 8 minutes by commuter rail from the city center, and Copenhagen's is 12 minutes. But Paris-CDG and London-LHR are far from their city centers.

<comment><USER11> You still have to trek through the airport, wait in security lines etc etc. so in reality it takes much longer than the time it takes the train to travel from the airport to the city. That said flying is often if not nearly always cheaper for short hauls in Europe.

</comment></comment></comment><comment><USER12> For anyone wondering why Eastern Europe is so badly connected, it's mainly because the Russian Empire used a different gauge to the rest of Europe:

[https://en.m.wikipedia.org/wiki/Russian_gauge](https://en.m.wikipedia.org/wiki/Russian_gauge)

In the Baltics there is an ongoing project to upgrade and switch to standard gauge, which will provide a direct link from Berlin to Helsinki (to start with via ferry, but a tunnel is planned). However a lot of trade done there is with Russia via rail, so it's beneficial for them to keep that as seamless as possible - switching passengers is one thing, cargo is another. Lithuania apparently transports 57% of goods by rail.

[https://en.m.wikipedia.org/wiki/Rail_Baltica](https://en.m.wikipedia.org/wiki/Rail_Baltica)

<comment><USER13> The U.S. once switched rail gauge. Once preparations were complete, it took only three days for the entire southern U.S. to change from 5' to 4'9".

[http://southern.railfan.net/ties/1966/66-8/gauge.html](http://southern.railfan.net/ties/1966/66-8/gauge.html)

</comment></comment><comment><USER14> For any non-Europeans looking to explore Europe by rail, I can highly recommend getting an Interrail ticket that gives you unlimited rail travel for various durations. For a reasonable price you can get a multi zone ticket that will let you explore most of Europe for a very reasonable price.

<comment><USER15> Non-European citizens can't use Interrail. There's a similar product called Eurail but with the discounts available for advance-purchase tickets, the pass often doesn't make sense if you're traveling with a relatively specific itinerary and can plan ahead.

The surcharges on the various high-speed trains really killed the cost- effectiveness of the Eurail pass.

edit: There's a good overview of why the Eurail pass if often a bad option at Seat61: [http://seat61.com/Railpass-and-Eurail-pass- guide.htm#Should](http://seat61.com/Railpass-and-Eurail-pass-guide.htm#Should) you buy a railpass or pay-as-you-go

<comment><USER4> > Non-European citizens can't use Interrail.

Not true. You have to be a _resident_ to use Interrail. So if you don't have a European passport, but live there, you can still use Interrail.

Note also, "European" here means a lot more countries than the EU. Other eligible countries of residence include: Albania, Andorra, Belarus, Bosnia- Herzegovina, FYR Macedonia, Gibraltar, Iceland, Kosovo, Liechtenstein, Moldova, Monaco, Montenegro, Norway, Russian Federation, San Marino, Serbia, Switzerland, Turkey, Ukraine and Vatican City.

Source: [http://www.interrail.eu/help/ordering-pass/do-i-need- interra...](http://www.interrail.eu/help/ordering-pass/do-i-need-interrail-or- eurail-pass)

<comment><USER16> Whole lotta folk from San Marino! May as well mention sea land then.

</comment></comment></comment></comment><comment><USER17> For sake of comparison, Europe vs. Contiguous U.S.

[https://mapfight.appspot.com/europe-vs-usc/europe-united- sta...](https://mapfight.appspot.com/europe-vs-usc/europe-united-states- contiguous-size-comparison)

vs. All of the U.S.

[https://mapfight.appspot.com/europe-vs-us/europe-united- stat...](https://mapfight.appspot.com/europe-vs-us/europe-united-states-size- comparison)

For fun, here's a similar-ish map for calculating travel time in the ancient Roman World.

[http://orbis.stanford.edu/](http://orbis.stanford.edu/)

<comment><USER18> Wow, I've never seen a map of the world before. /s

<comment><USER19> Please keep comments substantive.

</comment></comment></comment><comment><USER20> Would love to see map of Europe reflecting distance by train, drawing countries with fast rail system smaller and those with slow rail system larger.

<comment><USER21> Yeah, I'm also pondering different ways of visualizing this data.. I wonder if it is possible to distort the map of europe so that map distance actually corresponds to train travel time. I guess not in general, because of metric curvature: you could have a small (fast) triangle that surrounds a large (slow to access) area.

</comment></comment><comment><USER22> I was confused as to why Ireland was showing up as accessible in these maps. Mainland Ireland isn't accessible by train from Europe. I found this excerpt in the original blog post which makes the concept slightly misleading:

 _Any points on water were assigned a swimming rate of 100 minutes / kilometer in order to create dense contours at the coasts._

<comment><USER23> You can buy through tickets from Great Britain to Ireland, there are stations adjacent to the passenger ferries.

<comment><USER22> Right, but that requires spending at least three hours on a ferry, therefore contradicting the concept of the title.

<comment><USER24> In Holyhead (ferry port in Wales with many daily connections to Dublin) you board the ferry from the end of the train station platform of the high speed rail. The slow ferry takes three hours, the express ferry takes 1.5 hours. When I was there in 2006 there were many direct trains from there to London Euston.

The other option might be ferries that carry trains. I believe they don't do this across the Irish Sea, but I've been on a train that drives onto a ferry across the Baltic sea connecting Berlin and Malmo (Sweden). You can get off the train while on the ferry, but you don't have to.

<comment><USER22> >...the express ferry takes 1.5 hours.

I take this ferry every few months and it takes a minimum of 1hr 50mins on a good day.

>The other option might be ferries that carry trains.

Not between Ireland and England.

My point is, I can get a train direct to any major airport in London and fly to a significant chunk of Europe in under 2hrs and continue my journey via train so why not factor in those options too?

<comment><USER11> When was the last time you bought a plane ticket from the train ticket desk in the train station?

<comment><USER22> Really? _That 's_ the criteria?

<comment><USER11> It might seem inconsequential and I can't answer on behalf of the author of the article however if you have ever done the rail and sail you might understand it a bit better. I get what you are saying but the difference in faff between taking a plane and casually walking from the train to the boat is huge.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> As a French, I wonder how British people judge their rail system in 2015

<comment> david-given It's crap.

At least, in my opinion --- I live in Switzerland now, so my standards are... different. But, basically, overpriced, unreliable, dirty, and slow. The only places where the train system actually works well is the south-east of England, where it's horribly expensive. I used to live in Reading; a peak time day return into London, which is maybe half an hour a way, costs about £40.

Bear in mind that I know perfectly well that Britain's rail network is _amazing_ compared to, say, America or eastern Europe. But the local _perception_ of it is that it's bad. (Not helped by several rounds of botched privatisation, too.)

Incidentally, I'd love to know how to get from Ireland to Great Britain by train. Do they float?

<comment><USER26> > I live in Switzerland now

How did you get into Switzerland? I've been considering / researching the possibility for a while, but it seems like a very long process to legally enter the country. Did you find it that way? Any recommendations?

[I looked for a personal email, but there isn't one listed on your profile.]

<comment><USER27> Is it so much harder for UK citizens? For (most) Schengen people it is really simple.

<comment><USER23> Schengen is irrelevant here, British people have the same rights as every other EU citizen (i.e. go, then register as a citizen which is just a formality).

But the person asking is American, which is presumably very different.

</comment></comment></comment></comment></comment><comment><USER28> One of the real reasons why i love living in Frankfurt. I feel closer to all of Europe than everywhere else on the continent.

<comment><USER29> To the world really, because we do not have to drive to Frankfurt first to use a proper airport.

I look forward to the Frankfurt-London rail link operated by Deutsche Bahn. Strange times, when the UK thinks about moving away from the EU and staying alone on their little island like in the good old times and yet London is nearly as close to Frankfurt as Frankfurt is to Berlin.

<comment><USER28> Actually i've recently had a chat with people from DB Strategy, they've said that the Frankfurt-London connection is now technically possible and has been tested, but the ICE that should be used on it has a lot of technical issues that push the chances of having technical issues on one trip to above 20%.

That is a reason why they have pushed that connection off into the undisclosed future.

He wasn't a technical person, but the way he described it was that everytime an ICE crosses a country border, it has to change the settings on how the train operates, to comply with local laws and standards. This is stuff like how rapidly it is supposed to brake, etc.

He says that there is a 1 in 10 chance that when doing that "settings-swap" some part of the train software crashes irrecoverably and the whole train has to be rebooted, which can take up to 15 minutes.

So when crossing from germany into belgium, then france, then uk and back again there would be a high chance that those trains would have to be rebooted somewhere along that track, and that would wreck havoc within the international train system.

</comment></comment></comment><comment><USER30> I enjoy traveling by train, but it's a bit of a shame that it's often economically unwise to pick a train over a coach bus if you're sensitive at all about the price. Difference can be 3x or higher, though some countries are better than others. (Italy, the UK, and Eastern Europe have pretty cheap trains, I've found.) Rail passes seem to only make things better if you're travelling a bunch over a short period of time, not once every few weeks. Still, I'd much rather travel by coach than air!

<comment><USER31> In the UK it's often cheaper and quicker to hire a car, so I'm not sure why you think UK trains are cheap?

<comment><USER32> For DjangoCon Europe this past week I flew into London, stayed a few days being touristy, went to the conference in Cardiff, then came back to London and am flying out (home to the US) in a few hours.

I took a train to and from Cardiff. It was £22 each direction, roughly 2.5 hour trip from London Paddington to Cardiff Central, and the same in the opposite direction.

I doubt very much I'd have been able to hire a car for that trip for £22 each way. In fact, I probably couldn't even have gotten out of _London_ for £22.

<comment><USER23> In Britain, you pay a lot for peak-time travel, or last-minute travel.

London to Cardiff and back is £218 at peak times(!) on a fully-flexible ticket, £74 with a flexible off-peak ticket, and as low as £40-ish (which you paid) if booked well in advance for particular trains.

It would have cost about £50 in petrol (return) to drive, plus £40/day for a small rental car.

<comment><USER32> How "last-minute" are we talking here? I booked the ticket on May 24, for the trip to Cardiff on May 30.

In the US, where intercity trips above a couple hundred miles are typically by air, six days out would be "last-minute" for most people, so that's my frame of reference.

Also, the return to London corresponded with the hordes of One Direction fans (whose suddenly-announced concert in Cardiff forced a shuffling of the conference schedule since every hotel in town sold out instantly).

<comment><USER33> In Germany I usually buy my ticket on my phone on the way to the station (I do have a BahnCard 50, i.e. 50% off all journeys). I could typically save a few quid if I booked non-flexible tickets a few days in advance (3-7 days). For longer distances (>400km) the savings become more noticeable and my trips less spontaneous, so I do tend to book those in advance as far as possible.

In the UK (I had a Railcard), I booked everything as far as possible in advance (quite often on the day advance tickets became available), as prices for advance tickets do vary quite a lot when you get close (weeks instead of months) to the day of travel. If you don't care which train you take, then you'll usually find affordable tickets a week in advance, but when meeting people at a fixed time, booking close to the day can yield unwelcome surprises.

</comment></comment></comment></comment></comment></comment><comment><USER34> I recently moved to Canada from Europe and one of the "cultural shock" were the difference in scale on the maps and what is commonly considered being "a short drive".

I did a tour of the most active hackerspaces of Europe in six months while I was in high-school. It was a fantastic experience and it is the cheap cost of night trains that made it possible. Granted those were not super confortable but I do not regret the experience one bit.

<comment><USER35> I would appreciate if you would list these most active hackerspaces of Europe!

</comment></comment><comment><USER36> Cool graphics. Once you get used to the colour scheme, you really see how connected are various European cities.</comment>


Morte: an intermediate language for super-optimizing functional programs - <AUTHOR> http://www.haskellforall.com/2014/09/morte-intermediate-language-for-super.html <comment><USER2> I love the work this guy does. He consistently and often develops really interesting Haskell libraries. Does anyone know his background?

<comment><USER3> I was a PhD student in biochemistry, but most of my PhD work was structural bioinformatics because I worked in a protein design research lab. Working on my PhD project is how I got exposed to (and addicted to) Haskell and programming in general, so I decided to change career paths to software engineering so I could feed my programming habit and now I work at Twitter.

<comment><USER4> What sort of work are you doing at Twitter? Does Twitter use Haskell?

<comment><USER3> I work on an internal analytics framework at Twitter.

Twitter doesn't use Haskell. The most Haskell I've done there is to write an internal tool in Haskell (A shell-based interface to HDFS). Everybody here knows that it's my mission to introduce Haskell more at Twitter and I have two long plays to do so (Morte is part of one of them).

You would think that it would be easier to introduce Haskell at a Scala-based company because of the strong similarities between the functional half of Scala and Haskell, but it's actually quite the opposite: the marginal benefit of switching to Haskell is smaller so it's harder to make the case to adopt Haskell. This is why you see Haskell take hold more easily in companies programming primarily in languages far-removed from Haskell (like PHP or Python) because then the marginal benefit of switching is much higher.

<comment><USER5> That reminds me of Joe Armstrong's talk—the best place to introduce new technology is in a company which is already experiencing technology failure. At that point, the pain is clear and options to resolve it command power and premium.

<comment><USER6> Could be why so many Groovy shops switch to Scala.

</comment></comment></comment></comment></comment></comment><comment><USER7> I'm confused by this. It seems to be promising a normal form for all expressions in a Turing-complete language—but that's equivalent to guaranteeing termination, isn't it? Moreover, it promises to produce equal results for equivalent programs, which would allow a solution of the halting problem, wouldn't it?

I suspect that I've got some bad conceptual misunderstanding here; but, on the other hand, I also have a hard time believing that, if beta- and eta-reduction alone were sufficient to express such powerful optimisations, it would have taken this long to discover that.

<comment><USER8> >Moreover, it promises to produce equal results for equivalent programs, which would allow a solution of the halting problem, wouldn't it?

I believe this is the case. So the article's promise (as I read it) can not hold true in the general case.

On a similar note, just write a program that runs the collatz conjecture and prints the result and a program that prints 1. Run them through Morte. If they compile to the same code, then boom, you've solved the Collatz conjecture :)

<comment><USER5> You can't encode the Collatz Conjecture (in that way) in Morte. It would require an infinite amount of computation and infinite loops have been disallowed. You would have to get a little bit more creative with the dependently typed parts of CIC and then finding a program which matches the needed type is as of today an unsolved task.

</comment></comment></comment><comment><USER9> I have always wondered why I couldn't do something in Haskell like:

        {#- COMPILECOMPUTE #-}
        factorial 0 = 1
        factorial n = n * factorial (n - 1)

And have the compiler transform something like "factorial 5" into "120" at compile time, since factorial is a pure function, it seems like that would be doable? Of course you'd have to use your pragmas very judiciously to avoid crazy compile times.

But there are some instances where, for code readability, I'd want to say "f 123" rather than whatever "f" evaluates to at 123, since "f 123" shows where the value comes from, but at compile time I'd like that to be optimized out.

<comment><USER7> You _can_ do this, at the cost of some heavy lifting to express it: [http://www.haskell.org/haskellwiki/Type_arithmetic](http://www.haskell.org/haskellwiki/Type_arithmetic).

P.S. This type of inlining was also discussed at [http://stackoverflow.com/questions/19259114/why-are- constant...](http://stackoverflow.com/questions/19259114/why-are-constant- expressions-not-evaluated-at-compile-time-in-haskell).

</comment></comment><comment><AUTHOR> > Also, if you're wondering, the name Morte is a tribute to a talking skull > from the game Planescape: Torment, since the Morte library is a "bare-bones" > calculus of constructions.

Clearly a mark of good taste.

<comment><USER10> Morte means "death" in Latin and also in my native language (Portuguese).

<comment><USER11> Almost every native English speaker knows the meaning of morte. We have a words like mortician which is a person who cares for the dead, mortgage which is a plan to kill you debt, rigor mortis which is the stiffness of death and mortal which means subject to death. Yes, we almost all know what morte means. That is why it was the name given to a talking skull in a game. Just like love and hate, however, we often use native words in English for our most basic concepts. Death is one of those.

<comment><AUTHOR> You also have Mallory's "Le Morte d'Arthur" (no talking skull involved in this one). That said, it was just a small nod to Gabriel's appreciation of the game, which after so many years I still consider the best RPG ever made.

<comment><USER3> I agree. Planescape: Torment was the best RPG ever. I still get chills thinking about it.</comment></comment></comment></comment></comment>


What's Driving the WSJ's "Truth About Broadband" Series - <AUTHOR> https://www.forbes.com/sites/roslynlayton/2019/08/31/the-agenda-behind-the-wsjs-truth-about-broadband-series <comment><USER2> What drives thinly veiled corporate propaganda?</comment>


Ask HN: how did you get started in programming? - <AUTHOR>

Age? Language? Catalyst? + Story <comment><USER2> 12: HTML + playing around on the internet. It is hilariously sad how badly designed sites were 12 years ago. "Hacking" emails was something so easy that a 12 year old kid with little knowledge of html could do.

I started learning CSS and newer HTML(back when I learnt HTML code was written in all caps) when I was 17 to earn some money to buy me books and a laptop for university. Did some Java in highschool and built small applet games. Then I learnt php at my first coop job(go UWaterloo!) learnt Javascript, Jquery, C# at simultaneous coops. Finally learnt Advanced Javascript(rather how to write javascript right) and fell in love with it and beyond that just kept improving my skills in online development. I keep them refines by working on multiple side projects.

</comment><comment><USER3> 12 - AMSTRAD BASIC - I wanted to create my own games. So I started to copy some games from books I had. But quickly I was bored because I found it was too slow to get things done. So basicly after the presentation screen I stopped there. Later when I got a PC, I did a calculator with BASIC again. But got bored again. I didn't knew about C language or others. I did some software cracking (reading ASM and patching) but got bored again. Now I do mostly Python, I'm proficient in several language, I'm happy :-)

tldr: I learned programming slowly at my rhythm but that's Python that triggered my "hacker" fiber and some reading like ESR books and books about how internet and firefox were created...

</comment><comment><USER4> 5 - QBASIC - My parents tell me how I spent lots of time on the computer programming mostly in QBASIC. Sounds crazy but I have a few memories of reading the help and trying to context clue words. It was hard because I couldn't understand most of the words. Read the programs that came with QBASIC to guess what statements did what. I specifically have a memory of context cluing that the word integer meant number. Of course I couldn't do much. I could only change the screen color and print text to the screen, afaik. I tried to create my own version of dosshell/win 3.1 in QBASIC.

</comment><comment><USER5> 19, Perl, I was going to school for Physics and I spent a lot of time mining spectroscopic data from mysql databases, converting binary dumps to csv files, etc... so I started to automate it. Eventually learned that I enjoyed writing scripts and automating things more than writing papers.

Got into networking/security, web development, and data infrastructure design. Started writing code in Python, C, and Java and catching myself up on the fundamentals of CS. Took a job as a developer, then another, then another....

</comment><comment><USER6> 13/14 - HTML/CSS - Angelfire - This turned quickly into BASIC for TI graphing calculators. Wasn't until 17ish that I started learning VB, Java, and PHP. Now I focus mostly on Ruby & JS.

Programming was always fun for me because I could build useful things that people use. It was great to write helpful formula apps for the calculator and give them away to friends. Same went for helping people with simple web sites and eventually applications.

</comment><comment><USER7> 10, Atari BASIC. It was around 1990. My father bought me a computer, for playing games, but I quickly discovered I could make my own (lame) games by myself. They sucked for sure, but I was fascinated by the power I had on the machine. How amazing and frustrating that was to see the machine was doing exactly what I told it, and nothing more.

</comment><comment><USER8> 19 - iOS - Ray Wenderlich Tutorial</comment>


Why It's So Hard to Create New Processors - <AUTHOR> https://semiengineering.com/why-its-so-hard-to-create-new-processors/ <comment><USER3> This article is spot on. Verifying hardware is a much more complex task than designing it. I've spent ~1.5 years on verifying custom cpu core re- implementation (next gen) that was almost binary compatible with previous gen, we had own gcc backend and LOTS of legacy programs written in assembly. And even then it was a nightmare. Even after tons of time spent on covering most obscure cases, our FPGA platform, running our multi-task FW, would often crash: cue in hours of back-stepping FPGA trace dumps, where you have a total recording of program counter, load-store IF, internal registers for aroung 100k clocks and you have to trace back the execution and see where the fault occurs. Usually it would turn out to be some weirdly obscure combination of recently-fired interrupt, some pipeline stall with execution of some weird instruction. fixed in 3 seconds when found. goddamn that was exciting. I wonder if there are any people here on HN that work on advanced HW verification techniques that are currently ongoing in the industy?

<comment><USER4> Yep — and I've already said too much. I get upset how secretive this industry is; I truly believe it's at least 30 years behind the curve (and falling further) compared to SW. I also firmly believe it is the EDA vendors who sell this mindset because $$$.

<comment><USER5> As a hardware designer (well, mostly a hardware verification engineer), I feel that this kind of comment is a pure misconception from software people that gets echoed time after time in HN.

I generally see a lot of more effort and good quality tools in hardware verification than in software validation. The hardware design and verification industry is not really 30 years behind the curve compared to software development.

Hardware and software are two very different problems with their own set of constraints. It's actually more often that you see software barely tested being deployed in the wild to millions of users rather than hardware. Hardware bugs tend to get more attention because you can't really deploy a patch to fix the issue.

<comment><USER6> Having worked with both HDLs and software tooling, I don't think it's too far off the mark. HDL tooling is terrible software: it's hard to use, buggy, expensive, opaque, and the support is terrible unless you're a huge customer. Hardware verification and development works still through sheer effort, and because there's a strong business incentive to get things right the first time due to the huge costs of a respin. With better tools this could be done faster and cheaper with similar reliability, but perhaps there isn't enough to be gained to make it worth the effort (and either way I don't think the companies currently working on the tooling are capable of making anything else: this would need disruption to really make a difference, and the barriers to entry are huge).

<comment><USER7> Hardware design flows are complex and do have bugs, but not in the way it sounds. In general HDL synthesis produces a design that matches what the user wrote, so a logic bug in a synthesis engine is rare. The bugs exist in the 'other stuff', like trying to convince the tool to be efficient about how it created the design, or smart about how it's trying to close timing. Getting a working though the tools design is easy, getting an optimal design is hard. HDLs themselves have issues, but there is a wealth of quality checking tools built around them that mitigate them, and indeed mitigate/flag poor coding from inexperienced designers (to a degree). What's interesting is FPGA tools, as much as they are derieded, are much better than their Asic counterparts in terms of functionality and user friendliness.

</comment></comment></comment></comment></comment><comment><USER8> Its probably better stated, as its not hard to create a new processor, anymore than it is to create a toy OS.

The hard part is creating something that is competitive with top of the line commercial processors that have thousands of man years of R&D poured into them. Its not just verification, but the huge effort that goes into eaking out another couple percent on something like a branch predictor, or optimizing some "edge case" that turns out to be a significant portion of a benchmark if its not done correctly. Then there are all the general optimizations that give you a 10% uplift here and there. Worse, yet if you go with something that doesn't have a large installed software base (x86/arm/power?) because your going to be spending crazy amounts of effort doing compiler+application optimizations as well.

</comment><comment><USER9> If anyone is interested in this sort of thing, I would highly recommend checking out the documentary Rise of the Centaur. It's about a company that I previously hadn't heard of who was making an x86 compatible CPU based in Austin TX.

They show a lot of the verification process throughout the film, including an exciting moment when the chip boots Windows for the first time.

<comment><USER10> On Amazon prime video: [https://smile.amazon.com/Rise-Centaur-Glenn- Henry/dp/B01FSZU...](https://smile.amazon.com/Rise-Centaur-Glenn- Henry/dp/B01FSZU6FK)

<comment><USER11> Couldn't watch it outside of the US but it's also on Vimeo: [https://vimeo.com/ondemand/riseofthecentaur](https://vimeo.com/ondemand/riseofthecentaur)

</comment></comment></comment><comment><USER12> For our software friends wondering what's so special about processors: they are the most parallel state machines designs out there. Most other state machines have fairly well defined and narrow inputs and outputs. For performance reasons, a cpu pipeline is the biggest collection of state machines interacting with each other directly.

Therefore most formal methods blow up on cpu designs and random coverage is really hard to define and even harder to reach.

</comment><comment><USER13> I try to keep a list of useful open source hardware verification tools: [https://github.com/ben-marshall/awesome-open-hardware- verifi...](https://github.com/ben-marshall/awesome-open-hardware-verification)

</comment><comment><USER14> Anyone working with this stuff: what are some textbooks on verification etc. Preferably a bit of theorem proving too but I'm not sure if that's in the same area of study?

I'm coming from a physics background so I never really know where to start when I inevitably start looking this stuff up at 4am.

<comment><USER15> The super basics are here: [https://www.amazon.com/Art-Electronics-Paul- Horowitz/dp/0521...](https://www.amazon.com/Art-Electronics-Paul- Horowitz/dp/0521809266)

You'll first have to become 'fluent' in EE, but for a physicist, it's just spending the time and getting used to things. Not terrible, long, but straightforward.

As towards what the article is talking about, you need to be trained in it. Honestly, you have to apprentice with the Greybeards (they are mostly men, but not always). There are other ways, like reading through Intel docs or the manuals for ICs or digging through forum posts from 2003. But those guys in the basement with funny newspaper clippings from the 80s or old xkcd printouts are a _much_ better return on your time. They have tons of knowledge about specific chips and machines, stuff that is nearly impossible to recite unless prompted. You just got to spend long lunches blabbering with them, despite their strange political and societal views. Just listen to them, then write down every little thing they said. They are _gold_ in terms of hardware.

<comment><USER16> Art of Electronics is a nice book, but pretty irrelevant to verification.

</comment></comment></comment><comment> ur-whale >"This brings a whole new set of challenges because they are speaking a completely different language, both technically and mentally."

Both true, but also an incomplete picture.

It's not just the mental models and the language but also the _culture_ that is very different.

H/W guys are bred, born and raised in an environment that thrives on secrecy and where nothing is ever free.

The way they transact with one another, the tools they use, and in the end, the very thing they produce all exude that culture.

It is _exactly_ the software industry 40 years ago.

<comment><USER17> It's not just HW, it's all big, niche industry, today.

Networking for example is the same. If you want to test high scale network equipment OR virtualized network functions, you will buy hundreds of thousands of dollars worth of closed-source testing hardware, software and/or professional services from one of a few big vendors. You will not let anything about your algorithms and designs slip to the outside world, and neither will your test vendors.

Edit: the same is true of most of the software world in general. Sure, you have Microsoft and Google and many others collaborating on Linux, or releasing Kubernetes, VS Code, Go and so on. But the core IP that is key to their business? That is staying in-house, fiercely guarded, developped and tested by an army of engineers.

The main difference is that there are far fewer well-defined software classes that can be tested generally, so it doesn't make too much sense to look for a 'software testing' industry, like you can for hardware. There are some tool vendors, but they offer far fewer guarantees, since it's hard to imagine a product that could find a large proportion of the bugs in both the Haskell compiler and World of Warcraft.

</comment></comment><comment><USER18> _" Verification of a processor is different from the verification of other pieces of IP, or even an SoC."_

Not sure I understand this. An SoC is a processor, plus more stuff (memory, I/O), right? Is the idea that it might be easier because the "more stuff" abstracts away some inner details?

<comment><USER16> The processor is usually already verified on its own before it is put into a SoC.

Using the raspberry pi example from another thread: broadcom designed the BCM2835 SoC, which included an ARM1176 core. Broadcom probably didn't do a ton of verification for the ARM1176 core itself, since ARM already verified it.

</comment></comment><comment><USER19> Have there been any attempts to use generative testing (e.g. quickcheck) or dependent types to verify processors? I am not sure how quite how this would be integrated into the synthesis of the processor, but it seems to be in line with the general "declarative" approach to building RTL through VHDL I remember from undergrad.

<comment><USER20> There is a lot of random testing in processor design. To what extent you'd call it property-based testing can be argued. Intersting factoid: Koen Claessen, one of QuickCheck's inventors, also co-designed Lava, a circuit designer DSL in the Haskell eco-system.

If by dependent types you mean theorem provers, then that is used, but rarely -- hand-verification doesn't scale to modern processors, usually you model check against some temporal logic formulas that the processor meets its specification. If OTOH you mean using HDLs (= hardware description languages) that use dependent types, then mostly not. Arm's ASL (= Architecture Specification Language) has a tiny bit of dependency build in to reason about length of bit vectors.

</comment></comment><comment><USER21> Genuine question - other than increasing the number of players, what is the horizon of value for new processors to begin with ? Is there anything more impactful than squeezing a bit more performance/watt ?

<comment><USER3> cost. it is stupidly expensive to include any sort of commercial MCU core(s) in your chip. Ancedata: ARM won't even talk with small ASIC fabless companies, even if they are willing to shell out big upfront costs and royalties per chip that ARM demands. Having free or affordable alternaties is a great driving force for the industry.

<comment><USER22> > ARM won't even talk with small ASIC fabless companies, even if they are > willing to shell out big upfront costs and royalties per chip that ARM > demands

[Citation needed] There's literally 0 upfront costs for a Cortex-M3 [0]

[0] [https://developer.arm.com/products/designstart](https://developer.arm.com/products/designstart)

<comment><USER23> That's pretty recent for ARM, as they face growing pressure from companies switching to RISC-V for their internal MCUs.

</comment></comment></comment></comment><comment><USER24> I would say a modern CPU is a massively parallel program that has lots of shared global state. No wonder that it's hard to design them.

Or is that view to simple?

<comment><USER25> It's more like you have a massively parallel program that you compile ten million times, but only five million complete the process usable afterwards and of those five million, they may or may not have all the parts of the program you intended in the binary. And you have very little insight into it while the compilation takes place.

And you have to check/debug those 10 million compiler passes at various stages, and each design change may require developing a new debugger or disassembler from scratch to plug into the compiler at each stage of compilation.

What I'm saying is that CPU designs aren't programs, because you can generally trust the compiler to be infallible (and compiler bugs are there, but they're rare). In a CPU process you have to consider the physical impact of the design on manufacturing, what yields you get, how the product is binned, and so on. There are feedback loops between the packaging, testing, and design teams to alter the silicon before production ramps up to go to market. There are tons of moving parts to the actual design process itself, let alone what is being designed.

</comment></comment><comment><USER26> I don't think this is quite right. Based on my (admittedly limited) experience, it takes a lot of work to design and verify a fast processor, but unless your processor is very similar to existing ones (in which case why bother?) it takes way way more work to write all the software needed to support it.

I guess everyone underestimates how long it takes to write software - <USER2> hardware designers.

</comment><comment><USER18> Interesting when coupled with how many we are losing. It wasn't that long ago that PA-RISC, Sparc, Alpha, Power, MIPS, etc all credibly competed with one another and Intel. Now it's almost all x86-64 and ARM.

<comment><USER27> ISAs are consolidating sure, but the interesting parts of the chips are also consolidating. A few years ago several companies were designing new Arm cores; now it's pretty much just Arm and Apple.

<comment><USER16> There are still other companies doing new core designs: Huawei/HiSilicon (Taishan v110 in 2019), Marvell/Cavium (Thunder X2 in 2018), Samsung (M4 in 2019), Fujitsu (A64FX is 2019), Nvidia (carmel in 2018).

If you count semi-custom cores derived from ARM designs, then add Ampere Computing and Qualcomm as well.

<comment><USER27> In the server space that's true, Arm cores are proliferating. But Samsung shut their core group down and Qualcomm is moving that direction.

</comment></comment></comment></comment><comment><USER28> Another important question is why is it so hard to use the old ones.

</comment><comment><USER29> >Brute-force solutions to verification closure aren't feasible.

Perhaps a fuzzy/AI type approach?

Yeah does seem like a intractable problem for sure

<comment><USER13> Hardware verification engineers call it "constrained random verification", but it's basically fuzzing. This has been the backbone of most commercial hardware verification flows for a long time.

</comment></comment><comment><USER30> Why is it supposed to be so complicated to do "processor verification"?

Why can't you simply upload the design to an FPGA, and then check that it can:

1\. Boot all available operating systems (Linux, *BSD, Windows, etc.)

2\. Successfully compile and run the testsuites for a bunch of open-source software (several languages like Rust have a standardized repository and method to build and run tests, so this is very easy)

3\. Correctly run stress testing software (Prime95, etc.)

4\. Correctly run several software unit tests that you write to exercise instructions that may not be produced by LLVM/GCC

5\. Correctly run tests you write to exercise specific processor/cache states

6\. Properly handling fuzzed code without freezing the whole CPU (using afl- fuzz)

Start with the simplest possible in-order core so that you get it working very easily, and then evolve to your desired end-state with a series of small commits, and if the verification fails use `git bisect` if needed to find the offending commit, insert any instrumentation you might need to detect the issue and fix it.

I don't see why you would need a specialized tool for that, or even what a specialized tool could possibly do.

<comment><USER31> Have a look at the extensive errata Intel publish for their CPUs. There are hundreds of mistakes in the chips' behaviour, and yet each buggy CPU would pass your set of tests with flying colours.

While you could never release a CPU that didn't pass the tests you describe, they don't even begin to exercise all the corner cases for a chip. Multiplying two specific numbers together, while the instruction crosses two memory pages, when an interrupt arrives? How do you even test for that kind of thing?

<comment><USER32> I feel the SW world is affected by some analogous bugs though. Any sort of race condition between two different threads accessing the same resource, maybe throw in some other piece like having the data always be valid unless a third thread happens to free some downstream resource at the same time...

We know about techniques to reduce large classes of errors. Data races in particular can be prevented by some languages statically. Other types of "once in a blue-moon" errors that happen as a result of two coupled systems doing something in tandem can be reduced by introducing stronger boundaries between the systems, and then you can test each system independently and make sure it works regardless of what the other system does (I.e. dependency testing, or maybe even fuzzing).

These approaches aren't bulletproof, but I think they do illustrate a point: that there _are_ techniques to reduce the likelihood of the errors you highlight. Whether they do it at a competitive cost to existing industry practices or not, I have no idea.

<comment><USER33> Hardware is usually orders of magnitude more reliable than software, so what makes you think that they aren't already using those techniques or something better?

<comment><USER32> Hmm? Maybe they do, I _hope_ they do. I was replying more specifically to this part: > Multiplying two specific numbers together, while the instruction crosses two memory pages, when an interrupt arrives? How do you even test for that kind of thing?

I.e. trying to despell the idea that large systems are intrinsically difficult to test.</comment></comment></comment></comment></comment>


Is Geotagging on Instagram Ruining Natural Wonders? Some Say Yes - <AUTHOR> https://www.nytimes.com/2018/11/29/travel/instagram-geotagging-environment.html <comment><USER2> Do you know what is ruining natural wonders? People are.

<comment><USER3> That's not true. The concept of a natural wonder is ruining natural wonders.

In Australia, there is a big rock called Uluru. It is so popular it has its own airstrip, and about quarter of a million people come from all around the world to see it each year.

North of Uluru a fair way, there's a rock formation called the Bungle Bungles. About 25000 people come to see the national park it's in yearly. It is, according to my biases, better in almost every way than Uluru, and sees a tenth of the traffic.

I think that if nobody had a checklist of "natural wonders", they'd have to actually explore an area to find things that excited them, and while they'd have a similar short-list in each area, most people would have a different winner.

Condensing an area into a single natural wonder doesn't do the rest of the wonders justice, I feel.

<comment><AUTHOR> With respect to Uluru it's also worth mentioning that the Aboriginal people consider it to be sacred and don't particularly appreciate many of the tourists who elect to climb over it. From wikipedia

"The local Aṉangu do not climb Uluru because of its great spiritual significance. They request that visitors do not climb the rock, partly due to the path crossing a sacred traditional Dreamtime track, and also due to a sense of responsibility for the safety of visitors. The visitors guide says 'the climb is not prohibited, but we prefer that, as a guest on Aṉangu land, you will choose to respect our law and culture by not climbing'"

[https://en.wikipedia.org/wiki/Uluru](https://en.wikipedia.org/wiki/Uluru)</comment></comment></comment>


A Billionaire in the Oval Office? - <AUTHOR> http://unanimous.ai/a-billionaire-in-the-oval-office/ <comment><USER2> I'd prefer Bloomberg over Trump, but we should not elect billionaires...

<comment><USER3> What is your reasoning? Because they are out of touch with most people? I would argue that Trump is closely linked with the most 'value' (in money or financial instruments) in America.

Oh, I see what you're afraid of.</comment></comment>


Ask HN: Why is Docker so popular in comparison to other containerization tech? - <AUTHOR> Has it only been the first mover effect or are there (decisive) features that other technologies don't have? <comment><USER2> They have focused relentlessly on making it easy to use for every day development. Solaris Zones, BSD Jails, and LXC were around first, but were (relatively) arcane and finicky, and no one ever looked at bringing them to non-linux (or non-bsd for jails) dev laptops. OTOH, Docker made a good (enough) implementation, and went for developers, developers, developers. They brought it to Mac, to Windows, they engaged a wide audience and mostly listened.

Focusing on day to day development for users came with tradeoffs -- ops oriented features came much later, getting serious about security came later, etc. These are all important things... but less so than ease of use and adoption.

The pattern is not new, the same thing happened with Rails -- the concepts in rails were not novel, the extreme ease of getting started and getting to _useful_ really fast was the killer feature (to get it going). After it got a big ecosystem, it got the quality aspect (mostly by replacing itself with merb).

The lesson: if you want wide adoption, build something good and reduce every barrier to adoption you encounter. The most important features are the ones getting in way of people using it, not the ones people using it are having. You'll need to solve them, but solve adoption first.

<comment><USER3> > making it easy to use for every day development

Can you elaborate here? I've started using Docker for development recently (pretty much just to avoid local mySQL troubles in a Rails project) - we use a docker-compose file to start an official mySQL image and the local rails repo.

Seemed easy enough, until I realized I have to restart my container every time my code changed. Then I had to integrate docker-sync, which is a great library but not official. I'm still surprised code sync is not included out of the box from Docker yet - am I missing something?

<comment><USER4> Yes. Volumes: [https://docs.docker.com/engine/admin/volumes/volumes/](https://docs.docker.com/engine/admin/volumes/volumes/)

Mount a directory from your local filesystem as a volume and file changes within that directory will appear immediately in the container.

<comment><USER5> This even works when one uses docker build to compile things. Just have a shell script that after the build copies the compilation result to a local directory mounted as a volume in the docker container.

<comment><USER6> Why not compile inside the volume already?

<comment><USER5> This is what I have used before docker build started to support multiple images. That is, I had a shell script that run an image with a compiler that put the executable to a local directory. Then normal docker build would pickup the executable into the final image. But this does not take advantage of caching with docker build and made the infrastructure more complex.

Dockerfile with multiple images addressed that nicely. Now single docker build command takes care about everything and generates production ready images. A shell script is only necessary during development to extract some files from the image to avoid restarting already running container.

</comment></comment></comment></comment></comment></comment><comment><USER7> \- It is the complete solution: the image format is easy to understand and well agreed upon, they also offer an image registry (they host it or you can host yourself)

\- Great UX: Docker Inc is probably one of the small companies that provide great developer experience. They have a natural skill for it. Another one would be HashiCorp tools. These companies understand developers well. When a tool naturally speak the same language as you, you fall in love with it.

\- They have a great development story: lxc, jails, zones, ... you can't get them to work on Windows or Mac. But Docker for [Mac|Windows] offers a complete development story (you don't have to be on Linux to develop Linux containers).

\- Solid image ecosystem: They paid external parties to keep the official images on Docker Hub (like mysql, nginx) solid i.e. up-to-date, free of critical security vulnerabilities. I don't know of any solid base-image ecosystems in other containerization ecosystems.

\- Big companies got behind it: This is a bit backwards, but since docker got popular, companies like Red Hat, IBM, Microsoft, Google started to contribute to it early on. This brought in a lot of mind share and publicity to project (and disagreements).

\- They got a solid PR machine: They do great community management. Docker Captains program is a solid one. They had many DockerCons they're all pretty good (although one would argue past few ones got kind of corporate/enterprise-y maybe even boring). But they had a few good ones.

\- The tech has merit: "docker run", "docker ps", "docker rm" the whole docker CLI is very intuitive overall. This is a bit of an supporting argument to UX, but it hides a lot of the details of how the docker-engine works.

<comment><USER8> Indeed, Docker was not perfect, but was easier to use than LXC at the time it got popular. There is a certain barrier in usability(cost) and usefulness(reward) that must be passed so that hype (natural or induced) can do the rest. Looking from a reward/cost perspective, this is how Docker appealed to me:

Rewards: \- Hygienic build&deployment - being able to specify your requirements down to the OS lib versions \- Lower barrier to deployment (easy for users to try out my app and for me to try theirs)

Costs (mostly time): \- LXC was something that I understood, but could't make good use of to be worth it the time spent. Docker on the other hand required less time to do something useful (e.g. pull and run an image, turn an app to an image).

Time to learn a new tech depends on its complexity (concepts and relationships) and how it is revealed to the user (docs, tutorials etc). Willingness to learn the complexity depends on the distribution of rewards along the time axis. I think Docker guys did a great job improving the latter.

> But Docker for [Mac|Windows] offers a complete development story

Docker on Windows has been a nightmare for me (crashes all the time) and I have abandoned altogether. I'd happily get rid of Windows too if it wasn't the required in my work environment.

<comment><USER9> I wouldn't call docker easier to use than say libvirt. (Which it had as a backend until they dumped it.)

Docker setup is especially bad at employing custom images easily and composing small pieces of software instead of huge packages... at least that was my experience.

</comment></comment></comment><comment><USER10> Docker based on LXC and added a few things. The fact that they were well funded made outreach easier compared to the open source LXC project and Docker became many peoples first introduction to containers.

Once they got traction they quickly dumped LXC around version 0.9 and created their own engine in GO. All the while misleading and letting misconceptions and plain untruths about LXC being 'difficult', some 'low level tool' or 'bash scripts' spread unchecked by their ecosystem. So the model is not only take open source tech, but bash them in public and forget about giving anything back.

LXC is much simpler to use and understand both conceptually and in practical use. Docker is basically a hijack of an open source project supported by the SV crowd and the level of support and misinformation can be verified by HN posts in the last 3 years. Docker posts routinely clock 600-1000 votes in short periods of time and are full of marketing and misinformation.

That's history but the culture continues. Docker and other related projects continue to liberally wrap open source projects while downplaying them, ie overlayfs, aufs, kernel namespaces, cgroups, Linux networking tools, nginx, consul, haproxy and more. Someone had to create those projects and make them usable. This would not be a bad thing if they actually made things simpler, but they keep adding more and more layers of complexity.

What incentive is there for people to create these high value open source tools if vc funded companies like Docker and corporate funded projects like Kubernetes simply wrap them, downplay them and hog the attention and funding? These kind of projects basically cut the tree they sit on for their own short term benefit.

</comment><comment><USER11> I can tell you what attracted me to Docker as an early adopter (while @Mailgun):

* Very good user and developer experience. Docker felt like git for containers. Docker run, docker ps and other commands are awesome.

* Very good and innovative build system. Dockerfiles are great and allow to build complex containers and efficiently package software in a convenient way.

* Solving image distribution problems. Docker pull/push are great.

* Layering system leading with build and run optimization.

Overall docker not only provided a great UX, but combined all the patterns above in a very expressive system, great job done by the team.

</comment><comment><USER12> I strongly believe it's the ease of use factor. If you know nothing at all about containers, but have just installed docker, you can immediately "docker run httpd" and watch a web server install itself and do something useful in seconds.

That's really powerful, and the surprisingly diverse selection of pre-built containers on the docket hub makes it even easier to bootstrap your project and figure the details out later. I don't have the necessary expertise to compare it meaningfully to other container systems so I can't speak to it's implementation details, but the first time user experience is more than enough to explain it's popularity.

</comment><comment><USER13> Just my cynical view, but I'm thinking it's because is got a friendly looking whale as a logo and the developers took the approach of making it friendly to use for someone who doesn't understand the underlying quite complex technology.

I was involved in making the decision about introducing containers to my workplace, my original recommendation was to just go with chroots or buy into something that we could get vendor support for on RHEL6 (so not docker) and when asked to explain how this would benefit out desired outcome, the docker website basically had copy-pasteable words that did that job for me, chroots however just described he technical implementation of a chroot.

So eventually I just went "fuck it, we'll run unsupported and take the risk, then well move N thousands of servers to RHEL7 because I don't want to have to explain to managers how cgroups and Linux kernel constructs work". About a year later most of the enterprise has a plan to move all their existing EL5/EL6 workloads to EL7 just in case they need to use docker some time in the future.

Which, if I'm honest, I kind of regret, but it doesn't impact me that much so meh.

<comment><USER14> Which bank was it?

<comment><USER15> Some banks aren't doing this :p

</comment></comment></comment><comment><USER16> Because they spent two hundred million dollars promoting it.

<comment><USER17> They raised the money after, and largely because, it had popular groundswell amongst developers.

</comment></comment><comment><USER18> I came from using Vagrant at a previous company and from my experience, Docker is just more lightweight, portable, and user friendly. Share a file/folder? Expose a port? Get an image and run a container? You can do it easily using Docker cli. You want a UI tool that makes that task even easier for you, get Kitematic.

Also, Docker just has more development and maturity at this stage that it makes deployments easier.

</comment><comment><USER19> I wrote something a few years ago about why docker matters: [https://circleci.com/blog/it-really-is-the- future/](https://circleci.com/blog/it-really-is-the-future/) (excuse the slightly flippant tone). I think the answer to why it succeeded instead of the others is that it solved those problems (in particular the container format and the UX around it) much better than the alternatives.

</comment><comment><USER20> Despite what everyone is saying, Docker is not great UX, not easy to use etc.

It has great _er_ UX and it's eas _ier_ to use than the available alternatives.

<comment><USER21> Do you not think that if everyone is saying it has a great UX and it's easy to use then perhaps it _does_ have a great UX and is easy to use?

<comment><USER20> We as developers are not spoilt for choice. The tools we use are universally crappy, horrible etc.

Any improvement is hailed as great UX/DX even if the improvement is only marginal. Or if the improvement only touches on certain aspects and totally ignores all other aspects.

Docker is an improvement over the cesspool of tools that existed in the space before it. It's UX is ok at best, and is hailed as awesome because it just barely covers the basic needs of a developer: list a thing, build a thing, remove a thing. Outside of really a handful of basic things it fails miserably and relies on multiple disparate hacks by desperate developers.

The only reason it's being praised is that _all other tools are so much worse_.

<comment><USER21> Chef Ramsey makes a cake, everyone has a slice and says it's the tastiest and best looking cake they've ever had.

Jamie Oliver comes along and says "No, no! You're wrong, it's not a tasty cake and it looks terrible, it's just tastier and prettier than you've had before".

Does the fact that everyone, other than Jamie Oliver, is claiming the cake to be incredibly tasty and the best looking not by default make the cake both tasty and great looking?

Surely the proof is in the eating?

<comment><USER20> >it's the tastiest and best looking cake _they 've ever had._

I've italicised the relevant part.

Let's say, all the cakes they've ever had were baked dirt. Now Chef Ramsey has come along and baked soft dirt with sugar and sprinkles on top.

\- Is it still dirt? Yes.

\- Is it the the tastiest and best looking cake they've ever had? Also yes.

\- Is it not a tasty cake and it looks terrible, it's just tastier and prettier than you've had before? Yes.

This also goes to show that arguments from analogy are inherently false, including mine ;)

</comment></comment></comment></comment></comment><comment><USER22> They nailed the developer experience which allowed so many people to build upon their interface.

</comment><comment><USER23> The main reason is that the underlying Linux container tech has (especially in the past) such terrible usability. Try to use cgroups or other older tools for running your browser, for example.

</comment><comment><USER24> Can you list some alternatives you'd like to see it compared to?

<comment><AUTHOR> Sure:

\- LXC

\- Rkt

\- *BSD jails

</comment></comment><comment><USER25> Isn't it simple? It was the first and only easy interface to LXC, which I think was the first and only Linux containerization framework, and Linux is the most popular server OS.

</comment><comment><USER26> I'm not even aware of any other container system that lets me run something like "docker run node" right now. So I vote for the infrastructure and hosted images which made it kind of like the GitHub of containers.

<comment><USER27> [https://linuxcontainers.org/lxc/manpages/man1/lxc- execute.1....](https://linuxcontainers.org/lxc/manpages/man1/lxc- execute.1.html) -> lxc-execute is kind of similar to docker run, UI wise.

But it's not the same thing as the Docker "package".

</comment></comment><comment><USER28> This may be a good forum to ask. For an absolute newbie to Docker but someone who has used LxC a few times in non-serious situations, what would be a good learning resource? Just the docs? Any good book? Any blog post series?

<comment><USER27> Read they're docs, they're quite good.

Then just start using it. docker --help, look at the commands available, see what they do.

It truly is quite intuitive and as you hit some barriers just google for answers, which you will find (yay, popularity!).

Then create some Dockerfiles.

After a few weeks you should know more than enough to use it to cover local development needs. If you want to go to prod with it then that can get quite tricky, depending on your scale. But, baby steps :)

</comment></comment><comment><USER29> I would say UX. Things like this need lots of people to adopt it to hit critical mass.

UX lowers the friction level. It's more likely to be accepted by the simplest decision maker. This is why companies like Slack do so well.

</comment><comment><USER30> Because it is robust and lightweight. It tackled all the facets of apps isolation.

<comment><USER31> It most certainly does not. If you are using Docker containers for security or QoS you're doing it wrong, and I think even Docker the company would concur.

It's a fantastic packaging and deployment format, but it's not a universal solution.

<comment><USER32> Do you have any __current __sources for the security part?

Everything I read is along the lines of "this is not secure, we think, but do not have any specific examples. But be careful"

<comment><USER31> [https://news.ycombinator.com/item?id=16030107](https://news.ycombinator.com/item?id=16030107)

</comment></comment></comment></comment><comment><USER5> For me the main reason to stick with docker is the fast build. Other solutions just do not have anything that usable to create images during development.

</comment><comment><USER33> for me, because when i first heard about it my options were Chef/Puppet (with Vagrant)... so Docker was much, much simpler to get started.

in the past years they've been adding a lot of complexity though... :meh:

</comment><comment><USER34> It's not only a container tech, it's the container format and distribution.

</comment><comment><USER35> What other containerization technologies?

<comment><USER36> \- chroot

\- Zones (Solaris)

\- Jails (FreeBSD)

\- OpenVZ (Linux)

\- LXC (Linux)

\- VServer (Linux)

\- lmctfy (Linux)

\- WPARs (AIX)

... and more; see [https://en.wikipedia.org/wiki/Operating-system- level_virtual...](https://en.wikipedia.org/wiki/Operating-system- level_virtualization)

<comment><USER37> Docker images is a simple quick-and-dirty packaging solution with all the pieces necessary (building, versioning, storage and deployment). I believe this is why it won.

Don't think any of those other options had anything similar. The alternatives, like building and deploying apps with native OS packages (e.g. .debs) were significantly more complicated and less portable. (While it's not that Docker images are better than native packages, they're doubtlessly generally easier.)

</comment></comment></comment><comment><USER38> the best part of docker isn't the containerization tech but the networking IMO.

</comment><comment><USER39> first mover advantage

</comment><comment><USER40> It's got a good name, and was an early mover in the field.

<comment><USER41> lxc and jails are both good names (in fact, I think bsd jails is the best name of those).

lxc and jails both predate docker by quite a while.

<comment><USER42> Jails is an absolutely terrible name when you need to sell your company, rather than your friends, on using this technology.

"Pff, yeah right, no one rejects a technology just because the name has negative connotation in a different context", except that is literally what happens.

<comment><USER43> Ah, the cockroachdb argument. We're rapidly approaching the point where any meaningful names are already in use, and everything uses nonsense names anyway.

<comment><USER42> Nonsense names are great, like "MySQL" or "Kubernetes". Names that the people you're trying to sell it to recognize from normal language like "jails", "cockroachdb", not so much. If you're in a startup and you need to sell it to your CTO: whatever. If you're in a 2k+ employee company and you need to sell it to general management: you are not going to be using jails, or cockroachdbs, or anything else that makes non-tech people raise their eyebrows in the bad way when they hear/read them.

<comment><USER44> Kubernetes isn't a nonsense name. Κυβερνήτης is Ancient Greek for helmsman. Which is not unrelated to Docker.

<comment><USER42> If someone needs to know ancient Greek to understand a name, the name in modern English is almost guaranteed a nonsense name, and that's great.

</comment></comment></comment></comment></comment></comment></comment><comment><USER45> Everyone keeps mentioning Windows support, but - and someone please correct me if I'm remembering wrong here - wasn't it the new Microsoft with Nadella's open source friendliness that went the extra mile to get Docker on Windows to happen?

<comment><USER46> "Docker for Windows" runs Linux containers in a Linux VM, with a thin Windows CLI and quite a nice install screen. It required no special help from Microsoft.

The product that runs Docker containers natively is called "Windows Server 2016", and this required a vast amount of work from Microsoft. Native Windows containers are not seen much in the wild.

<comment><USER17> > _Native Windows containers are not seen much in the wild._

Cloud Foundry has them for Windows 2012, though they're quite leaky. Windows 2016's support is much better and the resultant containers have much of what you want in terms of proper process isolation and dense utilisation.

Disclosure: I work for Pivotal, we contribute to Cloud Foundry. The Windows container team sits two desks over from mine.

</comment></comment></comment><comment><USER47> Docker is primarily a packaging framework + a runtime mechanism.

It's like VM + hypervisor in one box.

Thinking this way, it's easy to understand why it's popular. It did things that no other have done.

<comment><USER48> It's not like that at all in any way.

You're using existing OS packages in your base (apk in Alpine or apt in Debian/Ubuntu) to pull in all your dependencies and then placing your app on top of that. (Or using the base go, python or ruby container and letting it use pip/bundle to pull in dependencies and put your app on top of that).

It only runs a single process, unlike a VM (or even an LXC container which is more similar to a VM).

Containers are (mostly) immutable, so you're typically not running security updates on the same thing that provides all your dependencies. You should rebuild your containers when there are security updates .. but people rarely scan for those or do that; relying more on isolation and cgroups to keep those containerized apps safe.

There's very little tooling within the container, so when things go wrong, you're either execing into a single container and apt-getting all the tools you need (which you gotta do each time you rebuild it) or you need to use something like sysdig.

It's popular because you can create an easy to replicate artifact that doesn't rely on a ton of system dependencies with a really basic (although kinda primitive) package building system (Dockerfile) and that is way more lightweight than a VM while providing a pretty heavy layer of isolation.

<comment><USER46> Small correction: Docker containers can run any number of processes. For example the standard apache container runs five.

<comment><USER17> They can, but Docker-as-Docker won't support multiple processes. Instead you have to provide your own logic to spin up multiple processes.

</comment></comment></comment></comment><comment><USER49> My guesses: Docker has been around a while, is mature and is battle tested; it was a bit ahead of other options when critical adoption decisions were being made. I remember using LXC in Debian (and Ubuntu) around 2013 and hitting a lot of disturbing bugs, due to immaturity...

<comment><USER14> It's not mature and not battle tested. There is barely anyone using it.

They few who do are walking on eggs. it's hardly possible to find two experiments running the same ecosystem between all the distributions and the commercial services. Centos, Debian, RedHat, CoreOS, kubernetes, AWS ECS, Google container, cloud foundry, openshift...

</comment></comment><comment><USER50> Docker is popular because: 1\. containers are technologically better than virtual machines 2\. Docker has added a popular cloud container image repo which is used for accessing container images. 3\. this Docker cloud repo has morphed into a popular platform for selling/distributing application software which is built into container images.

</comment><comment><USER51> I don't know of any other containerization technology except really small linux distros that you can run in virtual box. Maybe that's why dropbox is so popular. They marketed the hell out of it.

</comment><comment><USER50> Docker is very popular because: 1\. containers is technologically superior to virtual machines. 2\. Docker has create a public cloud repo for sharing/selling container images. 3\. The Docker cloud has become very popular way of distributing application software which is built into container images. Number 1 is technological reason, number 2,3 are non technical features which Docker added to containers and have become popular for enterprise software.

<comment><USER45> Your premise is fatally objective and missing crucial nuance. There is by no means a consensus that Docker containers are "technologically superior" to virtual machines.. just lighter.

<comment><USER9> Not even much lighter really. Compared to paravirt enhanced KVM at least. Maybe even Xen. Especially when you have cleancache and frontswap on.

I would like to see actual facts for the alleged performance benefits. :)

<comment><USER45> Oh I very specifically said "lighter" and not faster, referencing only the distribution size. But not much lighter, even then.

It just makes no sense to me to throw away an actual virtual machine for a mere abstraction of one.

</comment></comment></comment></comment><comment><USER52> It depends on what you mean by Docker. As a container, it's adds a lot of conveniences, despite that at the end of the day it all boils down to cgroups and namespaces.

If you mean as a container orchestrator, I have no idea. Honestly Mesos seems much better than Docker or Kube in this regard. It scales better, is more reliable, and is more flexible. The one downside as I see it is there's a steeper learning curve.</comment>