Linus Torvalds on Why ARM Won't Win the Server Space - <AUTHOR> https://www.realworldtech.com/forum/?threadid=183440&curpostid=183486 <comment><USER4> It's extremely hard to agree with Linus on that. One problem in his argument is that he believes that everybody has a kernel hacker mindset: most today's developers don't care about environment reproducibility at architecture level. The second problem is that he believes that every kind of development is as platform sensitive as kernel hacking, and he even makes the example of Perl scripts. The reality is that one year ago I started the effort to support ARM as a primary architecture for Redis, and all I had to do is to fix the unaligned accesses, that are anyway fixed in ARM64 almost entirely, and almost fixed also in ARM >= v7 if I remember correctly, but for a subset of instructions (double words loads/stores). Other than that, Redis, that happens to be a low level piece of code, just worked on ARM, with all the tests passing and no stability problems at all. I can't relate to what Linus says there. If a low level piece of code written in C, developed for many years without caring about ARM, just worked almost out of the box, I can't see the Ruby or Node application to fail once uploaded to an ARM server.

<comment><USER5> Node and Ruby applications do fail on ARM though, when it comes to native libraries and extensions. And now your whole distro is different than your development machine, which adds complexity.

Do I really want to be debugging why node-gyp fails to compile scrypt on the ARM distro on the new Amazon A1 ARM instance (which it did in my case)? And if I solve that, what about the other 2451 dependencies? Let's pessimistically say there's a 1% failure rate, I'll be stuck doing that forever! Nah, I'll just go back to my comfy x86 instance, life's short and there's much code to write :)

I think I'll side with Linus on this one. I saw first-hand how non-existent the x86 Android market was, despite Intel pouring megabucks into the project. If the developers don't run the same platform, it's not going to happen, no matter how great the cross platform story is in theory. Even if it's as simple as checking a box during upload that "yes, this game can run on X86", a huge chunk of the developers will simply never do that.

<comment><USER5> On a related note, if Apple does switch to ARM chips for their laptops, that will make mainstream ARM server-side development more viable than any cross- platform story ever can. Or kill the Mac desktop. One or the other :)

<comment><USER6> It will most likely kill apple laptops as a developer platform.

Unless you are deeply disconnected from the hardware, CPU architecture does matter. Most developers using macbooks I know have VMs for either Windows or Linux works.

It might be conceivable to use the ARM port of <Insert your Linux distribution here>, or the ARM version of Windows 10. But it would also require a good desktop virtualization solution for ARM. If Apple release its own solution or creates a partnership with let's say VMWare to release something at the same time an ARM macbook comes out, it might work, but barely, and I'm not sure if developers are at the top of Apple's priority list. In the end, with the Linux subsystem in Windows, choosing Windows as a development platform might make more sense.

As a side note, if Apple switches to ARM, I foresee a lot of cringing. The switch from ppc to intel was annoying for a lot of Apple users at the time, but there were a lot of goodies (bootcamp, better autonomy, better performance, VMs) that kind of made it up for it, basically, nothing was lost in term of possibilities, a few were actually gained, and only the transition was a bit painful. With a switch to ARM, you may gain a little in term of autonomy, but with macbook pro already at +8 hours (~a work day), not sure it's a game changer, at best you will stay the same in term of performance, and you will lose in compatibility with other OSes.

<comment><USER7> I think kill is too strong. Certainly some developers will need to be on an Intel chip, but not all. How many developers use their laptops as a dumb SSH terminal? While some C extensions to scripting languages will need some love, the majority of major interpreted or VM driven languages work already.

My feeling is it will be net zero as far as ARM servers are concerned until the hardware is made and is viable. Perhaps Apple ARM laptops will help with marketing ARM as a viable option, but we already develop on OS X in order to deploy to Linux without any great calling for OS X servers.

Cloud server "hardware" has also drifted from what you see in real hardware. There are numerous times in my career I've had to explain to deveopers of all experience levels that their SSD is several orders of magnitude faster than the tiny EBS volume they're reading from and writing to.

In short, I think architecture mismatch just isn't that important to most Web App oriented companies. My girlfriend works at a travel industry giant and they're at the opposite end, putting IBM mainframes to good use. They don't have much use for Macs and most of their developers seem to be on Windows instead of anything resembling what they're deploying on. For the segment of our industry that does care, they'll have options and will choose them regardless of what Apple, Google, and Microsoft do with ARM, Intel, Power, MIPS and other architectures.

<comment><USER6> Maybe it would be interesting to look at the past.

Was the PowerBook as heavily used as a developer laptop as the MacBook is today?

(or Power Mac vs desktop PC as desktops were more common at the time).

I was not in the industry at the time (2000 - 2006) so I don't know the answer.

<comment><USER8> While the Intel switched helped, at least I thought it was great, the big deal was that OS X was a tremendously usable Unix on amazing laptop hardware.

I'm not sure the architecture mattered as much as that did.

<comment><USER9> Agreed. I switched when there were still G4 PPC laptops just because OS X was a usable Unix with good hardware. The switch to Intel was good, but it wasn't because I struggled with the architecture. It was for the more powerful CPUs and battery life.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> It's of course impossible not to respect Linus' opinion and first hand experience in this space, but doesn't this whole post completely ignore the 100 ton blue whale in the room? Namely _smartphones_. That's an entire enormous segment of the industry and it's nearly 100% (or entirely 100%?) literally develop-on-x86-deploy-on-ARM. Smartphones also fit

> _" This isn't rocket science. This isn't some made up story. This is > literally what happened"_

right? I mean, I can see arguing that going up into the cloud is different in some ways then going down to smartphones (although the high end ones are now going to outperform plenty of old dev machines in burst power). There are certainly differences in scaling and such. But the maturity of the tech for cross development of high level software isn't the same as it was in that era either. And if we're talking about bottom-to-top revolutions, embedded and smartphones seem to be at a lower level and much higher volume then PCs.

Finally there is clearly an upcoming disruptive fusion event coming due to wearable displays. When "mobile" and "PC" gets merged, it certainly looks like ARM is in a strongly competitive position for some big players, and having more powerful stuff up the stack will matter to them as well.

None of which is to say he won't be right at least in the short term, but it still is kind of odd to not even see it addressed at all, not even a handwave.

<comment><USER11> Smartphones are a good argument for both views IMHO. Native development (as in native machine code executables) on Android is still a terrible experience even though they had a decade to fix it. It's much better on Apple platforms, maybe because they actually cared about developer-experience and native code is a "first class citizen" there.

It goes beyond the different instruction set of course and most of the time this is indeed mostly irrelevant (unless you've arrived at processor-specific optimizations), but the "develop on the same platform you are running on" still has the least painful workflow IMHO.

I wouldn't mind an ARM-based Mac though ;)

<comment><USER12> I bet Apple's experience moving from PowerPC to X86 gave them a leg up as well, and in both cases (PowerPC/X86, MacOS/iOS) they had the power to force developers to cross-develop to maintain access to their platform. Nobody is in a position to force server-side developers to switch to ARM.

<comment><USER13> Don't forget 680X0 to PowerPC.

</comment></comment></comment></comment><comment><USER14> I disagree with Linus for a couple of reasons. The main one being that not every service in a product needs to be running ARM for it to be useful. There is nothing preventing heterogenous solutions in the cloud and if third parties vet their code on ARM then deploying your DBMS on ARM and your web server on x86 (or whatever services most of your business logic is in) is totally valid for cost or performance reasons.

Secondly, it seems likely that there will be ARM MacBooks by 2020, that kind of instant market penetration for arm in the dev space might mean the exact opposite of what he's saying; why would I deploy on x86 when all of my development is on my ARM machine already?

<comment><USER15> Your second point is just re-enforcing what he said. If MacBooks move ARM, they will have solved developer piece. Since so many developers use Apple stuff, ARM will be native and servers would follow.

In the 90s you could see a shift from people having a SPARC workstation in their office for service development, to just using a x86 PC with Linux or Windows on. Then after while developing for Linux/x86 and deploying to Solaris/SPARC made little sense, so you just put it on Linux/x86 in the end. The thing is maybe 95% of things just work and you spend all your time sorting the other 5%.

<comment><USER16> At least for now, the 'move to ARM' is just wishful thinking by developers who think they can shovel phone apps onto the desktop. Unfortunately phone apps on the desktop stink so that's not going to be a big thing.

<comment><USER17> Moving to ARM has nothing to do with moving phone apps to desktops.

Almost all iOS apps already run natively on x86 today. When developers use the iPhone simulator they are running natively compiled apps linked against an x86 version of the iOS framework. If all Apple wanted to do is allow iOS apps to run without any usability changes on Macs that would be easy (and ugly).

There are already x86 based Android devices.

</comment></comment></comment></comment><comment><USER18> ...except that ARM laptops and desktops are the next wave, as ARM dominates mobile devices and tablets already, it's sneaking up on Linus from behind.

Additionally, Linux is the server platform, abstracting the ISA to a large degree, particularly with regards to what most people consider web applications development, tooling, etc. Docker and Kubernetes serves as this paper for many, as well.

This all will matter in the coming rise of Risc-V devices, as this ISA begins to eat parts of the ARM empire.

Linus would know a thing or two about challenging X86 (transmeta) and this probably shows in his display of his emotional wounds. I don't think his fear holds for all time nor in the coming decade.

<comment><USER19> He is right though from a historical perspective. All he is saying is x86 won over the server space because its where people develop. If the arm development platform becomes a thing, then he will be proven right again.

I have a slightly different take. Arm on the server has a chance now because cloud and thin clients are increasingly common and so even your "at home" machine, as he calls it, could be in the cloud, the same arm machine used for deployment.

</comment></comment><comment><USER20> I feel this is why Ubuntu is one of the most popular server distros.

Ubuntu is pretty unsuited for servers IMO. It's just complex and error-prone, IMO. The packages aren't always server-quality.

But it's great for the desktop IMO, because Canonical actually tests against real hardware for you.

So the fact that people use Ubuntu for desktop/laptop development makes it popular in the cloud, which I've always felt was unfortunate.

You could do some extra work to test your web app on a more minimal distro or on a BSD, but why bother? Ubuntu works to a degree, so you save that step. Same with x86.

<comment><USER21> Can you expand it on this?

I agree that I use Ubuntu on server because I use it on the desktop, that makes sense.

However, when I installed Ubuntu Server 18.04 recently it was _delightful_. There was this simple feature they added which automatically pulled down my ssh key from my GitHub account. During the install it asked me to enable the semi-recent kernel livepatching (and old fashioned unattended-upgrades) it even suggested plexmediaserver as a snap package - which was my goal.

Of course installing actual servers is a bit archaic in the age of docker, but it made me feel like I made the right choice of OS.

<comment><USER20> I guess I'm making an argument around predictability and stability. To be honest I haven't used Ubuntu as a server in several years. Maybe they have improved things.

But I think they can mostly improve things "on top", but not the foundations. Patching over problems by adding layers on top generally isn't great for stability, and is bad for debuggability.

I'm comparing Ubuntu to BSDs, where there's actually a manual, and files are put in consistent places. (as far as I understand, I have less experience with them.)

It's also most of the same reasons that Docker switched its default image from Ubuntu to Alpine some years ago. Alpine is just smaller and makes more sense. Ubuntu does a lot but it's also sprawling and inconsistent.

Off the top of my head:

\- Starting services is a weird mix of init scripts and Upstart. And now they're switching to systemd. When they switch they don't update all the packages. There are some weird compatibility shims.

\- When you apt-get install apache2, it actually starts the daemon. (This is a problem with both Debian and Ubuntu.) This is bad from a security perspective. A lot of people don't know what's running on their systems in practice and what ports are open, and then they have to set up an extra firewall, which is more complexity.

\- the file system is generally a mess, e.g. /etc. There seem to be multiple locations for everything, e.g. bash completion scripts. I think this is a function of the packages being old and patched over.

In general the documentation feels scattered and incomplete. The common practice seems to be googling stuff and pasting things in random files until it works. And then automating that with a Docker container.

That's now traditionally how servers were administered pre-Google :) System administration knowledge / quality seems to have taken a nosedive with the rise of the cloud and cheap hosting. I'm not saying that it's all bad, but it's a downside.

The great thing about Debian and Ubuntu is that there is so much software packaged for it. I think that's generally the reason that people use it. There is a network effect in the package ecosystem.

<comment><USER22> Fwiw Ubuntu server has gotten a lot better than it was. Initially it was more like a little bloated, better tested, sometimes oddly configured "Debian testing" \- now it's more of a proper Debian derived distro, with a fairly decent "server" version.

You should no longer have to expect problems doing an in place upgrade from lts to lts release of Ubuntu (which has been true for Debian stable as long as I can remember).

As for running packaged versions of things like apache and them autostarting... I see your point, but I don't think it's a _weakness_ \- it's more of a difference of opinion.

One thing canonical seem to be doing right (which I initially thought of as a bad case of NIH) is lxd/lxc, juju and zfs integration. I've yet to play with it seriously, but it does come with a lot of shiny stuff out of the box. That said - it appears (light) containers are winning the mindshare - and I can see how an email server as a container/appliance might be preferable to a custom scripted lxc "VM"/image - if you get the upstream supported container, you likely get some help in keeping the stateful data out of the container.

The benefit/problem of lxc/lxd is that you can just keep working with the "VM"s as virtual servers.

Anyway the end result is a lot like modern freebsd jails - in a good way.

</comment></comment></comment></comment><comment><USER23> He is right. I'm still waiting for that ARM-based laptop with awesome battery life, which can run a proper OS with good speed and hence is not just a toy. Apple switching to ARM won't make much difference since I gather it will only run OSX, which will probably be pretty much iOS at that time. Yes, you will probably be able to run Linux on that thing, but almost nobody will actually do it because it makes much more sense to buy a cheaper x86-based machine instead. Yes, there are ARM-based laptops running Windows, and they are a joke. For instance, c't just tested the Lenovo Yoga C630 WOS, and it's a disaster, plain and simple.

<comment><USER24> If Apple makes such hardware, it will be just a matter of time to port Linux to it. Probably easier than getting it to run on yet another weird internal architecture of a smartphone.

And you can run Linux on that Yoga. It's just Windows for ARM that is the disaster...

<comment><USER25> They won't be selling the hardware, and macbooks don't run linux very nicely lately

</comment></comment></comment><comment><USER26> War story time. File based encryption[1] (which used ext4 encryption) was supposed to have been released with Android M. It slipped by a full year because of this issue which Linus identified. At the time, we didn't have xfstests ported over so it could run in an Android/Bionic environment. So the only way I could test it was by taking the SOC kernel to which we had backported ext4 encryption, bashing it hard enough so it would compile on x86 again, all the while cursing the SOC vendor who had added "value added features" to their SOC kernel in a completely broken way such that the kernel would no longer build on x86, let alone boot on it, and then test the ext4 encryption feature backported to the Android kernel on x86 using kvm-xfstests (this was before gce-xfstests[2]). Unfortunately there was an ARM specific bug that we couldn't catch while testing on x86, and it showed up only when we were dogfooding the feature on two testers' phones. We couldn't reliable repro it on our development phones before the ZBB deadline. So we ultimately ended up having the pull the feature and wait a full calendar year for Android N because we could launch it.

As a result, a colleague developed android-xfstests[3] so we could actually do the necessary testing on an ARM platform. If we had this at the time when we were first trying to launch File Based Encryption for Android, it would have avoided a huge amount of hair pulling.

[1] [https://source.android.com/security/encryption/file- based](https://source.android.com/security/encryption/file-based) [2] [https://thunk.org/gce-xfstests](https://thunk.org/gce-xfstests) [3] [https://thunk.org/android-xfstests](https://thunk.org/android-xfstests)

</comment><comment><USER27> I have to strongly disagree with Linus here. While he got the history up to this point right, in the present it is very easy to develop for multiple platforms at once.

CI platforms make it super easy.

Cloudflare made a big effort to make all software run on x86 and aarch64, and it wasn't that hard at all. Most things today work out of the box.

Many things that don't work, were not very robust to begin with, and fixing them to work on both platforms only made the software better as a whole.

Cloudflare could move to ARM servers in a days' notice, and would already do that if Qualcomm hasn't closed the Centriq shop.

The only real advantage I see for Intel today is AVX512, which can't be beat for some workloads, SVE would be able to compete for _some_ workloads, but not all. However most workloads don't use vector processing instructions anyway. Still, I hope ARM improves their SIMD architecture further.

As for the ALU performance difference, it is very easy to bridge. As evident by Apple's CPUs and Qualcomm Centriq simply having four ALU units gets you to about the same ALU performance as Intel.

Branch prediction, cache performance and the interconnect are still much better at Intel processors. However Intel already peaked branch prediction performance, so the difference can only get smaller with time.

<comment><USER28> The CI angle was my first bet as well.

You build and test in the cloud nowadays. Heck, you might develop on MacOS, and deploy on Linux. There are places where it matter, SIMD processing for instance, or low-level kernel development. But if you are on any kind of VM system, or ecosystem with a good compiler history, you are likely to have a smooth ride.

If, in addition, the price point of the ARM machines are at 2/3 of the X86-64 ones, you are in trouble. x86 had another important point up its sleeve: price. And I might expect this was a confounding variable in Linus argument. Not only did you have the x86 at home. It was also like 1/8th of the price of an Alpha, PA-RISC or Sparc machine in price point. And at the time, x86 was dog slow compared to Alpha, and its I/O was laughable compared to HPPA/Sun.

If you have a sizeable invoice at AWS, and you can shave the EC2 price by 2/3rds, you are going to have a lot of wiggle room to make sure your system runs on that aarch64.

</comment></comment><comment><USER29> > That's bullshit. If you develop on x86, then you're going to want to deploy > on x86, because you'll be able to run what you test "at home" (and by "at > home" I don't mean literally in your home, but in your work environment).

I know lots of developers who work on macOS, and deploy to Linux. Back when Mac was PowerPC, I knew lots of developers who worked on PowerPC, and deployed to x86. Or worked on 32-bit, and deployed to 64-bit. Or worked on single-core, and deployed to multi-core. Or green threads to kernel threads. Or big RAM to small RAM. Or the opposite of all of these.

This disaster he anticipates just doesn't seem to be a problem in practice. It's nice when your server is exactly the same as your workstation, but it's never been required. Even today, I have x86-64 on my desk and x86-64 on my server, but they're not the same CPUs, and they don't have the same features (or core count, or RAM, or OS, or ...).

> Which in turn means that cloud providers will end up making more money from > their x86 side, which means that they'll prioritize it, and any ARM > offerings will be secondary and probably relegated to the mindless dregs > (maybe front-end, maybe just static html, that kind of stuff).

Are databases also "mindless dregs"? There's a lot of Postgres/MySQL/Mongo/... instances out there, and even if you're worried about the costs of porting, you only need to port those once (which AFAICT is already done for all of the major ones).

If my database service told me they could cut my price by moving my Postgres instance to ARM, I'd click that button in a heartbeat. I have zero fear that anything would be fishy due to it being a different architecture than my workstation.

> Do you really not understand? This isn't rocket science. This isn't some > made up story. This is literally what happened, and what killed all the RISC > vendors, and made x86 be the undisputed king of the hill of servers, to the > point where everybody else is just a rounding error.

I've worked with many companies, and in every case, going with x86 was 100% because of price. We didn't avoid SPARC or POWER or Itanium servers because of endianness or any other technical reason. It was just a lot more expensive.

Likewise, we switched the servers to 64-bit when it was cost effective, not at the same time that our workstations switched.

</comment><comment><USER30> I disagree with Linus here, in large part because modern system architectures are so distributed. There are so many parts of modern deployments which are not developed in-house anymore. Databases, organizational productivity software, monitoring systems... the list goes on.

So let's say the developers of PostgreSQL or Prometheus or JIRA or Mattermost or any one of numerous others were to come to their audience and say, "we support ARM 100%, you can run our software on ARM in production with no issues, and in fact it's the favored deployment model because it'll cut a significant slice off your operating costs." Does Linus seriously think that, what, customers aren't going to run ARM servers because they can't run the vendor's binary on their development machines?

Maybe there's a chicken-or-the-egg problem here. But just off the bat, PostgreSQL and Prometheus are both supported and built for ARM. There are non- official binaries available for many other projects. Developers are using cheap Raspberry PIs as local desktop development platforms. The more development the ARM ecosystem sees, the easier it'll be for enterprises to justify moving away from x86.

<comment><USER31> > Developers are using cheap Raspberry PIs as local desktop development > platforms.

IMHO this is still a problem, it is easy to get RPi-style devices but for development a more powerful device would be great, however this is much harder to get. Sure, cross-compilation works but is usually tedious to setup and work with.

I've had the chance to work on one of these powerful ARM servers for some time. I was connected via ssh and mostly working with tmux+vim, I could compile natively with all these available cores and memory - development was a breeze.

Nevertheless there were some pain points compared to x86: Often no ARM64 binary packages, except for the packages provided by the distro. No precompiled binaries. Software wasn't working/building e.g. because Fedora on ARM64 uses 64K page size. perf didn't support as many performance counters as on x86. Sometimes I would've really liked to use rr (the reverse debugger) but it only supports x86. Pretty sure I've encountered more pain points that I don't remember and also I think I didn't discover all of them.

Development is almost exclusively done on x86. So it is only natural that x86 is the most tested and optimized architecture. This wouldn't be a problem if your offering would be much better/cheaper in some way and although I think ARM servers can compete in certain areas it doesn't seem to provide enough benefit to be worth the trouble. I agree with Linus that they should be focusing on developers and software and let the machines bubble up into the server segment.

<comment><USER32> There are more powerful ARM SBC boards out there. Take a look at Pine64.org for example. They have a Rock64 board that supports up to 4 gig of RAM with quad core CPUs. They are under $50.

<comment><USER31> I personally would like to see an ARM competitor to the Intel NUC. 4GB and 4 cores sounds great but might not be enough for development or certain benchmarks. Something like 16GB RAM and at least 8 fast cores with a fast SSD should be enough. It seems that you can only get ~50$ devices or >2000$ servers (if they even sell single ones). I guess we have to wait for Apple to sell a MacBook with their own cpu.

</comment></comment></comment></comment><comment><USER33> > and how stupid your argument is

Didn't Linus recently go to sensitivity training or something? I think he missed out on the bigger picture.

It seems like his argument is more about shaming anyone who disagrees with him, instead of expressing an opinion.

<comment><USER34> Calling argument people have stupid is not insensitive or offending (in the love people but hate their ideas sense).

People may be offended if they identify their ideas too much, but that's insensitivity from their part.

</comment></comment><comment><USER35> So RISC-V CPU designers should aim for the developer market then? First win the hearts and minds of developers, then everything else will come.

<comment><USER36> But if there is nothing to develop for... Chicken and egg problem.

However, I think that Windows on ARM will soon become a real thing, this could bring the developers, it will also help the ARM devices spread among Linux users.

Regarding what Linus wrote, I do not buy it. There is so much stuff can run without cross-compilation: js and other scripts, java, c#. Also look at things like Android and iOS, you are already happily developing for ARM, you can develop for servers with the same approach.

<comment><USER37> I do buy Linus' argument, you need to be able to do native development.

My ARM64 development machine is a Pinebook, ARM32 a Cubietruck. Both have only 2GB of RAM and could really do with more. At least the Cubietruck has SATA so mine has a SSD.

<comment><USER38> With rich language runtimes, as app developer I don't care if my application instance runs on top of an OS, hypervisor or just plain bare metal.

The only devs that need to care about the native part are the ones writing the glue between runtime and hardware.

<comment><USER37> Somebody has to maintain the runtimes, they are not typically set up to be cross-compilable, I'm doing JVM development on my ARM systems right now.

<comment><USER38> Apparently you stop reading at the first sentence.

<comment><USER37> I'm not working on "glue between runtime and hardware".

<comment><USER38> If you are maintaining a JVM runtime for ARM or writing JNI stuff then it looks to me you are doing just that.

Otherwise you completely lost me there.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER39> The fact that my local CPU is a different ISA from the server is barely a consideration after I've already had to account for the fact that the database is local instead of remote, that I'm using PCIe-attached storage instead of a network-mounted volume, that my local workload is wildly different from the deployment environment's... Ultimately, cross-development is much more meaningful as a software problem than it is as a hardware problem — but, given the popularity of macOS as a development environment, it seems clear that we're happy developing on something unix-y and deploying to another brand of unix-y something-or-other.

Linus is then ignoring the single biggest factor in this equation: performance-per-watt. ARM offers better performance/watt numbers than anything Intel can provide right now. Recent offerings on the server space suggest ARM is managing to scale up without sacrificing too much on that front, and Intel doesn't seem to be keeping up.

If you're operating a data centre, power supply is the single hardest limit to your capacity to scale up. Putting up another building at a pre-existing site is cheaper and easier than finding a way to pipe more power into it, so it doesn't even matter that much that servers are individually less powerful.

Google is investing in ARM, Facebook is investing in ARM, and I can only imagine several other big players are looking into it too. I wouldn't write it off this easily.

</comment><comment><USER40> Linus is saying basically the same thing I said in HN comments some time ago. This was regarding the Qualcomm Centriq server processors. I asked where I can buy _right now_ a standard ATX motherboard + CPU combo that I could mess around with at home. For any sort of reasonable price. The answer was deafening crickets.

As compared to how easy it is to buy a decent quality $119 motherboard + $179 AMD Ryzen quad core CPU.

Not just that it was literally impossible to buy a motherboard+cpu for any reasonable price, for a midtower desktop PC format. Also near impossible to buy a 'cheap' 1RU server or similar for any sort of reasonable price, as compared to $400 Dell R620 with two-socket, 8-core-per-socket older Xeons I could buy on eBay.

There is a very real chicken-or-egg problem with economies of scale. If the top ten Taiwanese based motherboard manufacturers don't think it will make money to build boards for it, they just won't. Super low quantity of 'evaluation' boards that you have to 'contact a sales rep' to acquire are not going to gain widespread adoption.

</comment><comment><USER41> Wouldn't the potential counterargument be that ARM desktops/laptops could potentially still come about? Probably more likely in the laptop space. I mean, I think about something like the latest raspberry pi, and yeah, it's slow as heck right now but it seems like something like that could gain a lot more ground if it was higher priced but gave you performance like an iPhone X or latest Galaxy. (And btw: from what I've read the iPhone X costs about $400 in parts, and $110 of that is in the OLED screen, and then there's other costs like sensors that you might not need. So let's say you get the cost of that down to $250 and make the price like $500. That seems like a competitive machine) We already saw Apple (successfully) transition from PowerPC to X86; it seems totally plausible that they could do the same thing with ARM at some point down the line, or someone could be a raspbery-pi style machine with a lot more horsepower.

</comment><comment><USER42> Lots of users mucked around with arm boards years ago untill most realised this wasn't something Arm was interested in with no drivers and those that existed not optimized for performance. It was all a hack with open source developers being kicked like footballs from ARM to SOC vendors and back. Interest waned.

A NUC can be had for about $120 that sips power and has full hardware support for storage, pcie, graphics and is seamlessly compatible with the entire software ecosystem.

ARM is more comfortable with vendors ie soc, routers, nas vendors than supporting an open platform with access to optimized drivers and off the shelf parts. Thus the entire mobile ecosystem is closed and tightly controlled. Even early devboard makers like Odroid have moved to an Intel platform for their latest N1 dev board.

<comment><USER43> I've come to the conclusion that neither ARM nor the actual SoC vendors are interested in desktop or server hardware. The majority have canceled their server projects or sold them off.

</comment></comment><comment><USER44> I build Go, Rust, C/C++ code for ARM/ARM64 (and MIPS) and it's completely transparent. 99.9% of the code just works, occasionally I need to worry about some details, but just once in a while.

Go definitely is the easiest, it's built in:

      GOOS=linux GOARCH=arm go build

Rust is almost as easy to cross build (as long as you have the right GCC toolchains installed!):

      cargo build --target aarch64-unknown-linux-musl

Sooner or later, Intel's hegemony on the server CPU market will end and it'll likely be some combo of ARM and maybe RISC-V (not sure about POWER). Cloud vendors like AWS and Google are looking closely at the margins and as the AWS a1.* [Graviton] shows, they're even willing to make their own CPUs if necessary.

<comment><USER45> That's all fine and dandy for the software you build, but then if it happens that the easiest way to get some job done is to extend an existing docker image or using an existing docker image in a multistage build, you might trip over the fact that the authors of those other images didn't care enough about your arch of choice to cross compile their stuff.

<comment><USER44> I've build & run docker on arm and used it to build my own arm based docker containers from scratch, but yeah, 99.9% of containers intel only.

Having said that, docker is perfectly usable on arm, just no one bothers to.

<comment><USER45> Yeah docker was just an example of modern binary dependencies. We tend to forget that kind of pain since many modern languages strongly favour source dependencies (every user of a library ends up compiling it from sources).

</comment></comment></comment></comment><comment><USER46> Native development is interesting to kernel developers much more so than the average server software developer.

Most server code is CPU architecture independent, and largely operating system independent. It's written in Python, Java, JavaScript, C#, PHP. Most of it is also already cross-developed on macOS or Windows, then deployed on Linux. The fact that all 3 run x86 doesn't make this much easier. Increasingly it's also targeting server-specialized hardware that isn't practical to carry with you (8 GPUs won't fit in a laptop, and Google won't even let buy their TPUs)

If CPU-native development is important, that's actually a good sign for ARM servers. Laptops and desktops are increasingly going to be ARM powered over the next few years.

</comment><comment><USER47> Interesting theory. I wonder then what will happen when Macbooks start shipping with >A12 CPUs [1], will we see more ARM based servers then?

[1] [https://en.wikipedia.org/wiki/Apple_A12X](https://en.wikipedia.org/wiki/Apple_A12X)

<comment><USER48> It literally won't matter, as Linus explained. As the infrastructure will be x86. This is what he's saying. Unless a new paradigm emerges on drastically different tech, ARM has too far to go to catch up to create an isomorphic environment where the code is created. A few tenths of a percent of mac developers won't make a jot of difference.

<comment><USER31> Not sure that's what he was saying. I understood that instead of targeting the server-market directly first, he recommends that it would be better to target the developer market first. And if it's popular there, it will naturally bubble up from there into the server market.

Sure it will take more than a few tenth of a percent of mac developers but it won't need total domination like x86 either. But just imagine if Apple really just switches its MacBook (not even the Air or Pro ones) to ARM64 that would already be millions of users. A lot of devs will at least try to compile their app for that platform for the first time. Quite some developers will be curious and get one just to test their software on it. I am pretty sure this is going to help the adoption of ARM64 way more than a slightly improved CPU or another experimental support by a cloud provider.

Personally I don't care too much about ARM64's success (although more competition would certainly be great), I am more rooting for RISC-V and I hope they follow a more developer-first strategy.

</comment></comment></comment><comment><USER49> Linus has been wrong before, but here he is clearly and unambiguously wrong. You don't - you NEVER - develop to the details of an architecture. The only exception is when you're doing OS development. Since things that people want to run in the "cloud" are exceedingly rarely OS development, the idea that you'd care about the architecture is quite silly.

We have vendors ardently trying to get us to commit to their specific environments, like NVIDIA, but they've been dropping bricks on their feet by not actively encouraging software developers to not target specific versions of CUDA. Anyone running TensorFlow knows how much this sucks. After a while, people are going to get fed up with the problems that come from a poor yet popular implementation and are going to start preferring less popular, yet less proprietary, more deployable solutions.

The idea that Intel and AMD can continue to keep the x86 platform performance advantageous forever is pretty ridiculous. We are already seeing Arm CPUs which are significantly better than x86 in performance per watt. How long will it be until we have high end consumer Arm and low end server Arm that completely overlap x86? One year? Two? Three? Possibly four?

x86 isn't over - that's not what's happening here. But it certainly hasn't "won" forever.

<comment><USER50> > You don't - you NEVER - develop to the details of an architecture.

You don't _try_ to develop to the details of an architecture. You even try not to. Then you try to run it on a different architecture, and you find (some of) the places you developed to your specific architecture even though you didn't mean to.

And that's why Linus is right. It's easy to be architecture-specific by accident. It's _really hard_ to not. And it's going to take time and effort to go to a different architecture. In the real world, few people want to waste their time doing that.

</comment></comment><comment><USER51> > Some people think that "the cloud" means that the instruction set doesn't > matter. Develop at home, deploy in the cloud. That's bullshit. If you > develop on x86, then you're going to want to deploy on x86, because you'll > be able to run what you test "at home" (and by "at home" I don't mean > literally in your home, but in your work environment).

But developers can ssh to the cloud, and develop there, right?

<comment><USER52> And he's wrong. In the enterprise, the vast majority of devs are on Windows, for Java for example. That doesn't mean that their deployment target is Windows, most of the time it's Linux. And the development differences, chance of bugs, performance profiles are way different between OSes than they are between CPU architectures.

<comment><USER53> I can install a reasonably performing x86 Linux VM on an x86 Windows host. If the same can be said for an ARM VM on an x86 host, please share how, I'd love to learn that trick.

<comment><USER52> Most of those enterprise devs don't even get to touch the test/staging/prod environment, yet they still do their jobs.

Regarding the VM, today I guess it would be a bit ugly through Qemu but tooling is always solved if there's a need.

<comment><USER53> > Most of those enterprise devs don't even get to touch the test/staging/prod > environment, yet they still do their jobs.

They can usually run at least some part of the stack locally. Maybe not the full thing, at least part of it.

> Regarding the VM, today I guess it would be a bit ugly through Qemu but > tooling is always solved if there's a need.

Tooling doesn't magically fix perf. Have you actually tried that specific setup? Searching online I get results like [https://raspberrypi.stackexchange.com/a/12144](https://raspberrypi.stackexchange.com/a/12144) which aren't impressive, and I don't count as "reasonably performing".

Those results also jive with my experience - the meager demands of mobile apps are outrageously slow inside an ARM emulator running on x86 every time I've tried it, although I'll admit I haven't tried it on Qemu specifically. I can only imagine the horror of trying to run a full ARM server stack on x86 via emulator - perf is bad enough when it's native. For mobile, the first thing I do is fix the x86 builds (so I can use an x86 device emulator), buy a real ARM device out of my own pocket, and/or educate and/or fire my employer for their outrageously wasteful use of my time if they're serious about me and my coworkers using mobile ARM emulators on x86 for any serious length of time (that kind of waste of expensive resources like developer time can't possibly be a sustainable business decision.)

</comment></comment></comment></comment></comment><comment><USER54> ARM is pretty much universally used on mobile these days and also in a fair bit of low end windows and chromebook laptops. With Apple being rumored to launch ARM on desktop and new OSS risc-v chipsets becoming a thing a lot of the arguments that Linus Torvalds is making ring more true looking backwards than forwards. Also WASM is becoming a thing.

I don't think it's as black and white as wanting the same CPU and OS on your deployment platform and development platform. When it comes to mobile development, having those two be completely different is the norm. It's not much of a problem in practice. The differences between different platforms with the same cpu architecture and OS from different vendors is a much bigger headache than the difference between each of those and the windows or mac + intel laptops the software is being developed on. The difference between windows and macs still causes some headaches of course but is mostly managable. If it works on Linux it generally also works on mac. And if it doesn't that's a good sign of immaturity. All I'm saying here is that most mainstream technologies work across all three of those without much headaches.

</comment><comment><USER55> > I can pretty much guarantee that as long as everybody does cross- > development, the platform won't be all that stable.

Even if that's technically true, there's no reason there can't be ARM workstations. There aren't many because they're still generally slower, but there's no fundamental reason (that I know of) why an ARM workstation can't work just as well as an x86 workstation in almost every case.

<comment><USER56> Intel is already anticipating Apple moving to ARM for "desktop". Chromebooks already run a mix.

He'll be right until he is wrong.

<comment><USER57> He'll still be right, even then. His core argument is that until there are ARM computers that developers are working on in some real quantity, ARM in the server space has no chance. If that happens, then by his argument they now have a chance.

<comment> yoz-y It seems that the title gives a different meaning to the words he actually said.

</comment></comment></comment></comment><comment><USER58> Well all the kids are learning programming with ARM-based RPis so...

<comment><USER59> Are all these kids learning C or assembly? Because if they are learning Javascript, Python, etc. it doesn't matter in which processor their code is running.

<comment><USER60> Exactly. That's what I don't get about this whole architecture argument. If I write a mess of Python and deploy on a local ARM SBC or drop it into a remote x86 webserver, what's the difference?

</comment></comment></comment><comment><USER61> I think this ignores the fact that these devs just write some js and push it to some cloud server and it uses APIs for storage on some other server.

The person running the infra for S3 or GitHub needs to think like Linus because they need to worry about that system level compatibility. But if I write an app using say node+s3 I trust that even though I develop on x86 I can run it under node on Arm.

<comment><USER62> I think he tries to address this but fails to acknowledge it elsewhere.

> This is true even if what you mostly do is something ostensibly cross- > platform like just run Perl scripts or whatever. Simply because you'll want > to have as similar an environment as possible,

I guess if you're doing stuff with low-level packages using node-gyp or C++ bindings then sure, but pure node-based workloads don't care about the architecture and, as you said, are expected to work everywhere.

</comment></comment><comment><USER63> I agree with Linus so far, as that the old RISC vendors got killed, because there were only a few, if any, offerings for consumers. Less so because of the code migration issues, but rather, because the volume of processor production was to low to compete with Intel back then.

However, there is an elephant in the room today: smartphones. ARM processors are suddenly a huge market, allowing for higher R&D expenses. And basically everyone already owns an ARM-powered device, most developers are actually developing for ARM today when they build apps.

So, and there I agree with Linus again, what is a bit missing is ARM-based desktop hardware. The best thing ARM could do to push their cloud processors would be to make affordable motherboards with their processors available to hobbyists and enthusiasts. But ARM is coming to the desktop anyway, be it through the ARM-based Windows laptops or that Apple builds an ARM-based Mac. And then the x86-architecture will come under severe pressure.

<comment><USER64> It'd be great if Sun had a Niagara box priced like a PC when that chip was first launched. I bet our software would be running much better on multicore machines today. Maybe languages such as Rust would have been created sooner.

I remember joking that writing code for the Xeon Phi would be useful because that was probably what a Core i15 would look like. Now we have Xeon Platinums with similar core/thread counts, but only on the very high end while 8-thread machines are mainstream with 12 being seemingly the next step.

On ARM we have something interesting with the asymmetric cores, something Intel has only hinted they plan to pursue. Some tasks can go to slower cores that sip power while others may be better suited to beefy cores that can do speculative execution on deep pipelines and that could heat a small house.

</comment></comment><comment><USER65> When reading it, please keep in mind that the HN title is not part of his message or what he actually mentioned.

He is talking about cross-development issues in ARM ecosystem. He has valid points and good suggestions.

</comment><comment><USER42> Its like Linus, Stallman and others create a free public pool where before only restricted private pools existed.

Now 20 years later the private pools have faded into the background and the public pool is a lake but its inhabitants are disconnected from the past, only vaguely aware of the era of private pools and couldn't care less how the public pool came to exist. Their reality takes for granted this huge public pool's existance on which they can build and sail their boats, so Linus's arguments are unlikely to resonate.

Users will gladly wade into a private pool this time by cloud providers and that suits closed vendors like Arm who maintain tight platform control - billions of phone socs with zero open drivers - just fine. It's like every generation has to make the same mistakes and go through the same cycle of pain or it slips generational memory.

</comment><comment><USER66> In the version of history I remember, x86 won because it was 2-5 times cheaper than RISC for the same performance, then in performance per Watt once that became the limiting factor.

All devs had SPARC machines and no intention to move to x86 on the desktop, until after Intel steamrollered the performance contest.

</comment><comment><USER67> linus worked at a 'failed' intel competitor "transmeta", which was trying to make low powered chips. intel/amd are tough competitors.

[https://en.wikipedia.org/wiki/Transmeta](https://en.wikipedia.org/wiki/Transmeta)

<comment> pickle-wizard I remember the Transmeta processor, they were neat. I only ever saw 1 Transmeta PC in the wild. IIRC it was a Sony sub notebook. Of course it was running in x86 mode and running Windows. I got to mess with it a bit and for the time it was a pretty good machine.

<comment><USER68> I saw an insane amount of Transmeta PCs in Japan circa 2005 during the "netbook era" running Windows. It was quite a surprise.

</comment></comment></comment><comment><USER69> He didn't even discuss the every ARM board being a snow flake needing its own BSP, custom bootloader, OS distro etc.

I am fine with x86 being successful - there are good reasons for it. I don't get why there's this ARM push in the PC and servers market. It's not making anything better.

<comment><USER70> > I don't get why there's this ARM push in the PC and servers market. It's not > making anything better.

What push? There's a rumor that Apple might make an ARM-based laptop, and there's a bit of dabbling with ARM-based servers, but there doesn't seem to be any real push for it. More like hedging bets if Intel continues to shit the bed on 10nm. Except AMD's Epyc looks to be the real contingency plan for that now anyway.

I don't see any reason to expect Apple's motivations for using ARM (more control over hardware) would apply to anyone else, and Apple is the only one with an ARM CPU design that is even worth discussing in a laptop anyway.

So... I wouldn't expect the other 90% of the laptop market to follow, much less any of the desktop/workstation market.

<comment><USER69> Push might have been a little strong word - but I was referring to server vendors selling ARM based servers (HPE Moonshot), Amazon making ARM instances available, MS having Windows Server for ARM etc.

Looks like it's just a plan B type thing or throw it out there and see what it does thing.

</comment></comment></comment><comment><USER71> > And the only way that changes is if you end up saying "look, you can deploy > more cheaply on an ARM box, and here's the development box you can do your > work on".

Linus is overestimating the amount of native code being produced.

</comment><comment><USER72> I thought the main reason x86 won was the tie in with Windows PC's and servers. Microsoft took the market. The apps were written for x86. So, most servers are x86.

Intel and PC makers also charged less for hardware. I always wanted one of those RISC systems. They cost upper four to five digits. On the high end, NUMA machines cost around six or more digits. People started building Beowulf clusters out of Intel boxes to knock off digits. Clustering got big. So, it was cheaper on small and large end to run Intel.

Compatibility with dominant system and low cost. Maybe add that they kept optimizing for workloads like gaming on top of that.

</comment><comment><USER73> Well AWS/MS might disagree as they are adding arm servers:

[https://www.datacenterknowledge.com/amazon/aws-launches- clou...](https://www.datacenterknowledge.com/amazon/aws-launches-cloud- instances-powered-custom-arm-chips)

[https://www.datacenterknowledge.com/archives/2017/03/13/arm-...](https://www.datacenterknowledge.com/archives/2017/03/13/arm- servers-can-soon-power-half-of-microsoft-data-center-muscle)

right now these are niche servers (much like the first x86 servers years ago)

</comment><comment><USER74> >If you develop on x86, then you're going to want to deploy on x86, because you'll be able to run what you test "at home" (and by "at home" I don't mean literally in your home, but in your work environment) ... This is true even if what you mostly do is something ostensibly cross-platform like just run perl scripts or whatever. Simply because you'll want to have as similar an environment as possible

Eh, Android's existence is a pretty big counterpoint to that. And you could just run an ARM VM on your desktop if it's that important to you.

</comment><comment><USER75> Except that doesn't quite explain why soo many developers use macbooks (OS/X), yet deploy on Linux.

Personally I think ARM could be leapfrogged by RISC-V, though it will take longer than people predict.

<comment><USER76> I'm curious why you believe that. Aside from the hype and the association with the word "open", I honestly can't think of anything interesting or important about RISC-V.

<comment><USER75> There seem to be a few indications that there is opportunity for change, and RISC-V could make gains, eg

[https://www.nextplatform.com/2019/02/05/the-era-of- general-p...](https://www.nextplatform.com/2019/02/05/the-era-of-general- purpose-computers-is-ending/)

[https://www.sigarch.org/simd-instructions-considered- harmful...](https://www.sigarch.org/simd-instructions-considered-harmful/)

</comment></comment></comment><comment><USER77> AMD-64 won in the server space. At this point, x86 is just a legacy mode that's still in AMD-64 CPUs.

<comment><USER78> This, AMD64 killed Ithanium, and that's why "x86" won the server.

<comment><USER32> x86 was already on plenty of servers before Itanium. 64-bit hardly mattered until servers with >4 gig RAM became common in the mid 2000's.

</comment></comment></comment><comment><USER79> Linus is right that ARM won't win "as long as everybody does cross- development." And that's why ARM will win.

Apple will ship MacBooks on ARM in 2020, which will have comparable performance with superior battery life. Windows-based laptop vendors will switch to ARM to catch up; the desktop market will follow the laptop market, as it already does.

ARM's performance-per-watt advantages already make it a compelling cross- compile target today; when everybody's developing on ARM, it will be a no- brainer.

</comment><comment><USER80> ARM laptop and machines are here. Windows on a RaspberryPI is going to make the ARM processor more appealing to the dev market and once more people buy super cheap ARM processors ARM will show up in servers. People will dev for ARM and want ARM servers. Netflix uses pfsense wich is BSD and it runs like a champ on ARM and netgate even makes an ARM based unit that is for small offices and is amazing. ARM laptops also last for like 25 hours so I think people will be switching.

</comment><comment><USER81> Same reason why Ubuntu became so popular on servers. Good point.

<comment><USER78> No, Ubuntu became popular on the servers because of the immense documentation, and better maintained repositories.

</comment></comment><comment><USER82> The mixing in of reader abuse is really off putting. "do you really not understand?" etc. I know, I know, it's been discussed a thousand times, but still.

<comment><USER83> Yeah, he just sounds like an angry "get off my lawn!" guy. Even if he has good points, he losses you because its all so rude. I don't understand why he has so much anger. He made a huge impact on the tech world, has a family/kids. I hope he made some loot of linux. He needs to learn to meditate or something. Dude, just be happy. Hardly anyone cares about the CPU anymore (I have been writing code for > 20 years, 13 of which was low level C/C++) outside of OSes and drivers. Its like 98% high level, warm and fuzzy, languages now. It's great.

</comment></comment><comment><USER84> For this to happen we would have to use our phones as workstations - <USER2> connect monitor and input devices. That's current mission of Rob Landley with his Toybox project ¹. Years ahead of us, but probably will happen. Makes projects like Librem 5, Pinephone and Pinebook crucial in the long run.

¹ [http://landley.net/toybox/about.html](http://landley.net/toybox/about.html) \- see the Why section

</comment><comment><USER85> Vast majority of "cloud" user develop and deploy code written on languages that have "no idea" what architecture the CPU is running on. Eg, I code on Python/Django. Why would I care even a little bit about what set of instructions the CPU of my EC2 instance has? I don't. Because Python/Django will run exactly the same way on x86 and on ARM.

<comment><USER64> Now imagine some non x86 vendor adds a couple instructions that make Python code miraculously faster. Wouldn't you want that server?

<comment><USER85> Sure. But that's not my point. My point is that the entire argument that Linus has about users willing to test and deploy code on the same arch is flawed. I'd understand and agree with that argument if we we're talking about developing and deploying C (which is what Linus is used to), but that's just not how "the cloud" is used.

</comment></comment></comment><comment><USER86> I think the most important question here isn't whether Arm will win servers.

The assumption would be there, that the cloud market will continue to grow at its current rate and continue to be the valuable market it has been.

What scares intel most is that a new market will emerge for middle and high power compute spread more broadly, and this market will grow explosively faster than data center ever did.

The question is whether these new cores will bring about a new way of structuring the data center, where it is decentralized and spread out more instead of stuck in one building.

A good example is the infrastructure that must go into place to support 5G and self driving cars. Every lamppost will have a server on it.

Arm winning servers becomes irrelevant in that world. If fog / Edge / infrastructure combo blow up the way mobile did, Intel's little data center slice won't matter.

Developers will only adapt when there is a big new market to write software for. Data center by itself not sufficient to motivate the change needed.

</comment><comment><USER87> Correct me if I am wrong, but isn't this whole dev-prod hardware parity argument contradict with the rise of Docker? I don't know much about ARM or CPU architectures in general, but I have never heard companies testing their Dockerized applications with Docker on different CPU architectures just to make sure that the application works as expected; it can be because everything is on x86 already, but at this point Docker is assumed to be a cross-platform dependency that will be the same between different hosts / OS / architectures. What prevents an abstraction layer like this to be used when it comes to ARM, hence making the whole "If you develop on x86, then you're going to want to deploy on x86" argument incorrect?

<comment><USER88> x86 Docker containers will not run on ARM, even if you install Docker for ARM. Docker is an entirely useless layer to support cross-architecture deployment.

Heck, even full-blown VMs (the real ones, that docker partially replaced) are mostly useless, for performance reasons.

<comment><USER87> The question was not specifically around Docker. What I wonder is, why wouldn't there be a seperate layer that abstracts away these platform differences, just like Docker did for operating systems?

<comment><USER89> Like the operating system?

<comment><USER87> Exactly, but I am not well-versed about the challanges and potential difference that might surface in kernel level and since Linus thinks it is not applicable, I was wondering if there is / would be an equivalent in the user space that would allow reproducibility for different architectures, something like architecture independent containers maybe. I would like to believe that we have come to a position where we can consider the application and platform as completely independent entities, which would eliminate these concerns regarding dev/prod parity possibly.

All these questions are mostly out of my ignorance though, I am trying to understand why the architecture change would require a huge effort as long as the underlying stuff is well abstracted.

</comment></comment></comment></comment></comment><comment><USER90> I have a fundamental question. Do people think the move towards 'serverless', this desire for some percentage of developers to not have to worry all that much about the low level infrastructure their code is running on) to die? To accelerate?

It seems somewhat clear that some percentage of developers are interested in a fundamentally higher abstraction than what operating systems have classically provided. In a world where that's true for not just a small few but for most developers, it seems that something like processor architecture could be abstrated away. It's not like it's particularly easy to replicate serverless deployment environments on developer machines today, and yet it persists.

</comment><comment><USER91> Linus is absolutely correct, despite how it _feels_ like ARM should be winning in the datacenter.

If you want to win servers, you basically need to win developers too. So in the ghost world where ARM is dominating on the server, we're all writing code on ARM desktops.

</comment><comment><USER92> Site is down. Anyone's got a mirror/outline?

<comment><USER93> [http://web.archive.org/web/20190222120214/https://www.realwo...](http://web.archive.org/web/20190222120214/https://www.realworldtech.com/forum/?threadid=183440&curpostid=183486)

</comment></comment><comment><USER94> People will run their Java Scripts on whatever architecture AWS provides for them. Do Linus really think the bulk of developers, the line of business developing kind, cares or even knows what CPU architecture is running beneath it all?

</comment><comment><USER95> The only disagreement I have with his statements is that I think the Itanic and blind vendor buy-in is what sunk non-x86 processors. amd64 is what actually brought x86 ubiquity while ia64 never reached any relative market share.

</comment><comment><USER96> People developing on tablets (it's real, quote [0]: "Since 2015 I do most of my software development on an Android tablet. I am a bit fed up with PCs and notebooks, as they are tedious to handle and have too many mechanical parts. A tablet is more convenient, it can be held in one hand, and usually survives if coffee is spilt over it."), have ARM as their development environment, so if the Linus' argument is true, ARM has chances.

0 - [https://picolisp.com/wiki/?TermuxPentiPicoLisp](https://picolisp.com/wiki/?TermuxPentiPicoLisp)

</comment><comment><USER97> For a time Intel had PCs on a stick – it was like a large flash thumb drive, with a Celeron or something. (Maybe they still offer these.)

Are there any thumb drive ARM computers? (Preferably with 64-bit ARMv8 processors to match emerging server platforms.) With pluggable ARM mini- computers it might be easy to develop for ARM.

Raspberry Pi would be interesting, but they have old 32-bit processors and I'm not sure how well they interface with a normal computer. Servers are using 64-bit ARMv8.

Maybe smartphones would work as pluggable mini-computers? I heard that OnePlus devices were explicitly root-friendly, but I've never tried it.

</comment><comment><USER98> I am in the mindset that arm servers are the future. I have already some sites on arm. The issues with IME give a bad image of Intel. Monopolys are never good... But when Linus says: "And the only way that changes is if you end up saying "look, you can deploy more cheaply on an ARM box, and here's the development box you can do your work on"."

It matches my mindset. I am also looking for the good arm laptop. I do not need anything fancy: 4GB of ram (or better), 64GB SSD, wifi, bluetooth, usb (3 if possible), audio jack, good screen, usable keyboard, good touchpad.

</comment><comment><USER99> Why do i feel he is dead wrong, the stability of a system in dependent on Kernel Writers more than the system builders. ARM would make it for a very important reason the electric bill, more and more Data-centers are geared towards saving on their electric bills to be highly profitable, many are already using renewable sources like Apple and Google. Also if he isnt willing to support ARM for Server-space some other kernel will make it. Also most X86 host these days run in Virtualised environment, there is no difference if it actually runs on x86 or ARM.

</comment><comment><USER100> Companies care more about price than replicability.

Almost all the processor market is ARM already. Scale wins.

ARM desktops are likely if not inevitable.

X86 is not one unified platform.

Cloud hardware will become specialised and not like the hardware in your desktop.

</comment><comment><USER101> > Why? Same exact reason, just on the software side. In both cases. Where did > you find developers? You found them on Windows and on Linux, because that's > what developers had access to. When those workloads grew up to be "real" > workloads, they continued to be run on Windows and Linux, they weren't moved > over to Unix platforms even if that would have been fairly easy in the Linux > case.

There are lots of developers who developing on MacOS so why BSD server usage is very low than?

</comment><comment><USER102> Lest we forget, ARM has problems with instruction ordering not happening the way programmers might assume in certain cases.

[https://preshing.com/20121019/this-is-why-they-call-it-a- wea...](https://preshing.com/20121019/this-is-why-they-call-it-a-weakly- ordered-cpu)

[https://news.ycombinator.com/item?id=4673458](https://news.ycombinator.com/item?id=4673458)

<comment><USER103> ARM doesn't "have a problem" with the ordering. Rather, those CPUs take advantage of the re-ordering allowed by the specified ordering constraints. The behaviour is entirely expected. The default option for those c++ atomic operations is the strongest constraint (memory_order_seq_cst) - a programmer who specifies a more relaxed constraint better have a good reason for it.

<comment><USER24> Usually the problem is that it exposes latent software bugs hidden by x86 strong guarantees on memory access ordering and cache validity. Not that someone misused atomics, butt rather does not use them at all and "it works".

</comment></comment></comment><comment><USER104> The vast majority developers I know of who are targeting the cloud, do so using MacOS or Windows and then deploy to servers running Linux. It is clearly not the case you need the same environment "at home", as you have in the data center.

I do agree with the idea that a cross-compiled environment makew your target environment "more distant" and this is less efficient. However I don't think that's at all what's limiting ARM from cracking the server market.

</comment><comment><USER105> In early 2000 Java promised to be write once, everywhere. By the time it was between Windows and Sun.

20 years later we have finally reached this point for business applications, be them in Java, Python, PHP or some other VM based SDK.

The mainstream development future is in Lambdas and other such containerised technologies. If you can run Docker or similar "home" you can also deploy. Our DevOps models have radically changed and not sure if he realises this.

<comment><USER38> When I left C++ for managed languages, it was when the OS became irrelevant to me, with FFI being the only occasions where it still matters.

As you mention lambdas take it to the next level, and in such future Linux might not even matter anymore, hence his point of view.

<comment><USER106> When I left C++ for managed languages, operating systems and C didn't stop being relevant; what stopped being relevant for me was the grotesque extensions of C dialect desperately trying to be a high level, managed language in increasingly irrational ways.

<comment><USER38> C++ is still the best tool when my managed languages need some extra native help.

Maybe it will be eventually replaced by Rust one day on my toolbox, but surely not for C.

I left that in 1993, only using it instead of C++ when university professors required me to do so, and on my first job (until they also moved into C++/C#).

</comment></comment></comment></comment><comment><USER107> However this does not apply to consumer product, e.g. iOS. When building iOS app for simulator, it is running x86_64 natively in the macOS. When building iOS app for device, it is compiled into ARMv64. Cross-compilation happens every day of iOS development. And the process is 99.9% problem-free. The 0.1% problem comes from missing physical feature (e.g. touch screen / gyroscope) rather than instruction set / system issue.

</comment><comment><USER108> Flashback to 1999: Corel Netwinder tried to bring Linux to an ARM developer box. Still have one in my drawer. I guess we're still working on it...

</comment><comment><USER109> Linus' post reminds me of the Slashdot thread where people were saying the iPod would never catch on.

Saying there's no ARM "at home" is closed-minded to say the least - it's already here - clusters of ARM boards are everywhere, laptops are rapidly gaining ground and I'm currently buying the parts for my first desktop-class ARM machine.

</comment><comment><USER78> This will be Linus "640kb ought to be enough for everyone", except that Gates didn't actually said that, and Linus actually wrote this.

Someone please tell him what LLVM bytecode is, probably he doesn't know because he uses GCC, Apple showed recently how that can leveredged in scale, with Apple Watch's architecture change overnight.

ARM and RISCV are here.

<comment><USER110> LLVM bytecode is very cool, and arguably evidence that a more generic system could be successful, but it is not considered to be instruction-set agnostic. Yes, it allowed Apple to tweak the cpu in new Apple Watches without requiring updated app submissions, which is great. But it would not support a transition from, for example, x86 to ARM. I've seen this topic discussed multiple times, best summary I could find is from a quora post:

 _Everyone 's first thought (including the other two answerers here!) was that this would allow Apple to change processors entirely: if they wanted to make an Intel iPhone, for instance, they could just tell the App Store to start converting apps' bitcode to Intel machine code instead of ARM machine code for the new phones, and every app would instantly be "updated" for the new processor._

 _But in practice, this doesn 't seem likely. Even with LLVM's unusually clean separation between frontend and backend, the frontend actually does know certain things about the processor it's targeting (mainly details about its memory layout, or special processor features it can use to make certain code faster). These things get baked into the bitcode, and in practice ensure you can't really mix-and-match frontends and backends as freely as you would like. So suddenly switching processors entirely probably isn't in the cards._

 _What 's more likely is that it will allow Apple to make smaller improvements to their processors and then roll them out to existing apps. ..._

</comment></comment><comment><USER111> > as long as everybody does cross-development, the platform won't be all that > stable.

I tend to agree that it will suffer. But you can get ARM chromebooks, a bunch of vendors offer ARM-based Windows laptops, and Apple is planning on leveraging their SoC on OS X computers. So it may not necessarily be cross- development for long.

</comment><comment><USER112> For massive scale SaaS and non-native-code PaaS hosts, cost and perceived Intel lock-in risk will be the factor that drives them to deploy and evaluate these servers side by side with their Intel hosts. And, yes, those devs will have access to this hardware in their offices.

</comment><comment><USER113> This is why if Apple ever did a legitimate server offering with virtualization they would kill it. How many devs, develop on Mac and deploy to linux? And also now use VMs for containers. Imagine if MacOS supported containers natively. They leave so much money on the table.

<comment><USER114> Very unlikely. "My way or the highway" is ingrained in their corporate DNA. Can't think of an attitude less suited for server side work where optimization to custom workloads makes massive cost difference

</comment></comment><comment> yoz-y Besides talk about ARM Macs, there are already ARM Windows and Linux laptops here.

Also there is no need for anybody to "win" anything. ARM and Intel servers could be just used for different purposes and each find their own niche.

</comment><comment><USER115> Why is he still coding in Perl? Am I missing something? Shouldn't we upgrade to python or something? Do I have to learn Perl and code x86 software on a 15' CRT monitor while running Slackware to get it?

</comment><comment><USER116> I remember SteveY explaining that Jeff Bezos has an enourmous advantage _as a CEO_ because he had been at the top for so long that he had points of view no one else got.

Torvalds seems to me in a similar position

</comment><comment><USER117> But if you run your dev environment in the same cloud, doesn't it void this argument almost completely?

The onlything i would run locally would be the frontend, and it is web/mobile anyway.

</comment><comment><USER118> As mentioned in TFA... every time I've looked at ARM server offerings, it was always much more expensive than x86 options, and most of the time far less compute power.

</comment><comment><USER119> The corollary of what Linus says may be true for newcomers as well.

If you're selling ARM/RISC-V laptops, developers will not care because their prod apps are deployed on x86....

<comment><USER120> typo - devastated -> developer

</comment></comment><comment><USER121> I just read something this morning that Apple is looking at switching Macs to ARM. So, if that happens, an ARM server will match what at least some developers are using.

</comment><comment><USER122> > By: Linus Torvalds February 21, _2019_

> This is true even if what you mostly do is something ostensibly cross- > platform like just run perl scripts or whatever.

Please tell me this is irony.

</comment><comment><USER123> My phone is more powerful than my previous decade server.

If smartphones came with display ports, many more people could have a PC experience without having a PC.

</comment><comment><USER124> Modern DevOps and CICD should be able to abstract this away. Devs can develop on whatever they want and then push to ARM or x86 servers.

<comment><USER125> That's assuming you're not using any libraries that contain hand-optimized SIMD where one platform is better supported than the other.

<comment><USER124> Well, if [https://github.com/xianyi/OpenBLAS](https://github.com/xianyi/OpenBLAS) is any indication, you can just recompile the library for ARM. A proper pipeline would recompile everything on the appropriate platform without any developer intervention.

</comment></comment></comment><comment><USER126> If developers switches to ARM/RISC in their laptops then arm will win. Arm is also power efficient and there is global warming.

</comment><comment><USER25> Honest question: why is it not viable to run an ARM virtual machine for your development purposes - is it not accurate enough?

<comment><USER127> Not fast enough. Performance is king during development.

</comment></comment><comment><USER128> Linus will have major egg on his face when devs are issued 7nm ARM Macs that blow anything x86-based out of the water.

</comment><comment><USER129> Linus used to believe that his former employer would defeat Intel. He was wrong. And now he is wrong again.

</comment><comment><USER130> Translated from Linus speak: ARM servers will gain adoption if there are more ARM laptops.

</comment><comment><USER131> Please note that Linus is talking about SERVERS, not mobiles

</comment><comment><USER132> His argument doesn't hold up to iOS and Android development.

</comment><comment><USER133> Does someone know why ARM servers try to behave like intel servers.

ARM's forte is in tons of lightweight cores, compared to intel's small number of powerfull cores. There are tons of small mom n pop websites that have short burst of usages, followed by long bursts of calm. So why not give every site on a big server it's own small core, say for 60 seconds? After that you shut the core down, zapping all caches etc...

Things like meltdown and spectre simply disappear if the attacking code runs on a different core, so such an architecture would be good at isolating security sensitive data. In these GDPR times, it seems a good match for small sites, and one that Intel can't provide with their good but power-hungry cores.

</comment><comment><USER134> Ok. Perhaps ARM won't. But what about RISC-V? Aren't RISC-V laptops and servers going to start conquering the markets soon?

<comment><USER135> I really encourage you to read his entire post. There's a critical part of it where he says:

> And the only way that changes is if you end up saying "look, you can deploy > more cheaply on an ARM box, and here's the development box you can do your > work on".

So sure, if RISC-V laptops become inexpensive and accessible for developers so they can develop and target the same platforms, absolutely, it _could_ take over.

<comment><USER134> I've read it. And as far as I knos RISC-V systems are meant to be inexpensive.

</comment></comment></comment><comment><USER136> > Some people think that "the cloud" means that the instruction set doesn't > matter. Develop at home, deploy in the cloud.

> That's bullshit. If you develop on x86, then you're going to want to deploy > on x86, because you'll be able to run what you test "at home" (and by "at > home" I don't mean literally in your home, but in your work environment).

This is why having a compilation target that works the same way _everywhere_ would be so valuable. We're some ways away from this, but I think WebAssembly offers hope here.

As for the rest of the article, it'd be best if we could have a discussion style where people don't preemptively paint people who disagree with them, or who have a different perspective, strategy, or end goal, as "idiotic" and "stupid", or posit that people have a "total disconnect to reality." This is known as poisoning the well. It does not advance the discussion. In fact it is specifically designed to limit and stop the discussion. The reality is that it simply makes people angry and polarized, amps up the stakes, and ultimately leads to a toxic culture.

Thankfully, I think HN has a better culture. Linus can do better, IMO.

<comment><USER137> I disagree. Let passionate people on all sides have their say. I think it's more toxic to stifle passionate people.

It's amazing how everyone from Steve Jobs to Bill Gates to Linus Torvalds is labeled as "toxic" and yet the "toxic" environment they created led to substantial advancements.

And poisoning the well ( or any ad hominem derivatives ) doesn't stop discussion, it generally leads to more discussions - though often times more contentious and off topic. And though I agree that it can make people angry, polarized and amp up the stakes, those aren't necessarily bad things. Most of the time, it is actually a good thing and a basis for competition.

Finally, I'd say HN has a different culture, not necessarily better. Also, what you are doing could be viewed as a form of shaming and virtue signaling. And at the end of the day, if you don't like linus's style of communication, you don't have to read or listen to it.

I don't understand the mentality of "I don't like it so you should change".

<comment><USER136> > It's amazing how everyone from Steve Jobs to Bill Gates to Linus Torvalds is > labeled as "toxic"

To be clear, there is a difference between toxic people and toxic behaviors. The former, I think, doesn't exist. There are people who often engage in toxic behavior, and those who do rarely. Pointing out toxic behavior is the first step to correcting it. And correcting it is in fact the goal of community guidelines, in order to establish a more inclusive culture.

> Let passionate people on all sides have their say. I think it's more toxic > to stifle passionate people.

Let's be clear about what "toxic" means, and not let it degenerate to "loud and I don't like it." Toxic means that it actively damages open discussion, drives people away, and kills off conversations. It is the same sense as a toxic substance; kills.

> And poisoning the well ( or any ad hominem derivatives ) doesn't stop > discussion, it generally leads to more discussions

Ok this is manifestly untrue. Please read ([https://en.wikipedia.org/wiki/Poisoning_the_well](https://en.wikipedia.org/wiki/Poisoning_the_well)). Poisoning the well is a form of preemptive insult to discourage opponents from taking a position by making it seem toxic. The fallacy is aptly named.

> what you are doing could be viewed as a form of shaming

Yep! I am happy to shame toxic behavior when it is clear.

> and virtue signaling

Maybe. I'd be happy to do it anonymously if you would prefer.

> And at the end of the day, if you don't like linus's style of communication, > you don't have to read or listen to it.

This is the very definition of suppressing conversation: sending people away who don't feel like putting up with insults. It's counterproductive and unnecessary.

> I don't understand the mentality of "I don't like it so you should change".

This isn't some he-said she-said situation. I am pointing out _direct unprovoked insults_ designed to stifle discussion and establish a particular point of view. As I have made it abundantly clear, I find this completely unnecessary and I pointed it out because I really think we can stop doing this if we're just consistent about it. We'll have better discussions with more diverse viewpoints, not just hotheads shouting at each other.

<comment><USER137> When we are talking about toxic, I think we all understood it meant behavior. And the point of community guidelines isn't to make it the community more inclusive, it's actually to make it more exclusive. Limiting speech and thoughts are exclusive behaviors, not inclusive ones. But I'm all for HN or any other platform having guidelines.

I studied philosophy in college so I don't need an explanation of what poisoning the well is. "Poisoning the well" itself is poisoning the well and I don't want to get into the intricacies of ad hominems and logical fallacies. Many times, people misunderstand logical fallacies and use logical fallacies themselves to stifle debate.

Also, Linus wasn't having an argument or a debate. He was giving his opinion. He is allowed to say someone's argument is stupid : "how stupid your argument is.". He didn't call people stupid, he called the argument stupid.

Finally, ad hominems may or may not stifle discussion from the passionless or people who don't care about the topic, but it never stifles discussion from passionate people or people who care about a topic. Every major debate - <USER3> back to religious debates or debates about science or debates about slavery or debates about civil rights or anything else was "passionate". Can you imagine these debates being shut down because that's not what "polite company discusses"?

And why would it matter whether you virtue signal anonymously or not? You are already anonymous as HN is thankfully an anonymous forum. One thing HN is fairly good about ( as far as I know ) is anonymity.

I don't believe in ad hominems or attacking people. But if people want to use harsh language to express ideas they are passionate about, I say go for it. The same goes for you. You seem passionate about the subject and I support your right to express it in whatever manner you choose. What I find ironic is that under the aegis of "inclusivity and encouraging discussion", you are advocating for exclusion and stifling Linux Torvalds' speech. But as they say, the road to hell is paved with good intentions. Certainly, you can see that you are doing precisely what you claim Torvalds is doing - stifling speech ( or at least advocating for it ).

<comment><USER136> Well, we are going to disagree about this, so I won't drag this on forever. However I would like to be clear about a couple things.

> And the point of community guidelines isn't to make it the community more > inclusive, it's actually to make it more exclusive.

Again, mixing up behaviors and people. The _whole point_ of community guidelines is to serve the community--to include and support a wide range of _people_ , not a wide range behaviors. Certain behaviors just flat out drive people away--that is the very definition of exclusiveness. Being "inclusive" of these toxic behaviors leads to an exclusive culture. The worst, most exclusive cultures are the ones without guidelines, full of bad behavior. Inclusiveness requires _curation_ of behavioral guidelines. Let's not invert the sense of words when convenient for argumentation (e.g. calling community guidelines and behavioral standards "exclusiveness" because they discourage one type of bad behavior but encourage hundreds of other good ones).

> What I find ironic is that under the aegis of "inclusivity and encouraging > discussion", you are advocating for exclusion and stifling Linux Torvalds' > speech.

This seems to be the crux of the issue. First, it's an exaggeration to say that advocating against using insults and inflammatory language is "stifling" (see above). I actually want Linus to speak his mind--just do so without the anger channel. It's _really fucking annoying_ to some people. Even calling _ideas_ stupid is _really fucking annoying_ to the people who have those ideas. But the worst part is, for every Linus there is, there are dozens, maybe hundreds of people who are going to read something like that and just silently _leave_. That's a sign of a bad culture. That's toxic right there. And those people who leave are the meek ones who normally wouldn't speak up because they don't want to get the firehose and spotlight pointed right at them. They don't want their ideas called stupid or idiotic or be told they aren't dealing with reality. Those kind of people that actually can be very bright and have _very different_ (and valuable!) perspectives. The kind of people who just disappear and you never notice. And I've met plenty of people like this--if you ever people-managed, you find out hey, this or that person is leaving, and it's because they actually really didn't like being around this group. It's a loss. Most people just don't notice, but their community just got a little worse each time that happens. So you gotta find soft ways to stop it.

At any rate. Generally your comments can be construed as a defense of people that really don't need any defense. That makes it even worse when the community _explicitly stands up and defends_ loud, obnoxious, unnecessary behavior and lionizes these "hotheads". (To be clear, I am not suggesting you are explicitly doing that, it just has that ring to it). Trust me, hotheads need no defense. They need no coddling or encouragement to keep mouthing off. Many hotheads will stick around and annihilate a community, perhaps unconsciously, because it works. They win. So don't defend them. Defending bad behavior is a death spiral, as it sends exactly the wrong message about inclusion, and that's double bad.

<comment><USER137> It's semantics to say community guidelines serve to support a wide range of people and not behaviors. It has the same effect. It will drive away people who do not fit your definition of "good" behavior. It's no different than homophobic policies that said we don't discriminate against gays, we discriminate against "bad behavior". And certain behaviors can drive away people, but censorship definitely does.

And I have to disagree with you about the worst communities. The worst communities are those with too much guidelines. Of course some guidelines are necessary, but mostly those involving harrassment, not speech. The US is based on the idea of less guidelines. North Korea, China, Russia, etc are based on "lots of guidelines". And the death spiral can go boths ways. Just as much to totalitarianism as it can go to anarcy.

Also, it's not an exaggeration that "insults and inflammatory language" is stifling. It's the definition of it. Anything can be considered insulting to anyone. That's why we have principled understanding of free speech. The basis of free speech is that you have the right to offend. Otherwise, you claiming that the earth is round is offensive and inflammatory to a flat earther and grounds for censorship.

You seem to think that just because I think someone should be allowed to say "an argument is stupid" is me advocating for anarchy or harrassment. I'm not. Also, "insults" aren't that insulting to everyone. Language that you find offensive isn't offensive to me. And I don't consider Linux's language to be offensive. But you do. But that's the point isn't it. Everyone has a diverse upbringing and diverse opinions.

Also I'd stay away from the term "bad behavior" because that's the same terminology the chinese government uses to crack down on its own citizens and oppress them. It's rather paternalistic and authoritarian which reminds people of the worse form of nanny states.

And last thing, why do you care how linux speaks. He is his own individual. Does he come to you and tell you how you should speak? I just find it the high of arrogance that you ( a relative nobody just like me ) has the gall to tell someone like Linux how he should speak or behave. If you don't like it, just don't read what he says. That's what I find frustrating. Why do you feel like you get to tell others how to live their lives?

At the end of the day, the people at HN can do what they want because it's their property. Regular users like you and I won't change anything. Just like we aren't going to change Linux or the platform he uses to express his opinions. I wish I could have changed your mind but I think I failed so I'll just end it here too. I just find it strange that anyone on hacker news would be demanding that linus torvalds or anyone for that matter be censored.

</comment></comment></comment></comment></comment></comment><comment><USER138> With serverless, language VM's and other abstractions, I suggest this argument is going away.

People are not generally writing platform specific anything these days, this would be an exception not the rule.

That said, there are a lot of things that need to be done under the hood to facilitate 'our view on the world on ARM' ... and it might take time.

Once AWS starts to release a bunch of stuff on ARM - and the stacks we love and need are solid on them ... I suggest that could be the ARM wave that changes serverland.

</comment><comment><USER139> While it's not strictly the case, I'll tend to agree that I have a big productivity advantage by sharing an architecture between my workstation and my server.

I want a RISC-V workstation, and if the counterpart on the server side existed, that'd probably be the change I'd make (if any).

</comment><comment><USER140> I always thought the RaspberryPi was the exact device Linus claims doesn't exist.

</comment><comment><USER141> Man, he STILL, sounds like an angry teenager....

<comment><USER142> He doesn't use any injure, but still doesn't know how to answer without being agressive.</comment></comment>