SSD myths and legends: write endurance - <AUTHOR> http://www.storagesearch.com/ssdmyths-endurance.html <comment><USER2> In the middle of reading this, I saw a 2TB SSD Advertisement then realized SSD just consumable product, it is just like tires which just change it every 2-3 years, for safety concern.

<comment><USER3> Thats the right way of thinking about it but its not really feasible for consumers.

"Oh hey, this Laptop has a SSD drive which is super fast. You will have to replace it every year though and if you dont have a backup strategy you might lose all you data. Did i mention they are quite expensive ?"

<comment><USER4> You will have to make quite the effort to burn trough a consumer SSD in machine with normal use for a year. 256 GB * 3000 gives a lot of write information.

<comment><USER3> In theory yes, in practice i have been using SSDs since 2009 and had them both failing (different brands and tech) after about 2 years of daily usage. Now i see i probably use them more than the average consumer, but i cant remember the last time i had a normal HDD fail in one of my machines.

<comment><USER5> I've seen a number of normal HDD failures. I think that the first one that I've seen, been with a double-5 inch slot hard drive, which was failing to start spinning at times. You've had to turn the PC box abruptly, to make it spin ;)

But as of now, have to see an SSD fail. Longest running that I have now is a Linux box that've been up since Oct/2007\. (Heh, cool part about that linux box - there are no moving parts at all - no coolers, no nothing. Even in the power supply ;) )

<comment><USER3> Sounds cool :) I am not saying i have never seen HDDs fail, but in the almost 20 years i have been computing heavily, it might have been about 3 in my own machines, while i went through 2 SSDs in 4 years. Id still never go back to HDDs as my system drive :)

</comment></comment></comment></comment></comment></comment><comment><USER6> For those in the know, and for the benefit of laymen who don't have the time to do the research, what are the things that a buyer should look for in a SSD, in regards to reliability and longevity? Technology, brand, controller?

<comment><USER7> the only critical thing is physical size. some (mainly ultralight) laptops take drives that are thinner than normal laptop drives (this is also an issue if you're buying a new spinning disk drive, of course).

apart from that, assuming you buy a popular, current model, it will just work (and be reliable). this article is bikeshedding for hardware and enterprise geeks. so, just like anything else, get something reasonably priced from a reputable make (longevity of support, availability). currently samsung seem pretty good.

and finally, if you don't already use one, get one. it's by far the best way to improve "perceived speed" of your computer (well, assuming you already have enough memory).

<comment><USER8> My Intel X?? came as a superslim 2.5 package, but with a plastic bumper as part of the construction making it up to full height. Having to disassemble it first thing to remove the buffer went just fine, but wasn't a great start (warranty voided before I even powered it up?)

<comment><USER9> Had to remove enclosure of a Kingston drive to fit a notebook also. Just cycled all the storage in the desktop before voiding my warranty... Still sucks

In your case it's double ironic since Intel is the one that pushed for slim ssd because of the ultrabook shenanigans while everyone was doing a standard size. Not that having smaller drivers is bad...

</comment></comment></comment></comment><comment><USER10> Write endurance is very real.

[http://www.ssdaddict.com/ss/Endurance_cr_20130122.png](http://www.ssdaddict.com/ss/Endurance_cr_20130122.png)

[http://www.xtremesystems.org/forums/showthread.php?271063-SS...](http://www.xtremesystems.org/forums/showthread.php?271063-SSD- Write-Endurance-25nm-Vs-34nm)

<comment><USER11> The data given on the second link does not inspire confidence in consumer SSDs

<comment><USER12> How so? The tests are in no way normal usage, most writing hundreds of terabytes or more before failure, exactly in the expected range for those sizes of SSDs.

Consumer devices do not write remotely as much as people seem to think they do. Heck, I ran a very large, endless-churn financial system on a FusionIO platform, and two years in it had barely written several hundred terabytes.

<comment><USER11> I was looking at SSDs as scratch disks and the like, but some of the examples failed in months. Its an electronic device, sorry but other than purposely putting in an environment that is not conducive to its long term health, like heat or dunking it, I should be able to read and write to it to my heart's content.

I fully understand commercial grade equipment versus consumer, but in the world of no moving parts it begs the question, what does it take to make them more reliable. Because some of the devices didn't last long enough for me to lose the receipt

</comment></comment></comment></comment><comment><USER13> If anyone's interested in some more details for write endurance I'd recommend Anantechs SSD reviews. [http://www.anandtech.com/show/7173/samsung- ssd-840-evo-revie...](http://www.anandtech.com/show/7173/samsung-ssd-840-evo- review-120gb-250gb-500gb-750gb-1tb-models-tested) .

If the numbers in those are accurate it looks like write endurance shouldn't be a problem at least for any consumer drive. The estimate on that review was 12 years in a typical consumer setup, and I'd reckon that almost everyone would have replaced a laptop or PC in that kind of timeframe.

</comment><comment><USER14> Can someone explain the discrepancy between

> I still hear stories of users thinking it's perfectly normal and economic to > replace burned out Intel SSDs every 6 to 12 months

and

> How long have you got before the disk is trashed? ... 51 years!

<comment><USER15> There's a big difference between using consumer grade MLC flash and running a very high intensity workload and using enterprise grade SLC flash and only using 80 MB/s of bandwidth.

As a consumer you really don't have to worry about write endurance, though. I mean, how much do you write to your drive every day, maybe 5 GB at most? Even with MLC flash and a fairly small 120 GB drive that'll last you for a couple of hundred years. And even when it fails you can still read the data out, you just can't write new data. The problem you should worry about is a firmware bug in the part of the drive that handles the wear leveling abstraction. I haven't seen one of those in a while, but when many companies were just starting to develop SSDs they occurred fairly frequently, though usually the manufacturer would figure out the problem and patch the firmware for new drives after enough failures in the field.

<comment><AUTHOR> But, is a hybrid approach the best option? I mean an SSD for the software and an HD for the data?

<comment><USER15> I actually think the best approach, longterm, would be to use the SSD as a cache for the HD at a filesystem level. I believe something along those lines was just merged into the Linux kernel.

<comment><USER16> Yep, dm-cache essentially does what Intel Smart Response does except it's done in the OS and not the BIOS.

</comment></comment></comment></comment></comment><comment><USER17> Just a quick note: before you buy a "self-encrypting drive" (SED SSD), do your research and make sure you get one where you can control the keys.

A couple of months ago, I spent almost $400 on a Crucial M500 480GB SSD to put in a new laptop only to discover that apparently the drive itself handles the generation of keys.

Thus, _technically_ all of the data on the drive is encrypted but you can take the SSD out, put it into another machine, and access the data without any issues. Total marketing ploy.

<comment><USER18> The point behind it is you can delete the key in one place and not have to worry about erasing every part of flash in the machine (which, after remapping, may not even be possible through normal means).

</comment></comment><comment><USER19> What are these RAM SSDs of which the article speaks? I searched around for a bit, but couldn't find any place to order one (keep getting hits for PCs for sale)... are they available in say, 64GB in SATA?

<comment><USER20> He's referring to Ram disks in the article. ([http://en.wikipedia.org/wiki/RAM_disk](http://en.wikipedia.org/wiki/RAM_disk)).

The only result I found to buy was this software from AMD (to partition some of your RAM as a RAM disk). [http://www.newegg.com/Product/Product.aspx?Item=N82E16832385...](http://www.newegg.com/Product/Product.aspx?Item=N82E16832385001).

<comment><USER21> ImDisk[1] is free without restrictions and also open source. I just set it as a small system startup script[2] to create a ramdrive on boot into Windows. Makes for a nice throwaway drive for things that are read/write intensive (like compression/extraction) or putting a mini virtual machine on and running it purely in RAM. I also put things like my temp downloads there, page file, temp directories, browser caches/history. I suppose you could put the scratch drive for Photoshop on it as well, but I have never tried that.

Technically, you can also save all the files from the disk on shutdown, but you have to wait for them to copy back, though I prefer the volatile nature of the drive and having it wipe everything on reboot except for a predefined directory structure that gets recreated each boot.

RAM is cheap and I ended up buying 24GB for $150 or so a while back so having a ramdrive gives you a use for all that excess RAM. When I'm using my laptop with 4GB I miss having the extra space a bit.

[1] [http://www.ltr-data.se/opencode.html/](http://www.ltr- data.se/opencode.html/)

[2] [https://gist.github.com/yareally/6199239](https://gist.github.com/yareally/6199239) (my batch startup file example and link to a more detailed guide)

<comment><USER22> You put your page file in a Ramdisk? We need to go deeper...

<comment><USER21> Mainly to avoid the Windows page file from getting fragmented on a HDD, since fragmentation on a RAM disk is a non-issue. I keep my home desktop PC on most of the time and don't reboot often unless switching OSs.

<comment><USER23> If you have RAM free, Windows shouldn't be using your page file. If you have so little RAM free that Windows is paging, then using even more RAM to host the pagefile will make things worse. You might as well just turn off the pagefile entirely.

<comment><USER21> Windows likes to use it regardless of the amount of RAM you have free. Page File on Windows is unfortunately not as friendly as the swap area on Linux (where what you said is true).

Currently I have 8 GB free (out of 24 GB) and the page file is sitting at 944 MB (up from 96 MB) after rebooting about 12 hours ago. Windows has always been this way regardless of the version.

Turning it off is also not a good idea as Windows does use it regardless of free RAM.

<comment><USER23> If moving some data to disk helps performance by freeing up RAM, then putting the swap file back in RAM won't help because it's just taking up RAM again. Or, if you're just worried that Windows is putting things in the swap file that would perform better if they were kept in memory, it would be just as effective to disable the page file to force them to be in RAM. Either way, there's no improvement.

<comment><USER21> If you read my previous posts, the page file is far from the only reason I have a ram drive. Since I use it for lots of other things and I rarely use all of it, I might as well put my page file there as well. I also have 24 GB of RAM so not overly worried about storage.

I regret mentioning the page file, because it always starts some sort of silly debate on any forum it's discussed (Google yields tons of results).

<comment><USER8> Windows does things like swap out applications which are minimised or not recently used to allow more space for disk cache. You're neatly defeating the point of swapping them out by keeping the page file in ram.

You are still shuffling pages in ram in order to "swap" them though. The gp is correct. To achieve your goal, disable the page file. Having it on the ram drive entirely defeats its purpose; The ram backed page file won't help you in the case of memory exhaustion, and it doesn't allow the OS to put the ram to better use where it judges that possible. It's your machine and your perogative, but it is daft, and people are going to point that out.

<comment><USER24> Defeating it is a good thing. I don't want to waste time waiting for programs when I unminimize.

And windows gets cranky without any swap at all. When I 'disable' swap I always set it to 20MB or something.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> How does this affect cloud providers using SSDs? Are they using the reliable and expensive version? Do they use consumer ones and then swap them periodically?

</comment><comment><USER26> How does everyone else set up their dev systems? I just rebuilt my dev machine and I'm using one SSD drive for the OS, one SDD for data (database, Cassandra stuff), and then a regular drive to hold code and builds since that gets rewritten several hundred times a day sometimes.

<comment><USER27> > I just rebuilt my dev machine and I'm using one SSD drive for the OS, one > SDD for data (database, Cassandra stuff), and then a regular drive to hold > code and builds ...

So instead of one large storage pool, you have three. This means your chance to have your system grind to a halt with a disk-full condition is three times greater than it would otherwise be.

I've never understood this logic, but I certainly see it implemented a lot. It's a new version of disk hard-partitioning in *nix systems -- a scheme that stopped making sense once drives because large enough to contain all the resources on one unit.

With one drive containing all stored resources and no hard partitions, there's exactly one disk-full failure mode -- when the disk is full.

When Charles Lindbergh was asked why he wanted only one engine on his historic Atlantic-crossing plane, he replied that, since the multi-engine planes of his day couldn't stay airborne on one engine, his design was actually safer.

<comment><USER28> I used to have the same opinion you did until I started doing some really filesystem intense stuff (not even big, just lots of creating and deleting of files) and suddenly my entire system ground to a halt. Huge delays (top showed state "D"—waiting for device).

I moved that stuff to its own disk and my system became smooth as butter again.

I'm still not down with the traditional unix '"/usr" must be its own partition!' thing, but it _does_ help to separate your tasks onto separate disks.

</comment></comment></comment><comment><USER29> Zsolt Kerekes is a pretty sweet name. I don't think there is a name that says "solid state storage" better than "Zsolt Kerekes".

<comment><USER30> It is a simple Hungarian name and translates very roughly to Solt Wheeler.

[http://en.wikipedia.org/wiki/Zsolt](http://en.wikipedia.org/wiki/Zsolt)</comment></comment>


Ask HN: Workflow for switching between projects - <AUTHOR>

I always work in several projects in parallel. All of them use Git (for version control and sometimes deployments) and a task runner (grunt/gulp).
I switch between projects several times a day. This means closing the terminal window I'm currently on, opening a new one, splitting it in 2 (I use iTerm), navigating to the project folder in each of them, starting up Grunt/Gulp in one of them and managing Git in the other, and then switching projects in Sublime.
This routine gets very boring very quickly. I'm looking into ways to automate it, but the only solution I've found is to write a bash script for each project, which isn't ideal because:
1. It means a new custom script for each new project (the projects are usually quite small and have fast turnarounds, so I would have to do this a couple of time a month) 2. iTerms panes and bash scripts don't play very well along
I was wondering what other people's workflows were like? There must be a better way. <comment><USER2> I use tmux and either tmuxinator or teamocil to accomplish this. I use vim as my editor so it just works within the shell.

Each project can have its own tmuxinator config and load up tmux with a number of shells. There is even native tmux support built into iTerm2 but I haven't used it.

</comment><comment><USER3> Why not use a saved window arrangement? It'll restore your splits, and working directories too if you use the nightly build.

</comment><comment><USER4> Why not just use one iTerm window for each and minimize the inactive ones or use a separate virtual desktop for each project?

<comment><AUTHOR> But you still have to open the iTerm with split panel and navigate to the correct folders the first time you open a project every day. These are the steps I want to automate.

<comment><USER4> Why do you close everything out every night?

<comment><AUTHOR> That's a good question.</comment></comment></comment></comment>


Free integration platform (zapier alternative) - <AUTHOR> https://apiway.ai/ <comment><AUTHOR> We launched Apiway as a completely free integration platform where you can connect different applications for free. We earn from promoting software on our marketplace. Therefore, we can provide a completely free platform, without limits.

We believe that vendors should pay for integration. That's why we mixed two business models: SaaS and Marketplace. We want to make a "Free Zapier and G2 Crowd Mix"

Use case: the user makes integration between Facebook lead ads and Google Sheets. The interface gives him advice that he needs a CRM. He can choose Hubspot, PipeDrive, or another. Let's assume he chooses PipeDrive and starts using it.

As a result, the user set up the free integration and automated the business process. The vendor received a new client. We earned our interest. Everyone's happy :)

<comment><USER2> Have you discussed rates with Google, Facebook, etc?</comment></comment>


Look Out for a Hidden Test at Your Next Job Interview - <AUTHOR> http://lifehacker.com/look-out-for-a-hidden-test-at-your-next-job-interview-1587095685

<comment><USER2> I really doubt the seriousness of anyone who thinks that throwing away an empty cup of water is a good test of character. Interviewing (and probably working) at Hubspot sounds like having a high maintenance boyfriend / girlfriend that is always 'testing' you and judging you on how well you jump through their ridiculous hoops. No thanks

</comment><comment><USER3> I read of a university that tested potential professors. At a lunch, the applicant would have a piece of fruit tossed their way. If they bit into it, and did not use the provided fruit knife on the table, they weren't hired.

</comment><comment><USER4> > He waits to see if the applicant throws away the cup at the end. If they > don't throw away trash, he says, they aren't a good fit for the company.

Which is absolutely true, whether the test is an accurate test or an inaccurate test.

</comment><comment><AUTHOR> This kind of thing is almost a caricature of itself.

 _He waits to see if the applicant throws away the cup at the end. If they don 't throw away trash, he says, they aren't a good fit for the company_</comment>


HOW I WRITE QUERIES USING PSQL: COMMON TABLE EXPRESSIONS - <AUTHOR> http://www.chesnok.com/daily/2013/11/12/how-i-write-queries-using-psql-ctes/

<comment><USER2> Your caps lock seems to be working</comment>


Biden Calls for Repeal of Law That Shields Internet Giants from Liability - <AUTHOR> https://finance.yahoo.com/news/biden-calls-repeal-law-shields-185854232.html <comment><USER2> Dumbest misunderstanding of the technology industry, ever.

There is no real benefit from passing this repeal.</comment>


menubar app + s3 + ftp + itunes + backup = Bandwagon - <AUTHOR> http://ridethebandwagon.com/ <comment><AUTHOR> To YCNews readers: free 3-month account

Use coupon code: ycnbw Enter here: <http://buy.ridethebandwagon.com/>

Requires Mac OS 10.4 + iTunes 7+ and spotlight enabled

Would love thoughts/feedback.

<comment><USER2> _'... Mac OS 10.4 + iTunes 7+ and spotlight enabled ...'_

no mac but do you have screencaps & text or vid demo?

<comment><AUTHOR> Here are screenshots...

<http://ridethebandwagon.com/home.html#product-description>

<comment><USER2> Looks like a nice app. I especially like the descriptions here ~ <http://blog.ridethebandwagon.com/about/> Thanks for posting this it looks interesting concept of creating backups for iTunes ... they need all the help they can get to back their music up.</comment></comment></comment></comment>


Real World Functional Programming - Book excerpts - <AUTHOR> http://msdn.microsoft.com/en-us/library/hh314518.aspx

<comment><USER2> What's the relevance of this now? Did these just get added. This book is 2 years old???

<comment><AUTHOR> Hi, yes these were just published on the MSDN site. There's some material from the book, and some new content. For additional background, here's the author's post:

<http://tomasp.net/blog/real-world-book-msdn.aspx></comment></comment>


A lot of complex "scalable" systems can be done with a simple, single C++ server - <AUTHOR> https://twitter.com/ID_AA_Carmack/status/1210997702152069120 <comment><USER2> A site for proof. It keeps amusing me on what hardware/software Stack Overflow/Stack Exchange is running on: [https://stackexchange.com/performance](https://stackexchange.com/performance)

This is way less in HW than most people in the trade (from web devs to devops) seem to think when asked about it.

SO ranks #36 in Alexa right now: [https://www.alexa.com/siteinfo/stackoverflow.com](https://www.alexa.com/siteinfo/stackoverflow.com)

<comment><USER3> At the risk of exposing my ignorance - look at all those "Peak 5%-20%" labels. Doesn't that mean they have a lot more than they need?

<comment><USER4> Stack overflow hosts all (most?) Of their own baremetal servers in their own data center.

Looking at the specs of the machines, they are actually pretty basic as far as servers go. A server isn't barely worth the cost of it's chassis and motherboard if you put less than 64 G ram and 24 CPUs in it.

In other words, these are about the lowest specd proper servers you can get. So yeah, even their modest hardware is still overspecd for running their website.

<comment><USER5> "Their own data center" implies (to me at least) that they built their own data center. That doesn't sound right, so I looked it up, and it seems like they're colocating. That might be what you meant, but the data centers they use certainly weren't built or owned by SO.

<comment><USER4> As far as I can tell, you are correct. They manage their own metal servers/racks inside a colocated data center. Sorry if this caused confusion.

</comment></comment></comment></comment></comment><comment><USER6> Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed workloads 10x by moving them to a single server and a different software architecture more suited for scale-up, dramatically reducing system complexity as a bonus. The number of compute workloads I see that actually need scale-out is vanishingly small even in industries known for their data intensity. You can often serve millions of requests per second from a single server even when most of your data model resides on disk.

We've become so accustomed to extremely inefficient software systems that we've lost all perspective on what is possible.

<comment><USER7> Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point).

Are you saying this can be optimised to fit inside a single 10 core server in terms of compute loads?

<comment><USER8> Don't know why you're being downvoted but I'll assume your question is genuine.

You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM.

250 Gigs is tiny enough that you could probably much get better performance running on high memory instance in AWS or GCP. You'll generally have to write your own multiprocessing code though which is fairly simple - your existing library may also be able to support it.

I once actually ran this kind of workload on just my laptop using a compiled language that performed better than pyspark on a cluster.

<comment><USER7> I'd love to keep it in RAM if I could. The problem is, the library I'm familiar with (pandas) typically seems to take more memory than the original csv file once it loads it onto memory. I know this is due to bad data types but in certain cases, I cannot get around those.

However, even if I could load it all into memory at once, and assuming it takes 200 gb, I'm still using a master's student access to a cluster. So I get preempted like it's nobody's business. Hence why I prefer a smaller memory footprint even if I take up cpus at variable rates through a single execution.

I did try to write my own multiprocessing code for this, but the operations are sometimes too complicated (like groupby) for me to rewrite everything from the ground up. If I'm not reliant on serial data communication between processes (like you'd need to sort a column), I can get it done pretty easily. In fact, I wrote my data cleaning code with this and cleaned up the entire file in half an hour because single chunks didn't rely on others.

However, if you have some idea of how to run these computational loads in parallel in python or any other language on single compute instances (like the size of a laptop's memory of 16 gb), I'd really love to see it. Thanks.

<comment><USER9> Numpy supports memory mapping `ndarrays` which can back a DataFrame in pandas. This lets you access a dataset far larger than will fit in RAM as if it lived in RAM. Provided it's on fast SSD storage you'll have speedy access to the data and can process huge chunks at once.

<comment><USER7> Can you provide a link to this please? My current knowledge is that all numpy data lives in memory, and pandas itself has a feature to fragment any data into iterables so I can read upto my memory limit. I cannot use this feature due to the serial nature of some of the operations that I alluded to (I'd have to almost rewrite the entire library for some of these complicated operations like groupby and sorting).

I do have fast SSD storage because it's on the scratch drive of a cluster and from what I've seen it can do ~300-400 MB/s easily. I haven't had a chance to test more than that since I'm mostly memory constrained in much of my testing.

My current attempt is to push this data into a pure database handling system like SQL so that I can query it. But like I said, I work with a less-than- stellar set of tools and I have to literally set up a postgres server from ground up to write to it. Which shouldn't be a big deal except when it's on a non-root user and I have to keep remapping dependencies (took 5-6 hours to set it up on the instance I have access to).

My other option was to write the entire 250 GB to a sqllite database using the sqlalchemy library in Python, but that seems to fail whether I do it with parallel writes, or serial writes. In both cases, it fails after I create ~64-70 tables.

<comment><USER9> [https://docs.scipy.org/doc/numpy/reference/generated/numpy.m...](https://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html)

You can create memory mapped ndarrays, these act like normal numpy arrays but don't need to fit into RAM. Numpy maps the array to a binary file on disk. The array otherwise acts like an ndarray so you can build a DataFrame with it. Whenever you access an array index Numpy in the background (essentially) seeks that many values into the file to grab the value of that index.

Since you're on a fast SSD and Numpy is fairly smart you'll be able to access your arrays close to your drive's speed. It's slower than if the whole database was in RAM but far faster than distributing the data over a network to a bunch of worker nodes. Memory mapped files let you have array-like access to data on disk as if it lived in RAM. When building a pandas DataFrame from a memmapped ndarray I believe you just need to set copy=False in the constructor for it to Just Work.

I don't know what your data looks like but I doubt loading it into SQLite is going to improve your performance.

<comment><USER7> Unfortunately my data isn't all numbers. It has text too. The sparse examples in that link only show this for reading in numbers. Do you know off hand if it translates well? There is a dtype parameter, but it'll take me a few days to get back to this code, so I figured I'd check beforehand.

Your second paragraph is essentially what I want. I'm willing to wait a day for code that may run in 1 hour from memory, so time isn't entirely an issue unless it's starting to bleed into weeks. The read_csv function in pandas has a parameter called memory_map, but when I tried using it on a smaller 7GB dataset, it read the whole thing into memory (32GB instance) even when I set it to True.

SQLite is definitely not my best option here. It was the only server-less implementation I could find, so I tried to use it and it didn't work. However, a database like implementation will be helpful because each operation I need to do will require data that satisfies certain timestamp and arithmetic conditions. I figured it'd be best to load the whole thing into a db and query it for every operation to train my model.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> This is precisely the point made by McSherry, Isard and Murray in their lovely paper, "Scalability! But at what COST?" (Usenix HotOS '15). They demonstrate how much performance headroom there is in modern CPU and memory, and show how simple cache-sensitive batch algorithms running on a single core can outperform hundreds of cores running distributed map-reduce style jobs.

[https://www.usenix.org/system/files/conference/hotos15/hotos...](https://www.usenix.org/system/files/conference/hotos15/hotos15-paper- mcsherry.pdf)

<comment><USER11> Big data is about I/O not CPU

I'm a C++ veteran btw and understand the point but big data is about how to process petabytes of I/O not how to consume CPU.

<comment><USER12> This is true in a "water is wet" kind of way. The point is that a great many problems that can fit neatly into a single machine are being turned into I/O problems by being distributed onto clusters.

There's an incredible number of gigabyte and even terabyte scale problems that are consuming racks of blades when a little thinking and understanding of the problem being solved can be done pretty nicely on far fewer resources.

What's really happening is that to many people they think it's "easier" to simply rack more equipment into the cluster and end up shifting the complexity into cluster administration rather than programmer time.

<comment><USER13> >...and end up shifting the complexity into cluster administration rather than programmer time.

Sometimes, that is the right thing to do. The problem is when the cluster solution also adds to the programmer-time complexity.

</comment></comment></comment></comment><comment><USER14> As someone who has implemented a complex system in C++ in this decade, I'd say he's not wrong, but you need to carefully weight the pros and cons.

In our case latency and real time demands mattered a lot (NASDAQ feed parser), to the point of the (potential) slowdown of a garbage collector kicking in was enough to rule out Java and .NET. It runs entirely in memory and on 64+ cores.

We implemented our own reference counting system to keep some of our sanity, and to at least try to avoid the worst memory leaks.

This was an edge case, and for almost everything else you're probably better off implementing it in something that handles memory for you. If performance is an issue, least try it in Go or Rust with a quick PoC before jumping the C++ wagon.

<comment><USER15> Memory management in modern C++ is considerably easier than it used to be. Bespoke memory managers aren't really needed, you can do almost anything you need to without ever using new or delete.

<comment><USER16> Out of curiosity, what is the field for which you find this to be true?

<comment><USER17> If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.

<comment><USER18> This is the quickest way to kill your performance in C++. I guarantee it wasn't what Carmack was talking about.

I used sharedptr extensively in a game engine. Whoops: suddenly 20% of the frame time was gone, never to be recovered. Once that performance is gone it's almost impossible to get it back, short of rewriting every system.

<comment><USER17> Only use shared_ptr when you actually want shared ownership. If you stick to unique_ptr for owning references and then "borrow" that raw pointer in functions, then you get speed without sacrificing too much safety and still no new/delete.

<comment><USER19> Additionally, shared ownership should be rare. Most objects should have a single owner that is responsible for them. This is not only better for performance, but also makes the system easier to understand.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> I'm 15 years in writing high performance Internet servers in C++, and I can confirm higher level languages provide an illusion of capability, but once you're talking high performance with high compute requirements and scaling your service, the cost efficiency of C++ is exponential better than any other language. The higher level language ecosystems are bloated beyond repair. I was able to use one 32-core physical server running a C++ http server I wrote providing a rich media web service and replace an AWS server stack that cost my client 120K per month. The client purchased one $8K 32-core server and co- located it behind a firewall at a cost of $125 per month. And the C++ server ran at 30% utilization, plenty of room for user growth. Their AWS stack of a dozen C#, Python, PHP and Node apps were peaking their capacity too. If course, my solution caused existential questioning by the non-geek CEO and the CTO, but they were in crisis and needed to radically revise how they provided their service or close.

<comment><USER21> The thing that worries me about stories like this is that there is frequently (as is the case here) no mention any sort of HA or backups. No details on what disaster recovery looks like. Those are business critical considerations that cost money, and just disappear from the discussion when people say "hey i saved all this money dropping everything down to a single server!"

<comment><USER20> Well, in my case described above, single server solutions include an automated backup sub-system, and my servers expect multiple instances of itself to running on the client network, and these multiple instances synchronize with one another via additional endpoints specific for the purpose. The whole issue of HA and backups is critical and one of the areas my approach shines.

<comment><USER21> You're still not actually answering the question of how you are HA and backing things up with only a single physical server and nothing else.

If you're backing things up to the same server, that's not enough. If the HA instances are running in the same server, that's not enough.

If there are other things besides that one physical server and it's power/network, you didn't include them in the cost, so the comparison is disingenuous there.

<comment><USER20> I do say the deployed system ends up being multiple instances of my single server, which synchronize with one another. Those are separate physical devices each running my one server. Additionally, when a backup runs the data is stored locally as well as on a physical storage device separate from the hardware it is running. Typically clients already have a firewall/router which is used to distribute requests to the various instances. My deployed systems are not one server, they become a server mesh.

<comment><USER21> >. Those are separate physical devices each running my one server. Additionally, when a backup runs the data is stored locally as well as on a physical storage device separate from the hardware it is running. Typically clients already have a firewall/router which is used to distribute requests to the various instances.

Awesome! Really glad to hear this is the case. But those are all added costs beyond the single physical server you gave the price of.

Server cost * number of physical servers you have deployed Cost of your off- server storage Cost of your network appliance doing load balancing

It's still probably way less than the AWS bill, but it's not really fair to compare the total price of infrastructure in one environment vs. just a portion of the other.

</comment></comment></comment></comment></comment></comment><comment><USER22> Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale.

Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which had about 5x the average request latency of ours.

Then traffic doubled, and suddenly we were having to buy six-figure servers and carefully performance-optimize our code, and our rivals with the 120 Ruby servers didn't look so funny any more. And then traffic doubled again.

<comment><USER23> Web servers are usually trivially horizontally scalable.

You must have had significant in-memory shared state to encounter that problem. Right?

Had you adopted a less stateful model, you'd have looked rather pretty with two Java servers.

<comment><USER22> > You must have had significant in-memory shared state to encounter that > problem. Right?

"Significant" is in the eye of the beholder. The core of the system was easy to make shardable. But you'd be surprised how many implicit assumptions creep in, how easy it is for ancillary parts to end up sharing state when it's easy. Also note that just because your state's in a database doesn't mean having two instances of the thing that accesses it will work, you can easily end up with an access pattern that assumes only one reader (for example) even though on paper there's no in memory state.

> Had you adopted a less stateful model, you'd have looked rather pretty with > two Java servers.

Maybe. At the point where we're running 4 or 8 servers we'd have been facing much the same ops problems that they were. Java bought us an extra year or two of not having to deal with that, but also a significant amount of migration work when it just became impossible to stick to the single process model. At the end of the day we still kept the 5x latency advantage, which is definitely not nothing. But there were also definitely features that they brought to market quicker, and I'm pretty sure Ruby played a part in that.

Tradeoffs, tradeoffs everywhere. I left before the final outcome of that fight (for all I know it's still ongoing), but I don't think either company was being dumb.

<comment><USER23> > At the point where we're running 4 or 8 servers we'd have been facing much > the same ops problems that they were.

Yes and no.

You'd need some ops work, but you'd need to worry a lot less about managing your infrastructure provisioning to keep costs low, e.g. reserved instances, dynamic scaling, etc. and putting out fires when you inevitable exceed your tight perf margins.

You could overprovision 24/7 by 50% and write it off. Your competitors couldn't.

</comment></comment></comment></comment><comment><USER24> Yes, I'm always shocked by just how much performance overhead most languages have compared to C and similar lower level languages. It is a price worth paying for better language ergonomics, but I do wonder whether Rust might be able to give us the best of both worlds here.

<comment><USER25> I semi-seriously think the entire modern shape of the cloud is a result of Ruby being really slow.

Back when people were writing their backend business apps in C++, COBOL, Java, etc, if there was ever a performance problem, you could usually just get a slightly bigger machine and grow your thread pools a bit. But once the web took off and Ruby exploded onto it, you couldn't do that, because it's an order of magnitude slower, and doesn't really do multithreading. But, as long as you follow twelve-factor discipline, it scales horizontally like a champ. So, we took to horizontal scaling over multiple VMs (and caching things in Redis instead of local memory or Hazelcast or whatever), and that's been the unquestioned way to do scaling ever since.

<comment><USER26> The push for the need of scaling out started with Ruby and Python's lack of performance. The reason being pushed at the time was, "developer time was more expensive than hardware." Well, that didn't count the amortization of developer time over the lifetime of the product once the product was developed.

<comment><USER27> It's mostly a fallacy that a demanded product becomes "developed". Maybe a game that gains cult status and therefore a long tail end of life. But popular web services are in constant churn and in that space it's valid to trade hardware for programmer productivity.

<comment><USER26> Backend web development doesn't change much once developed. How many ways can one do CRUD on the backend?

<comment><USER28> In my experience this is only true iff the system never gets any more user facing features.

Every new non-insignificant feature requires a new REST-API route, or database table or modification of the GraphQL schema. And depending on how you designed the backend, even small redesigns of the frontend might require changes on the backend. Consider a simple app showing car rentals, where you initially have something like /car/[id], then the frontend guys realise that we need to show cars rented by each customer and it's necessary to have /customer/[id]/rentals.

(Of course it's possible to design a system without any schemas or normalisations, which would make your statement truer, but that's rarely attractive for other reasons.)

<comment><USER26> But all those are just extensions using the established frameworks and development practices for that backend application. Once those are developed, it's pretty easy to add new features. I would argue using Ruby and Python like script languages brings no significant benefit in development time. In fact they make it worse when working on an existing app, with the extra unit tests, difficulty in refactoring, and extra performance work.

</comment></comment></comment></comment></comment></comment></comment><comment><USER29> not how i read his tweet. he was lamenting that python is too slow for some server side development use cases. and he gave cpp as an alternative that would be simpler and faster. he even followed up citing java and csharp.

totally agree. if all your backend server is mostly complex serialization & de-serialization, and pushing bytes to other sub-systems, i think many other languages have advantages over python.

wondering why no mention of go or rust though...

</comment><comment><USER25> What's that, Frank McSherry? For horizontally scaled systems, we should ask what the Configuration that Outperforms a Single Thread is?

[https://blog.acolyer.org/2015/06/05/scalability-but-at- what-...](https://blog.acolyer.org/2015/06/05/scalability-but-at-what-cost/)

<comment><USER30> IMO, the linked article is much more insightful than the flippant comment here might suggest. It's a solid argument, backed by real world data, about how easy it is to make bad assumptions equating better scalability with better performance.

</comment></comment><comment><USER31> Carmack is moving to AI and inevitably has to deal with a lot of Python, which still bottlenecks process here an there despite all the effort to move computation to C extensions. I have a really high hope that he detours a bit and creates very-very good non-python tooling for ML.

<comment><USER32> Maybe he could make Python fast once and for all :p

<comment><USER33> Python has some fundamental language semantics that make it really hard, if not impossible, to create an implementation that can match Java. Pypy is probably the best you can do to optimize python, and it shines in tight numerical loops, but it gets less effective as code gets more complex.

</comment></comment></comment><comment><USER34> I've been programming C++ and assembly for 23 years. Few years ago I became a huge fan of Python. In my opinion Python is amazingly well suited for rapid first revision and can then be swapped out for C++ / asm.

<comment><USER15> This is fine as long as you can convince management to spend the money to rewrite your software. That's usually a hard sell though. In my experience this plan usually ends up with a python monstrosity that everyone hates but is forced to deal with forever.

<comment><USER35> Type hints and dataclasses are a game-changer for Python. Much easier to reason about programs that use them.

<comment><USER36> > Type hints and dataclasses are a game-changer for Python. Much easier to > reason about programs that use them.

Type hints made large python codebase switch from "Any change will blow up to my face" to "I can touch it carefully with protection gloves".

Through, it is still pretty easy to fool mypy and very far of the compile time guarantees that provide most static typed AOT compiled language.

And unfortunately, it also does not bring any advantage in term of performance.

</comment></comment></comment></comment><comment><USER37> My good friend built whole career doing exactly the same. "Replace a cluster of 10 Elasticsearch servers with 1 running a custom built C app and in-memory database".

Of course, it won't work out to replace 1000 Elasticsearch servers - that's where the advantage of a true "big data" tool will show - but none of the clients really have data "that big".

</comment><comment><USER38> Many people don't know about Python's GIL [https://wiki.python.org/moin/GlobalInterpreterLock](https://wiki.python.org/moin/GlobalInterpreterLock)

That's the reason why you need to go multi-process if you want to reach a similar level of concurrency in Python as multi-thread in C++. And that surely adds a lot of complexity.

As a very practical example of this, TensorFlow has a dedicated page with advice on how to make the Python part that reads the files from disk less slow. Think about that: The bottleneck for training a highly advanced AI with millions of parameters is in the 20 lines of Python code to read in a binary file...

[https://www.tensorflow.org/guide/data_performance](https://www.tensorflow.org/guide/data_performance)

<comment><USER39> Many other people "know" about the GIL, to the extent of believing there's no point using threads in python "because of the GIL".

I had a funny such experience lately in a job interview. I told the interviewer his misconception could be falsified with ~10 LOC summing a list with 2 threads.

<comment><USER39> Ok, I see some comments (rightfully) asking for less talk and more code.

# main.py

        import random
        from concurrent.futures import ThreadPoolExecutor as Pool

        items = [random.random() for _ in range(10 ** 7)]

        def run(items, n):
            step = len(items) // n
            with Pool(max_workers=n) as ex:
                res = [ex.submit(sum, items[i*step : (i+1)*step]) for i in range(n)]
            return sum(r.result() for r in res)

        if __name__ == '__main__':
            import timeit
            import sys
            n = sys.argv[1] if len(sys.argv) > 1 else 1
            time = timeit.timeit('run(items, %s)' % n, 'from __main__ import run, items', number=10)
            print("%s\t%.3f" % (n, time / 10))

      $ for x in `seq 1 16` ; do python3 -m main $x ; done

        1       0.172
        2       0.170
        3       0.166
        4       0.155
        5       0.149
        6       0.142
        7       0.144
        8       0.140
        9       0.136
        10      0.135
        11      0.135
        12      0.137
        13      0.135
        14      0.136
        15      0.136
        16      0.136

<comment><USER40> That's not exactly what I would call good speedup.

<comment><USER39> The point of the code is not to speedup the execution of summing a list of random number, but rather to speedup the acknowledgement of N random python developers that they have some misconceptions about the GIL.

I think it does that pretty well but, well, that's just like my opinion.

<comment><USER41> I'm a casual Python programmer (just bit of scripting) and I had no particular misconception about the GIL; I don't care because I write single-threaded cookie cutter scripts.

Your example I think is demonstrating the opposite of what you want to show. Those figures are atrocious.

<comment><USER39> The opposite of what? All I want is the GIL "discussion" to be based on facts rather than emotions, hyperboles and FUD.

I'm not even sure what the opposite of that is.

<comment><USER42> What were the misconceptions of your interviewer you were trying to prove wrong with your POC of multiple threads summing a list?

The way I read your post, it seemed like your interviewer told you that threads in python are not effective for parallel computation because of the GIL, and your example proves exactly that. The performance of your threads are absolutely horrible, if you were to do that in C++/Java/Go you would likely see a speedup in the order of min(16, cores). Your example proves that your threads are exactly serialized during the computation, which I assume was the point of your interviewer (but please clarify my assumption).

Perhaps your interviewer was instead proposing that threads in Python work like a charm?

<comment><USER39> 2 things:

1\. There is a common misconception that any threaded python code is slower than sequential. This is trivially false for IO-bound, and _possibly, potentially, in some cases_ false for CPU-bound. Said interviewer had that very misconception.

2\. More importantly - you and another parent are right. The code does not demonstrate what I thought it does. This is proven by removing the ThreadPool and leaving the code intact:

        import random
        import math
        from concurrent.futures import ThreadPoolExecutor as Pool

        items = [random.random() for _ in range(math.factorial(11))]

        def run(items, n):
            step = len(items) // n
            res = [sum(items[i*step : (i+1)*step]) for i in range(n)]
            return sum(res)

        if __name__ == '__main__':
            import timeit
            import sys
            n = sys.argv[1] if len(sys.argv) > 1 else 1
            time = timeit.timeit('run(items, %s)' % n, 'from __main__ import run, items', number=10)
            print("%s\t%.3f" % (n, time / 10))

        $ for x in `seq 1 11`; do python3 -m main $x ; done
        1       0.799
        2       0.749
        3       0.671
        4       0.715
        5       0.730
        6       0.704
        7       0.649
        8       0.631
        9       0.689
        10      0.613
        11      0.616

This is a result I'd have to silently contemplate before making any further comments.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER43> From the next tweet in the thread: "JAVA [sic] or C# would also be close, and there are good reasons to prefer those over C++ for servers."

<comment><USER23> Yep. C++ is the nuclear option, when you _know_ you need to keep memory reasonable, or capture the last bit of compute performance.

Neither of these are usually the case for web application servers.

</comment></comment><comment><USER44> You can choose a language that optimizes your hardware, or you can choose a language that optimizes your programmers. 99% of the time optimizing the programmers is the right call.

<comment><USER45> Yes, if by "optimizing programmers" you mean "optimizing the manager's corporate structure footprint and bonus incentives".

If the choice is between hiring one good C++ programmer or 15 really dumb Python "backend engineers" (and a team of QA's and sysadmins to support them), what do you think your pointy-hared corporate boss would chose?

<comment><USER21> This is a false dichotomy. You don't have to choose between 1 C++ developer and 15+ package of Python developers.

Personal productivity is generally going to be slower in C++ than it is in Python. In most situations you would gain more productivity out of a similarly skilled Python dev as you are a C++ dev, so you'd probably need to hire more C++ developers.

Unless your argument is "C++ devs are smart and Python devs are dumb", in which case, let's not start calling people names over the language they use.

<comment><USER45> > Personal productivity is generally going to be slower in C++ than it is in > Python.

No, it entirely depends on the skill and experience level of the programmer.

> In most situations you would gain more productivity out of a similarly > skilled Python dev as you are a C++ dev

No, a good programmer with equally good knowledge of both languages will code equally fast in both.

> This is a false dichotomy. You don't have to choose between 1 C++ developer > and 15+ package of Python developers.

You failed to see the point. Large teams of clueless programmers doing things slowly and badly is a feature of the system, not a bug. KPI's for managers don't include lowering headcount and cost cutting as an incentive. (And trust me, you really wouldn't like it if they did.)

> Unless your argument is "C++ devs are smart and Python devs are dumb", in > which case, let's not start calling people names over the language they use.

Yet that's effectively what you just did in your specious 'productivity' argument.

<comment><USER21> >No, it entirely depends on the skill and experience level of the programmer.

We're comparing programming languages. You control for the other variables - otherwise the comparison is meaningless. My argument is equally skilled programmers will generally be more productive in Python than C++.

>No, a good programmer with equally good knowledge of both languages will code equally fast in both.

Care to explain how? Huge amounts of the features in these higher level languages are explicitly to increase productivity, and it's generally pretty well accepted they're successful. In this very comment section there's multiple sets of people talking about having to implement their own reference counting systems, and all sorts of other things. Implementing those systems takes up productivity.

If any one language was the best at everything, we would only have one language. There's trade offs made, and that's why there frequently is a right language (or set of languages) for one job vs. another.

>You failed to see the point. Large teams of clueless programmers doing things slowly and badly is a feature of the system, not a bug. KPI's for managers don't include lowering headcount and cost cutting as an incentive. (And trust me, you really wouldn't like it if they did.)

This simply hasn't been the case anywhere I've worked at for a meaningful amount of time. Empire building was plenty discouraged at all of the places I have been employed long term and there were KPIs for reducing headcount, or maintaining it while taking on additional responsibility, and accounting was always happy to step in if costs were increasing without solid justification. Reducing headcount doesn't even have to mean firing people or managing them out - it can be not backfilling spots, helping people find teams with open headcount and transferring, etc. It's never bothered me any - if we have too many people for the amount of work, I'm more likely to get bored.

>Yet that's effectively what you just did in your specious 'productivity' argument.

You started this whole tangential discussion by taking shots at Python developers for no apparent reason. I'm not calling anyone names, or anything even remotely similar - different languages are frequently differently suited. There's obviously places where C++ makes a lot of sense.

I don't really have a horse in this race - right this moment, I'd do basically any serious project in rust, and cargo-script has me even doing small scripts in rust as well. Maybe Erlang or Elixir if OTP makes a lot of sense for the project.

</comment></comment></comment></comment></comment><comment><USER46> PCI-e 4.0 and 5/6 enables consuming absolutely insane amounts of data. I can't think of many applications that need that.

128GB/sec? sure! and 256GB/sec is coming.

With 10Gb, 40Gb and 100Gb internet links becoming mainstream the roadblock to having cool stuff is developers to make it.

<comment><USER47> Hold my beer, I'm firing up my [ insert dev stack ].

</comment></comment><comment><USER48> Scalability and performance are very different.

Opting for a performant but not scalable solution is basically an acknowledgement that:

\- The project will only succeed up to a certain point.

\- After the initial launch, no further changes will be made to the project since new additions are likely to cost performance and lower the upper bound on how many users the system can support.

Not many projects are willing to accept either of these premises. Nobody wants to set an upper bound on their success.

Also, it should be noted that no language is 'much faster' than any other language. Benchmarks which compare the basic operations across multiple languages rarely find more than 50% performance difference. The more significant performance differences are usually caused by implementation differences in more advanced operations and in any given language, different libraries can offer different performance characteristics so it's not fair to say that a language is slow because some of its default advanced operations are slow.

Usually, performance problems come down to people not choosing the best abstract data type for the problem. Some kinds of problems would perform better with linked list, others perform better with arrays or maps or binary trees.

Time complexity of any given algorithm is way more significant than the baseline performance of the underlying language.

</comment><comment><USER49> the title is misleading. sure, it advocates for C++, or Java, or C#, or others above Python, but clarifying the context: "a lot (not all!) of complex "scalable" systems can be done with a simple, single C++ server.", which I take as: "sometimes it's better to write a simple server in a low level language, than a complex server in a higher level language".

<comment><USER50> Submitted title was "John Carmack advocates C++ for server development". We've replaced it with a shortened version of what he said.

</comment></comment><comment><USER51> I'm not sure there's any such thing as a simple C++ server ;)

</comment><comment><USER52> people will bike to work to "save the environment" but won't use C direct on metal to reduce the carbon footprint of their code.

For a guy like Carmack, it may be quite frustrating working with the constraints of pytorch etc. He ll probably end up making his own pytorh frontend in C, which as a bonus people will use to deploy models.

<comment><USER53> I don't think it's that big of an language issue. Java isn't that much slower (logarithmically speaking). But design choices have arbitrarily incorporated huge amount of bloat and inefficiency.

<comment><USER52> yep, logarithmic. Well, the question is what's the motive about those design choices? Today's e.g. web app choices seem to be basically aesthetic with an eye for extreme novelty, and total disregard for how many times a buffer is being copied back and forth before reaching the user.

<comment><USER53> That is compared to python/ruby which would be multiple logs slower or scaling choices that makes the service resemble the microkernels. Usually dedicated, simple engineering can be very fast using either C or java

</comment></comment></comment></comment><comment><USER54> Keep in mind a modern commodity x86 server with 128 physical cores 4 TB RAM and decent amount of SSD storage and dual 100 gbe nics is about 70K. Ability to use something like Rust also changes equation significantly.

<comment><USER55> I believe the key is able to use something _native_, not interpreted code or bytecode, running on a virtual machine, which runs in a userspace process inside a virtualized server inside a bare metal server.

Said that, I believe Rust has a great future in data processing. Just take a look at Apache Arrow Rust bindings and Ballista ([https://news.ycombinator.com/item?id=20456273](https://news.ycombinator.com/item?id=20456273))

</comment></comment><comment><USER56> Has Carmack looked at Rust? I'd be very curious for someone like him to look at it in light of his experience with C/C++ and high performance systems.

<comment><USER57> Good question, not sure why you were voted down. And I'm not even a Rust fan.

<comment><USER56> I was curious mainly because I remember some long comments he made about the relative value of linting and static analysis tools, specifically going into Coverity's analysis of the Doom 3 code (I think), fixing everything it flagged for some subset of the code, and then asking himself whether it was really better or more of a hindrance that obfuscated code.

IIRC, his conclusion was mixed: a lot of it was obviously beneficial and worth having turned on, but much of it wasn't, and going forward he intended to make it a limited but continual part of his toolchain.

So my interest in whether he'd tried Rust was whether he'd compared Rust's changes like the borrow checker against his earlier conclusions on writing good C/C++. Cute that he's tried and liked it, but I'd really like to see a more in-depth comparison from him.

<comment><USER57> Right. He's got some kind of notorious programming style with who knows what kinds of object graphs. It would be quite a data point to know if he thinks that he can comfortably map it to Rust.

</comment></comment></comment></comment><comment><USER58> _Morals of this story: 1\. Always use the most performant language available to you. (What if your program gets a few million users?)

2\. Horizontal scalability is too much complexity/work. Just apply the correct amount of optimization when you initially write the code._

If you, like me, read this on mobile and did not click [more answers] link under that, then do it. That may save you a minute or two of derealization time.

<comment><USER59> I really had a hard time to decide whether I agree with that statement or not. If you design a whole service it is obviously not true. But if you develop something like a specialized Backend, say a database, you might want to reconsider the complexity of horizontal scalability.

</comment></comment><comment><USER60> This has been a repeated point since "enterprise" Java (a.k.a. Jabba) became a thing in the late 90s and early 2000s. A ton of enterprise code is comically inefficient and held together with scotch tape and used chewing gum.

It ends up boiling down to the fact that compute power is a lot cheaper than developer time and really good developers are more expensive and harder to find than inexperienced ones.

</comment><comment><USER61> From this thread, I feel a lot of people misunderstand what a scaleable system means for a scale-up startup or BigTechCo. It doesn't mean cost-efficiency. It means the ability to solve scalability issues at 10x, 100x, 1000x load by throwing money at it (aka buying more instances/machines).

Yes, John Carmack is right that a bunch of scaleable systems that see reasonable load, with well-understood traffic requirements could be rewritten more efficient ways. But how long would it take? How much would the extra work cost? And what would happen if traffic goes up, to another 10x, 100x? Can you throw still throw money at that problem, with this new and efficient system already in-place?

One of my memorable stories comes from an engineer I worked with when we rewrote our payments system[1]. He told me how at a previous payments company, they had a system that was written in this kind of efficient way, and needed to run on one machine. As the scale went up, the company kept buying bigger hardware, at one point buying the largest commercially available mainframe (we're talking the cost being in the multi-millions for the hardware). But the growth was faster and they couldn't keep up. Downtime after downtime followed at peak times, making hundreds of thousands of losses per downtime.

They split into two teams. Team #1 kept making performance tweaks on the existing hardware to try to get performance wins, and increase reliability during peak load. Team #2 rewrote their system in a vertically scaleable way. Team #1 struggled and the outages kept getting worse. Team #2 delivered a new system quickly and the company transitioned their system over.

What they gained was not cost savings: it was the ability to (finally!) throw money at their growth problems. Now, when traffic went up, they could commission new machines and scale with traffic. And they could eventually throw away their mainframe.

Years later, they started to optimise the performance of the system, making millions of $ in savings. The new system - like the old one - cost millions to run. But that was besides the point. Finally, they stopped bleeding tens of millions in lost revenue per year due to their inability to handle sudden, high load.

It's all about trade-offs. Is cost your #1 priority and are you okay spending a lot of development time on optimising your system? Go with C++, Erlang or some other, similarly efficient language. Is product-market fit more important, with the ability to have high uptime, that you can buy? Use the classic, horizontally scaleable distributed systems stack, and worry about optimising later, when you have stable traffic and optimisation is more profitable over product development.

[1] [https://blog.pragmaticengineer.com/distributed- architecture-...](https://blog.pragmaticengineer.com/distributed-architecture- concepts-i-have-learned-while-building-payments-systems/)

<comment><USER62> (u meant they rewrote in a horizontal scaling way... Add more machines to a pool to scale)

<comment><USER61> Yes - thank you! Corrected.

</comment></comment></comment><comment><USER63> What about in the context of trying to get to an MVP? Is the dev time speedup of using a dynamic programming language and stack significant over using a c++ backed? You wouldn't care much about performance when you're trying to figure out if you'll get traction.

<comment><USER45> No, it's not significant. Dev time depends on programmer skill, not the toolset. A good C++ programmer will develop your MVP many times faster than an average Python programmer.

Python programmers are much easier to hire, though - you already need a good C++ programmer on the team to hire another one, because HR and corporate management can't into proper hiring process.

This last factor is the overarching most important one for BigCorp Enterprise Inc., not development speed or cost.

<comment><USER28> > Dev time depends on programmer skill, not the toolset.

This is obviously not strictly true, always. A skilled programmer will use the proper tools for the job.

If you for example is tasked with writing a backend service exposing a GraphQL API, I think it would be foolish to do this in C++, and would bet that the average Python programmer would do it quicker than even a top-tier C++ programmer (if the latter would be hellbent on doing it in C++).

Especially when working with MVP's (or new projects in general), the ability to leverage already existing tools and frameworks are key to rapid progress. This doesn't necessarily have to be scripting languages, but the Python/Node/Go/etc developer would have a working GraphQL server up and running connected to a database of choice within an afternoon while the skilled C++ developer would have to spend at least a few days implementing a GraphQL server mostly from scratch [0].

[0]: A quick Google show that schema parsers exists for C++, but nothing matching the frameworks/library available for more web-fashionable languages.

<comment><USER45> Two points:

a) Writing a schema parser is not rocket science. In fact, for a good programmer implementing their own GraphQL library would be quicker than integrating some third-party library. So your first point ("average Python programmer would do it quicker than even a top-tier C++ programmer") is absolutely wrong.

b) There's no value in an MVP that does something generic that is already available in off-the-shelf libraries. Your GraphQL example is pretty pointless because it doesn't actually do anything.

> ...the Python/Node/Go/etc developer would have a working GraphQL server up > and running connected to a database of choice within an afternoon

Well, no. By the end of the week they'll still be arguing about which package manager to use, whether TDD is a good idea, what makes a microservice 'micro' and how to configure Kubernetes.

</comment></comment></comment></comment><comment><USER64> A single server cannot be made reliably available, though.

At single-node scale, development costs dwarf hardware costs of scaling horizontally.

</comment><comment><USER65> vibe.d is well worth a look: [https://vibed.org/](https://vibed.org/)

<comment><USER66> Isn't D dead practically? [https://news.ycombinator.com/item?id=21902953](https://news.ycombinator.com/item?id=21902953)

<comment><USER67> And yet dicebot continues to work in D, including for me for a while.

I'm hiring 25 D programmers, so I suppose it very much depends on what you mean by practically!

<comment><USER66> What company is it?

</comment></comment></comment></comment><comment><USER68> OTOH, we can assume one doesn't need to worry about using C++ unless they have many millions of users.

</comment><comment><USER69> This is why Google developed Go. Python like productivity with C++ like performance...

</comment><comment><USER70> What if those "scalable" systems were designed to "just work" and then after it succeed it was changed into pseudo ""scalable""?

</comment><comment><USER71> Been saying this for years.

Modern SW trends are horrific

</comment><comment><USER72> wow, so much speculation, zero empirical evidence.

<comment><USER73> His entire comment was based on empirical evidence. Which is why people are disputing it - because there is room to do so.

</comment></comment><comment><USER74> The catch is: until you can't.

</comment><comment><USER75> The analogy here is one of the best ice climbers in the world proposing an ascent of the Matterhorn. Please use testable, easy to prototype with, memory managed languages for production servers unless you are solving a very specific problem and _really_ know what you are doing.

<comment><USER76> Very apt. I think people are also forgetting "the bad old days" where you worked on a large-ish cpp or java app for a year, nothing worked right, schedules slipped, and then the whole thing was scrapped and teams disbanded to work on other stuff. That was very common. You can't count on having a team of Carmacks work on your blub app.

<comment><USER45> Replace "cpp or java" with "Python or Ruby" and how is it any different today?

</comment></comment></comment><comment><USER77> > ...My bias is that a lot (not all!) of complex "scalable" systems can be > done with a simple, single C++ server.

The second tweet of the discussion:

> JAVA or C# would also be close, and there are good reasons to prefer those > over C++ for servers. Many other languages would also be up there, the > contrast is with Really Slow (but often productive and fun) languages.

I'm afraid that Carmack has sided with his own anecdotal experiences of the 1990s to justify the use of C++ for server-side development for the 21st Century. This probably made sense at the time due to the availability of more C++ devs and less language choices, but today in the 2020s? I remain totally unconvinced by his argument.

He goes on to suggest Java or C# which still makes sense for many companies for generic server-side development if you are after a more secure backend. Kotlin is pretty much the most sensible for this. But for the sake of Carmack's engineering background however, it is unsurprising why Java/C#/Kotlin are technically unsuitable for high-performance gaming platforms if one was to create one. So what credible languages could be used to compete with C++? I hear Discord is having a great time with using Elixir (Erlang could also be used) and another gaming platform called 'Hadean' is using Rust for their platform.

<comment><USER78> "But for the sake of Carmack's engineering background however, it is unsurprising why Java/C#/Kotlin are technically unsuitable for high- performance gaming platforms if one was to create one."

This really is just not true.

Financial, real-time style High Frequency Trading apps are often written in Java - not C++.

Much of the JVM is not a VM, it compiles to machine code - in an optimised manner. For starters.

Given how difficult it is to develop safely in C++ I can hardly think of a reason to ever use it on the backend.

<comment><USER79> Trading apps generally process a small amount of data. Graphs are downright lightweight compared to what a 3D game pumps through.

Generally, for a 3D game manual memory management and explicit data layout are critical. For example, it's common to use custom memory allocators with a region for each frame, a region for each loaded level, etc... This is then much cheaper to simply drop on the floor than _any_ kind of object-by-object cleanup, whether that is reference counting, garbage collecting, a traditional heap, or whatever. Even Rust can't yet compete with this!

Similarly, many game engines use ECS systems or in-memory columnar data layout to (structure of arrays instead of arrays of structures) to enable SIMD instruction sets such as AVX.

Java can be coerced into doing much of the above, but it generally takes a ridiculous effort to approach what comes nearly effortlessly with a language like C++ or Rust.

Even C# is a better choice than Java, as it monomorphises more code and recently had a range of extensions[1] added to reduce GC pressure such as stackalloc, Span, Memory, MemoryPool, SequenceReader, ValueTask, etc...

I've bought and played several games written in C#, but other than Minecraft I'm not aware of any popular real-time games written in Java in the last 15 years or so. Meanwhile, Minecraft is not at all smooth on my very high end gaming PC in 2019 despite being 8 years old. (I'm sure this can be eliminating by tweaking some settings, but it's indicative of the problem.)

I say this as someone who used to professionally develop browser-based Java games back in the early 2000s and had to personally jump through hoops to reduce heap allocations to avoid GC pauses.

[1] [https://docs.microsoft.com/en-us/archive/msdn- magazine/2017/...](https://docs.microsoft.com/en-us/archive/msdn- magazine/2017/connect/csharp-all-about-span-exploring-a-new-net-mainstay)

<comment><USER78> Yes thanks for that but I think author was referring to game synch server not actual games. Synching minimal state data can use small data structures etc. But thanks though great comment.

<comment><USER79> If anything the server-side coding is _harder_. Many games do the full physics/simulation on the server to minimise cheating, and have to simulate from every player's perspective. Meanwhile the clients have a single perspective and most of the computation effort is offloaded to the GPU.

Additionally, most multiplayer games have the same codebase for the server and the client for the obvious reasons. Single player is literally "online play" with an in-memory channel to a local server.

All Quake-based games work this way, including the derivative Source-based games and a bunch of other engine variants. Unreal works this way too if I remember correctly.

You can't realistically write a client in C++ and a server in Java. You'd be practically doubling your development time!</comment></comment></comment></comment></comment>


Migrating Away from Google Analytics - <AUTHOR> https://freshman.tech/google-analytics-to-plausible/ <comment><USER2> "Google Analytics is free to use (for up to 10 million hits per month), but it's only because the company recoups the cost of running the service by using the aggregated data to track users for the purpose of showing personalised ads."

The above quote is a bit misleading. Google Analytics by default uses first party cookies, the data associated with which isn't used for targeting.

For users of Google Analytics who are using Google Advertising products they can also enable the use of the third party double click cookie for remarketing and other advertising targeting use cases. This can also be done without Analytics just using Adword tagging.

I strongly disagree with the premise of the quote that Google makes money off of Analytics by gathering data for targeting. Instead, the value proposition for Google is better described here, [https://www.quora.com/How-does-Google- make-money-from-Analyt...](https://www.quora.com/How-does-Google-make-money- from-Analytics) .

In a nutshell, Google hopes to show advertisers the value of their advertising buys on Google by attributing conversions on their sites to the correct marketing channel.

<comment><USER3> If it data isn't used for targeting, how exactly do they deliver you demographic information about your visitors?

it seems pretty obvious visitors are absolutely associated with profiles of people.

<comment><USER2> Looks like the answer is here and in agreement with what I said above.

[https://support.google.com/analytics/answer/2799357?hl=en](https://support.google.com/analytics/answer/2799357?hl=en)

In short, demographics are from the third party doubleclick cookie and are only available if you enable advertiser features.

Whether this enables Google to click a significant amount more of data on these third party cookies is unclear to me. So my original claim could be wrong. I would guess that a significant amount of the information would still be collected via AdWords tagging for conversions or display Ads.

There are also Apple and Android device IDs for mobile. I don't know if those are app specific or device specific and if they are only logged if advertiser features are enabled.

So the story is a bit murkier than I claimed.

</comment></comment></comment><comment><USER4> Props to the author for moving to an analytics tool that's more respectful to users.

For anyone who is interested in doing the same, I maintain an open source (and self-hostable) analytics tool called Shynet [0] that works without cookies or requiring any JS.

And to be clear, it isn't a SaaS -- the only way to run Shynet is to self-host it.

[0] [https://github.com/milesmcc/shynet](https://github.com/milesmcc/shynet)

<comment><USER5> I found your project when you commented on a similar HN post, and it's great. I've switched all of my sites from Fathom to Shynet. Nice work :)

(Fathom was good but the self-hosted version is in disarray)

<comment><USER4> That's great to hear! I'm glad it's working well for you. And yeah, Shynet is definitely in part a response to half-baked self-hostable tools like Fathom's offering.

</comment></comment></comment><comment><USER6> The opposite of not using Google Analytics is NOT use these new minimalist analytics tools. I'm so sick of seeing this narrative! There are plenty of mature analytics tools that do a thorough job for a small amount of money. For the curious, I've been using Clicky.com. I don't see how using tools that give you only an iota of information makes any sense.

<comment><USER7> The author says that he doesn't need a majority of the features Google Analytics has:

> I also realised that I wasn't even using up to 10% of the features provided > by the service. For the most part, all I cared about is the number of > visitors, page views, and where visitors are coming from (search engines, > social media e.t.c.) so all the other metrics that Google collects are not > important for me.

But I agree, not everyone shares that thought. I believe that Matomo is the only real self-hosted alternative for more advanced analytics.

<comment><USER8> I have been also building a more complex self-hosted alternative, similar to Matomo: [https://usertrack.net](https://usertrack.net)

</comment></comment></comment><comment><USER9> I've opened the following issue to get input about how woke hosts files should treat Plausible and Fathom. I'm interested to hear good takes, if you have any, on the subject of ethical tracking.

What to do about Plausible and Fathom? [https://github.com/StevenBlack/hosts/issues/1346](https://github.com/StevenBlack/hosts/issues/1346)

</comment><comment><USER10> I have been considering giving up Google Analytics for my websites too, but the thing that stops me is that GA integrates really well with Google search engine, providing detailed info about which keywords were used to find your site.

Do other analytics services also provide this feature? If so, I'm curious of how the manage to do it without being integrated with Google.

<comment><USER11> Matomo (ex Piwik) does have an "importer" for search keywords (from GA and Bing and maybe something else) but it's not free: [https://plugins.matomo.org/SearchEngineKeywordsPerformance](https://plugins.matomo.org/SearchEngineKeywordsPerformance)

It uses the APIs of the search engines to download/import the search keywords.

</comment></comment><comment><USER12> So glad to see any alternative to a Google product. It looks like we're slowly moving to a world where it will be basically just Big Corps using Google products and then eventually even they will succumb under sheer weight of "why are you supporting Evil Corp?" It's becoming uncool to use Google, and they deserve it.

</comment><comment><USER13> I reccomend GoAccess [https://goaccess.io/](https://goaccess.io/) as Google Analytics alternative

Not a service but a self-hosted server log analyzer and viewer, no javascript or any other page code necessary, plus it's fast and free.

<comment><USER14> I agree, works really well for us. Above all, no javascript!

</comment></comment><comment><USER15> This article has its heart in the right place at least.

Author mentions privacy as a factor but does not state why they think their new system is better.

> Unsurprisingly, Plausible's reporting for page views was consistently about > 30-40% higher presumably because ga is blocked by some clients. ga is gone > for good now though.

What? Why is the higher number de-facto the accurate one? Maybe ga excludes and/or bots recognizes return visits better. More != more accurate, as the YouTube ad money scandals showed

I like simpleanalytics a lot but it can't differentiate between one visitor viewing 3 pages and 3 visitors. Still, I appreciate their offering for visitor privacy

</comment><comment><USER16> Just two weeks ago I moved to Matomo from Google Analytics- it works great, and it feels like a tiny little contribution to make the world a better place.

<comment><USER17> Yep I did the same a few days ago and its working well so far. Many of the same features and layout but without all the Google baggage.

Also with 1 or 2 line changes to the default config, you can allow it to get around any adblock/privacy filters that may have their default JS blacklisted.

</comment></comment><comment><USER15> Curiously, matomo analytics, a more privacy friendly competitor, keeps getting blacklisted by Edge for distributing malware. It happens so often that matomo has a faq about it. I suspect this means our sites will not show up in search engines and you have to fill out annoying forms to get re listed. It's very annoying to get punished for spending a day trying to track people less

</comment><comment><USER18> "I also realised that I wasn't even using up to 10% of the features provided by the service."

Most of these people could get everything they want from just analysing the webserver logs. But I guess that isn't the modern way, a true web pro has just gotta load 12 frameworks to display a page of text.

<comment><USER19> My time is limited. I'd rather spend that time gluing together other bits of functionality than reinventing each bit one by one, unless I have to

<comment><USER20> I've deployed this in the past.

Analog[0], AWStat[1], W3Perl[2] are some that come to mind.

[0] - [https://www.c-amie.co.uk/software/analog/](https://www.c-amie.co.uk/software/analog/)

[1] - [https://github.com/eldy/awstats](https://github.com/eldy/awstats)

[2] - [http://www.w3perl.com](http://www.w3perl.com)

</comment></comment></comment><comment><USER21> For a paid product dashboard is a bit basic: long lists have no pagination and opens in modals; Linux is split into distros, while Windows and Mac are just single entry for each; no browser versions and no way to check what browsers are used on what operating system.

</comment><comment><USER22> Wish there was free competitor

<comment><USER23> PostHog Analytics [1] is open source, and sofar a very nice alternative (or complement) to Google Analytics.

It's more like Heap Analytics, in that it collects user clicks and other events from your site (or app) and allows you to _retro-actively_ define "actions" based on these events.

For instance, if you decide today to keep track of how many users click your sign-up button on page X, PostHog can graph this metric for you for every moment since you first installed it on your site.

You can combine actions into funnels, graph nearly everything, the GDPR support is first-class, there's support for heatmaps, feature flags (rolling out new features to just a percentage of your visitors), and user cohort analysis.

[1] [https://posthog.com/](https://posthog.com/)</comment></comment>