Solar storm risk and EMP attacks - <AUTHOR> https://chris-said.io/2020/06/18/everything-ive-learned-about-solar-storm-risk-and-emp-attacks/ <comment><USER2> I was a bit surprised by this exchange:

 _Q: Should I be more worried about EMP Attacks or solar storms?_

 _A: Opinions differ. But the EMP Commission is more worried about EMP attacks._

 _Q: Why?_

Here's where I expected the answer to be:

A: Because it's the _EMP Commission_. It's right there in the name. The _Solar Storm Commission_ is down the hall on the right.

(I personally imagine it in the voice of John Cleese)

</comment><comment><USER3> This reminds me, I've been toying with the idea of starting a github repository for survival and rebuilding society if a major SHTF scenario occurred. It would be crowdsourced and bake-able into an ebook for offline reading. Anybody interested in it as a "Show HN" post?

<comment><USER4> If such a major SHTF happens that it warrants rebuilding the society and figuring out how to survive, then Github will be down, internet will be a rare occurence and your computer isn't likely to work for very long either.

Electronic distribution and consumption of media isn't very resilient if one expects a large-scale SHTF event. For such a scenario, it'd be better to pump out tons of copies as cheap paperback books.

Of course, this depends on the definition and scale of the SHTF. Personally I don't think it's much of an SHTF if one still can consult e-books in any form.

<comment><USER5> An even better plan would be to learn the skills that you would put in that book. The amount of information you can put in a book is only enough for a rough starting point (and that's if the book only has one subject). The practical experience necessary to do these things correctly and efficiently is only really gained by doing them repeatedly.

A further improvement would be to build a small community that practices these skills and tries to be as independent as possible from the current grid. These are all orders of magnitude more complex and difficult than the last but just imagine how much more difficult it would be to bootstrap society in a real SHTF scenario.

<comment><USER4> I completely agree with you. Any book would be at best a memory aid or a reference.

One step above the "small community of off-the-grid preppers" would be to come up with ways to make today's societies more resilient to systemic hick-ups, small and large, using either the means available already today or those which are very close in the technology tree.

Doing so would require a lot of decentralization, e.g. by localizing production of electricity and other consumables and making these smaller units more autonomous. Think not big cities with a power plant each, but rather groups of houses sharing solar panels and a wind mill.

Then again, this would probably be a kind of a pipe dream, as decentralization is by definition not completely under centralized control. This creates all kinds of friction. For example, if everyone produced their own electricity locally, what would the power companies do? And what would the state do with the lost tax money? There are plenty of problems like this, and I doubt one can find a win-win situation where the nowadays centralized things would not lose in some way.

</comment></comment></comment></comment><comment><USER6> I interpreted the report as: "There are 9 unguarded remote sites which, if damaged maliciously or accidentally, would lead to complete collapse of the U.S. as a nation."

And then experts focus on 2 unlikely generalized scenarios of an area EMP effect incidentally affecting the sites. As for the more obvious catastrophe of someone specifically targeting the locations? I guess that's being discussed separately?[1]

[1] [https://money.cnn.com/2015/10/16/technology/sniper-power- gri...](https://money.cnn.com/2015/10/16/technology/sniper-power- grid/index.html)

<comment><USER7> I must have missed the "unguarded remote" part of the article. I read that there are 30 critical sites, any 9 of which would be a problem if taken out.

To me, this sounds standard risk management. It sounds like a clear picture that it falls in the "Low Likelihood / High Impact" quadrant, though, which is the toughest quadrant to address, because you get into exactly the kinds of questions you alluded to - unlikely, generalized ideas of what could happen, but having to balance that with everyday operations which are absolutely going to happen. Even with a known mitigation of stockpiling resources to recover from such an event, at a price tag of $300M, there are many other things that money could go to.

And while I'd like to feel safe knowing they have a solution ready, I don't know enough about the budgetary limits or what other programs would have to be reduced to make that $300M be available to give a solid personal opinion on any of this.

<comment><USER8> The US military budget is nearly $700bn annually, and NASA's budget is $22.6bn.

For the US government, $300M is pocket money. It's unbelievably negligent not to spend a relatively tiny sum which could avoid total collapse.

It's like owning a very grand house and not paying a few hundred dollars for insurance.

<comment><USER7> The US government is operating at a deficit. And budgets are assigned to specific departments, who still try to fit all their programs within it.

A better analogy might be that we all live in a very grand house, but can't afford the mortgage. Sure. a family meeting could certainly find the money in a drawer somewhere. But until we have that meeting, everyone is just sitting in their room doing their own thing.

<comment><USER9> This analogy is a fallacy that holds us back. MMT states that our nation is not a house.

</comment></comment></comment></comment></comment><comment><USER10>
      The chances of an EMP attack
      Q: I've heard a lot from you about how dangerous an EMP attack could be. But how likely is it that anyone will actually try to attack us with an EMP?
      A: Unlike the EMP Commission, most national security experts view EMP attacks as a second rate threat. While perhaps some small terrorist groups or rogue nations might launch a localized EMP attack that might take out a substation or two, it's unlikely that any country capable of launching a major EMP attack would actually do so.
      Q: Why not?
      A: Because to launch a major EMP attack, a country would need a large nuclear weapon. And if a country was planning on using a large nuclear weapon, it would make more sense — in the morbid logic of war — to conventionally drop it on a city than to launch an EMP attack which would at most cause some brief power disruptions in a few states. As physicist Yousaf Butt put it, "A weapon of mass destruction is preferable to a weapon of mass disruption".

This is very wrong and just tells me we should be even more cautious due to it being a blind side. The defense industry has for over a decade had highly deployable, targetable, non-nuclear EMP. (ex1: [https://www.youtube.com/watch?v=Lh1rgy25XhU](https://www.youtube.com/watch?v=Lh1rgy25XhU)) If there is one thing I know, it's that the tools we develop for war in other places tend to end up being used back at home. So I would also disagree with the casual dismissal of the likelihood of use domestically. Further, part of the entire reasoning in natsec circles for the increasingly egregious violations of the constitution is because of the increase in ability for non nation-state actors to be able to perform in new types of asymmetric attacks, and I would say non-nuclear EMP would be just one in that list.

The good news: I think most of these problems are solvable, and will assist us in being more ready to explore the extreme parts of our world and beyond. Making tempest and EMP shielding default in electronics manufacturing for example. It would also assist in reduction of the totalitarian surveillance regime... so I say lets all start talking about how to do shielding properly. There is a lot of misinformation out there about it. I do wonder though, how much of that misinformation is on purpose. Like encryption, at what point does the government decide to suppress a technology because it might hinder their power? Things to ponder.

<comment><USER11> It seems like an EMP attack would be best bet against the US for an actor with a very limited number of nukes. From the field strength maps I've seen, a single large nuke (not sure if NK's nukes are large enough) would affect all of the US, two nukes and they could hit the coasts.

Nuking a city destroys a city, destroying the electrical grid takes out most of a nations industrial production _and_ forces them to focus on short-term survival.

An EMP taking out the power grid across the US in a way that isn't quickly recovered from would likely be a lot more devastating - definitely in terms of industrial capacity, possibly also in human losses due to starvation and general collapse - than nuking two major cities with the same yield.

<comment><USER12> Any nuclear exchange will be focused on preventing the other side from using their nukes first and foremost. If the US has N locations to launch nukes from, then an attacker better have N+1 nukes if they want to spend one in the atmosphere for EMP too.

Delivering a conventional strike on N transformer stations would probably be a better idea since there would then likely be a conventional retaliation and not a nuclear obliteration of the attacking country. I guess with North Korea you can't know.

</comment></comment></comment><comment><USER13> Whenever I read about EMP it reminds me of an interesting device called an explosively pumped flux compression generator (EPFCG). It generates an EMP using explosives but the blast is not the goal it's generating a massive electrical current (millions f amps in a microsecond).

From what I can understand an EPFCG needs an explosive-driven ferromagnetic generator (EDFMG). I imagine the EDFMG is used like a fuse or blasting cap for the EPFCG.

I'm not sure of the size of the devices.

[https://www.globalsecurity.org/military/library/report/1996/...](https://www.globalsecurity.org/military/library/report/1996/apjemp.htm)

[https://en.wikipedia.org/wiki/Explosive- driven_ferromagnetic...](https://en.wikipedia.org/wiki/Explosive- driven_ferromagnetic_generator)

[https://en.wikipedia.org/wiki/Explosively_pumped_flux_compre...](https://en.wikipedia.org/wiki/Explosively_pumped_flux_compression_generator)

</comment><comment><USER14> If this subject interests you, I'd highly recommend the book "Sunfall" by Jim Al-Khalili. It's the physicist's first venture into fiction and is based around the threat of coronal mass ejections and a flipping of the Earths magnetic field. He also weaves hacking, encryption and other science themes into the plot.

As you might expect from a theoretical physicist, it's based on solid science, with a small amount of poetic licence that he describes in the afterword.

It's been described as "The Day After Tomorrow meets Neuromancer". Not sure I'd quite go that far, but it's a damn good read.

</comment><comment> Seb-C So I am not an expert in this field, and actually I only have a vague idea in what a transformer does, but is there a reason to believe this would not be repairable? We are not talking about a usual bomb, so most of the materials and infrastructure would still be there. It would not make any sense to build a new one instead of repairing or replacing the damaged parts.

</comment><comment><USER15> > EMP attacks...

For an EMP attack to be effective on modern electronics, you need to blow up a multimegaton nuclear bomb close enough for it to take out whomever is using that piece of electronics...

<comment> T-A No. Optimal altitude for an EMP attack is actually a few hundred kilometers up. The "base case" usually considered in these scenarios is a nuke going off about 400 km over Omaha. At that altitude, the ionosphere helps you create a "slow" E3 pulse which induces a field in the tens of volts per kilometer at ground level; and since you can see most of the continental US, you have plenty of kilometers to work with.

<comment><USER15> That long wave EMP was only a threat to early unprotected powerlines, not electronics.

What is a threat to electronics are VHF pulses from initial xray burst from the bomb.

<comment><USER11> What's the range of that pulse, and how does it relate to the field strength map seen on Wikipedia ([https://commons.wikimedia.org/wiki/File:EMP_mechanism.png](https://commons.wikimedia.org/wiki/File:EMP_mechanism.png))?

Naively, this map makes me think that a single such EMP means enough electronics break that every major power plant or industrial plant is down and not recoverable for weeks.

</comment></comment></comment></comment><comment><USER16> i'm assuming this is now being discussed due to the recent kurzgesagt video : [https://www.youtube.com/watch?v=oHHSSJDJ4oo](https://www.youtube.com/watch?v=oHHSSJDJ4oo) . Key takeaways are that solar flame detectors are only 33% effective and we don't have a stockpile of enough transformers.

</comment><comment><USER17> The UK Sky drama COBRA deals with this scenario and the difficulty of getting replacement transformers after a solar flare

</comment><comment><USER18> So does this mean that fewer if any people will suffer in rural underdeveloped areas like Africa and Asia?

</comment><comment><USER16> It seems to me we need massive localized solar and battery investments as a backup.

<comment><USER19> I think the EMP situation can cause permanent damage to solar panels and batteries, even in off grid configs, though they would indeed help people who are out of the damage zone.

Oh but most wargame EMP scenarios assume full blanketing of the USA with EMPs so no such zone will exist.

Grid resilience is probably more appropriate in this case and is independent of generation technology.

</comment></comment><comment><USER20> "Could Solar Storms Destroy Civilization? Solar Flares & Coronal Mass Ejections" by Kurzgesagt: [https://www.youtube.com/watch?v=oHHSSJDJ4oo](https://www.youtube.com/watch?v=oHHSSJDJ4oo)

<comment><USER21> tldr: Nope.

<comment><AUTHOR> I think the answer is not "nope" but "probably nope". That video is perhaps overconfident in our ability to detect these storms.

From the article:

> One-third of major storms arrive unexpectedly, according to the SWPC's own > 2010 analysis. And that's not just the small storms. According to a news > article in Science, the SWPC might be also be poor at identifying the > characteristics of severe storms, since they are so rare.

<comment><USER21> Sometimes you get close enough to zero that for any reasonable application, the answer can be rounded to zero.

</comment></comment></comment></comment><comment><USER22> A perfect time to bring out new energy source.</comment>


Things Every Programmer Should Know [wiki] - <AUTHOR> http://programmer.97things.oreilly.com/

<comment><AUTHOR> This is the official site for an O'Reilly book that came out recently. The contents of the book can be found on this page: [http://programmer.97things.oreilly.com/wiki/index.php/Contri...](http://programmer.97things.oreilly.com/wiki/index.php/Contributions_Appearing_in_the_Book)

Here's the book's Amazon page: [http://www.amazon.com/Things-Every-Programmer- Should-Know/dp...](http://www.amazon.com/Things-Every-Programmer-Should- Know/dp/0596809484/)

</comment><comment><USER2> the thing is, you browse the first half dozen articles and there's no programming in there. You can't expect it to help you become a better programmer if there are no good code examples to follow. Anything else is just lip service.

IMHO you'd be better off with Bently's "Programming Pearls".

<comment><USER3> I don't agree on the code examples, I think it can be a good book without code examples however this is not a good one :)

As a replacement I'd recommend Code Complete and Pragmatic Programmer

<comment><USER2> Those books are both good suggestions too ;)

</comment></comment></comment><comment><USER4> Am I the only one who thinks the default font is a bit small? A minor quibble, perhaps.

</comment><comment><USER5> Why 97?

<comment><USER6> Because 97 is two less than the product of two ones (eleven) and one number less than ten.

The same calculations lead me to believe that the world will end in 2012!

</comment></comment><comment><USER3> I've recently read the book actually only first "20~ things" out of it and it's a pretty bad book. I stopped reading after 20.

On paper "97 things" idea seems great but in practice it's pretty much rubbish. I've read 97 things a project manager should know as well and that wasn't good either.

<comment><USER7> Could you elaborate on why you think it's bad?

<comment><USER3> \- It's writing style is bad, somehow consistently bad

\- Structure is no good (I think this is expected though) but I was looking for an overall consistency like in Founders at work.

\- Most of the advices written in many words but can be summed to 2 sentences. And rest of the text doesn't elaborate the reasons like they supposed to mostly they are there to fill up space.

Overall I don't have one bad thing but it's like reading a bad novel, you don't feel like that the author or the book has a soul.

Again Founders at Work is a perfect example of having multiple people to contribute but keeping the book in a good writing format.</comment></comment></comment>


Ask HN: How to Grow Your Site? - <AUTHOR> Hey,
So I've been running a website since 2014 geared towards tech, science, and entertainment articles. Its a pretty broad category and one which thousands of other sites operate in. But nonetheless, I have given it a go because it is something I enjoy and is interesting (to me).
In the 3 years I've had it, we've had some very good days, but the rest are pretty average or even poor. The biggest roadblock to growing it is during the summer months, I am almost completely inactive on it as other commitments take up most of my time to think up, research, and craft articles.
I have tried a lot of free advertising options - for example, sharing some (ok, a lot) of articles here but I don't want to spam HN which I practically did when I first found this place. Rarely they are hits, most times not (which is understandable). I am also on some promotion websites. Of course, with little traffic reading the content, it means they never get submitted to places like Reddit or here which can drive lot of traffic. And while I'm on social media with it, there isn't usually much response there either as the accounts only have a couple dozen followers.
But basically, I came to ask, especially if you have a website/blog/forum, what you did to grow yours and also how you found some people to help you do it.
Getting anyone to help write a few articles here and there has been extraordinarily difficult, even now that I have started offering guest writers the chance for a backlink if it is connected to the article. It would be awesome to pay the staff, but the site generates little revenue and that goes towards paying for hosting. <comment><USER2> No silver bullet. I've built a community site to help operators build and grow their startups ([https://www.joinradium.com/posts/tagged/growth](https://www.joinradium.com/posts/tagged/growth)). You can check out the growth category and I'm sure you'll find great content on how to grow your business.

<comment><AUTHOR> Definitely a few good resources on there. I will check back and make use of what I can. Thanks!

</comment></comment><comment><USER3> I advised to change your design first. Try to find a template similar to [http://www.makeuseof.com](http://www.makeuseof.com) or [http://thenextweb.com](http://thenextweb.com). Good luck

<comment><AUTHOR> That was a change we just recently made within the past week. We went to a newer, nicer, more modern theme.

</comment></comment><comment><USER4> Can you post a link to the website?

<comment><AUTHOR> I can. It is [http://thesurge.net](http://thesurge.net)

</comment></comment><comment><USER5> i'm trying to do the same thing with my site.</comment>


In defence of swap: common misconceptions - <AUTHOR> https://chrisdown.name/2018/01/02/in-defence-of-swap.html <comment><USER2> In my experience, a misbehaving linux system that's out of RAM and has swap to spare will be unusably slow. The process of switching to a tty, logging in, and killing whatever the offending process is can easily take a good 15 minutes. Xorg will just freeze. Oh, and hopefully you know what process it is, else good luck running `top`.

Until this is fixed, I'll just keep running my systems with very small amounts of swap (say, 512MB in a system with 16GB of RAM). I'd rather the OOM killer kick in than have to REISUB or hold down the power button.

Some benchmarks with regards to the performance claims would be nice.

<comment><USER3> What function does the 0.5GB swap have?

<comment><USER4> I just wish Linux distro installers would make opting out the default option; no, I don't want to swap on my SSD. The last time I installed a distro, I still had to select the manual option for partitioning.

With an 8 Gig stick in my NUC, for normal desktop usage it never goes above 3.

<comment><USER5> And even then; they provision an absurd amount of it. I just did a fresh Ubuntu install. On a machine with 32 gigs ram, it creates a 32 gig swap partition by default!

<comment><USER6> They probably reason that you want your system to be able to hibernate, plus storage is cheap.

I have 8GB of swap for the 8GB in my laptop, for that reason.

On my desktop with 16GB, the w 2GB of swap it has is sometimes too little, and everything grinds to a halt.

<comment><USER7> 32GB of SSD isn't that cheap!

</comment></comment></comment></comment></comment></comment><comment><USER8> Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you _will_ touch.

Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes. Sure, you'd be able to get to grandma's half broken kitschy dog coaster that you can't bring yourself to throw away, but you'd also be harder to harder to fit and find your own stuff, the stuff you need all the time.

If you find yourself driving to and from the self storage place every day, you probably need a bigger home. But self storage is plenty useful even if you almost never visit it.

<comment><USER9> The issue is that the current OOM killer doesn't support this usage at all.

To extend the analogy: what do you do if grandma comes and fills your house with stuff? You need space to work, so you go and drop it off at the self storage place, but what if she just keeps filling your house up?

The OOM killer will do absolutely nothing until both your house and the whole self storage place are totally full. By that point, you've spent a huge amount of time just driving to and from self storage, so you haven't had time to do any actual work; it would probably have been better to tell grandma that you don't want any more stuff once she filled up your house for the first time.

<comment><USER8> Well, it doesn't help that when grandma calls and asks whether you have room for more stuff, the Linux kernel responds on your behalf, "Yes, of course I have room. I live in a TARDIS." And you then do all driving to the self- storage facility to maintain the illusion as long as you can. I really don't like overcommit.

Anyway, I agree with you that this behavior is annoying, but I think it ought to be possible to fix it (e.g., with memory cgroups or something like Android's lmkd) without giving up on the idea of spilling infrequently- accessed private dirty pages to disk.

<comment><USER10> The analogy is now getting in the way, rather than helping to clarify.

</comment></comment></comment></comment><comment><USER11> I'm a big fan of determinism and service uniformity. Having that rarely used and response time critical function/data/whatever swapped out increases service time variation at best and complicates all worst case response time calculations at least.

I understand from the land of JIT compilers, garbage collectors, and oversubscribed everything that this is not much of a substantial concern as these features are already traded away.

The swap may be the best case in a bad situation. I would argue along the lines of don't be in a bad situation...

I'm looking at you 8 of 16 GB used on cold boot Mac laptops... Looking at you with indignation and rancor Chrome.

<comment><USER12> One of the article's points is that running without swap doesn't necessarily alleviate that. The rarely-used code pages of your rarely-used but response time critical daemon can just as easily be dropped from the page cache and have to be refaulted in from disk, and in fact that's _more_ likely if there isn't swap available to stow the dirty anonymous pages from the cron daemon that wakes up once a day or whatever.

The solution for your rarely-used but response time critical daemon is for it to mlock() its critical data and code pages into memory, which works regardless of whether or not you have swap available. (Or, alternatively, use one of the cgroup controllers that the article alludes to, to give the critical daemon and related processes memory unaffected by memory pressure elsewhere in the system).

<comment><USER13> No they cannot. Code that is loaded into memory stays in memory when three is no swap. (Mlock is to prevent swap out or compression - major faults.) The exception would be lazy library loading. BIND_NOW is your friend in this case.

Essentially having no swap is similar to having everything mlocked - no major faults can happen except with mmapped files which will just use direct disk IO.

If you mean disk caches, when have you seen a multigigabyte executable?

<comment><USER12> _Code that is loaded into memory stays in memory when three is no swap._

That is not true. In the normal case (absent debugging, JIT, self-modifying code etc), pages of executable code are clean, shared mappings so they do not interact with swap at all.

As clean, shared mappings they are eligible to be dropped from the page cache in the same way as other clean file mapped pages.

(Your executable code pages essentially _are_ mmapped files. )

</comment></comment></comment></comment><comment><USER14> > Under temporary spikes in memory usage > With swap: We're more resilient to > temporary spikes, but in cases of severe memory starvation, the period from > memory thrashing beginning to the OOM killer may be prolonged. We have more > visibility into the instigators of memory pressure and can act on them more > reasonably, and can perform a controlled intervention.

Somehow that doesn't resonate with my experience. I tend to remember the cases where I can't even SSH into the box, because the fork in sshd takes minutes, as does spawning the login shell.

I'd really like some way to have swap, but still loosen the OOM killer on the biggest memory hog when the system slows down to a crawl. I haven't found that magic configuration yet.

<comment><USER15> Well the article does suggest a mechanism to apply.

As for the problem with SSH and login: You might well find that it is _not_ the fork that is the problem. You might well be surprised at how much chaff is run by a login shell, or even by non-login shells.

A case in point: I recently reduced the load on a server system that involved lots of SCP activity by noticing that, thanks to RedHat bug #810161, _every SCP session even though it was a non-login non-interactive shell_ was first running a program to enumerate the PCI bus, to fix a problem with a Cirrus graphics adapter card that the machine did not have on a desktop environment that the machine did not have. This driven by /etc/bashrc sourcing /etc/profile.d/* .

* [https://github.com/FedoraKDE/kde-settings/blob/F-26/etc/prof...](https://github.com/FedoraKDE/kde-settings/blob/F-26/etc/profile.d/qt-graphicssystem.sh)

</comment></comment><comment><USER16> Even for those who understand this well, it's historically been really hard to coerce the Linux kernel into applying the right swap policies to your application.

As the author notes much of this has been improve by cgroups, and there's always been big hammers like mlock(), even with those things it can be hard to prevent memory thrashing in extreme cases. I've seen swap disabled completely by people who understood how it worked as a last result because of that.

It's always seemed to me that this was mainly a problem of the kernel configuration being too opaque. Why can't you configure on a system-wide basis that you can use swap e.g. only for anonymous pages and for nothing else?

Similarly it would be nice to have a facility for the OOMkiller to call out to userspace (it would need dedicated reserved memory for this) to ask a configured user program "what should I kill?". You might also want to do that when you have 10G left, not 0 bytes.

<comment><USER17> Android has this last bit. as explained at [https://www.youtube.com/watch?v=ikZ8_mRotT4&t=2145](https://www.youtube.com/watch?v=ikZ8_mRotT4&t=2145) (linked in the article) though, Linux does not have the facilities presently to determine when you have "10G left" in a way that applies across all system configurations.

</comment></comment><comment><USER18> I recently reenabled swap on my Windows machine due to frequent OOM, even with 16GB of RAM while playing Overwatch and browsing on Firefox. It seems like both of these programs allocate vast swaths of memory but then do not actually use that memory very heavily. After I turned swap back on, I did not notice any degradation in performance but my system stability skyrocketed.

<comment><USER19> Windows has vastly different policies for RAM allocation and commit than Linux. Windows basically does not overcommit while Linux systems not only does it by default but quite depend on it for various loads to work properly. In consequence, the userspace has a tendency to handle RAM differently, but there is no magic: if programs are trying to allocate twice the amount of RAM and then only use half of it, Windows with a swap that can be large enough will work perfectly while without swap the allocation will fail. Under Linux, the situation is less clear: without swap it will succeed (well it depends on the fine details of overcommit that are selected, but you get the idea) although if you _really_ then use all that RAM, the OOM killer will start to more or less "randomly" kill "any" process to cope with the lack of RAM (as a last resort measure though; the caches and buffers are flushed before, etc.)

<comment><USER20> Windows also has two syscalls - one to reserve memory and one to commit memory. You can reserve+commit at once, but you can also just reserve a chunk of virtual memory that you commit at a later time. Accessing pages that aren't committed is a segfault. So you can say "I might need up to X contiguous bytes of virtual memory" and then commit as you go. IIRC, Windows will let you over- reserve, but not over-commit.

Edit: sorry, not two syscals, it's an option to the malloc-equivalent - VirtualAlloc

</comment></comment></comment><comment><USER21> Without wanting to impose my method or reasoning here, I run my dev machine without swap, and I'd rather have the same for the cluster machines I access.

This is for academic use only. I know how much RAM my machine has, and if I oom, it usually isn't because I tried to squeeze in just a tiny bit too much data, but rather because I made some stupid mistake and keep allocating small chunks of memory very rapidly. On a system with even a moderate amount of swap, this makes everything grind to a halt, and it is usually much faster to just reboot the machine and deal with the problems later _in the unlikely event that rebooting actually causes problems_.

<comment><USER22> If you're running a single-(or close to)-purpose machine, then seeing an explicit memory usage limit on the main app could give you even better/faster results.

</comment></comment><comment><USER23> We've disabled swap (by not configuring a swap partition) on every server we've deployed since 2009. It's a little irritating to have to manually remove the swap partition from various Linux' "server" default install options even today. Of course this means I'm still installing on bare metal that I own, so...dinosaur.

</comment><comment><USER24> I'd be more convinced by the argument that swap shouldn't be thought of as slow RAM if the author addressed the fact that it's generally known as 'virtual memory'—and it has been since at least System 370, so it's not simply a later misconception: [http://pages.cs.wisc.edu/~stjones/proj/vm_reading/ibmrd2505M...](http://pages.cs.wisc.edu/~stjones/proj/vm_reading/ibmrd2505M.pdf) . Instead the article just omits the term 'virtual memory' completely, and pretty conspicuously.

I also think that a convincing case for swap would have to discuss the concepts of latency, interactivity, and (soft) real-time performance, things that largely weren't to the fore in the salad days of the 370 family or the VAX. Virtual memory is the TCP of local storage.

<comment><USER15> That is not the argument.

The article actually says, four times over, that it should not be thought of as _emergency memory_. It's not emergency memory; it's ordinary memory that should see use as part of an everyday memory hierarchy.

And if you _are_ going to question the terminology, the elephant in the room that you have missed is calling paging swapping. (-:

<comment><USER24> Despite the repeated use of the word 'emergency' this is not quite obvious. For example in

> many people just see it as a kind of "slow extra memory" for use in > emergencies

the scare quotes are around 'slow extra memory' not 'emergencies'. Now granted in the last bullet point of the conclusion it affirms that VM is a source of slow memory, but earlier it uses 'memory' where it's referring specifically to RAM, for example

> Without swap: Anonymous pages are locked into memory as they have nowhere to > go.

Really the main reason my original comment was rubbsih is that I took the article far too much as a general discussion of swap when, as it said, it's largely about how much swap to enable on a given Linux system running some already-determined software.

</comment></comment></comment><comment><USER25> I have an older Chromebook (c720), which is really quite memory starved (2GB RAM), and have experienced ChromeOS completely frying the SSD simply through prolonged tab-heavy swapping.

Now, I've replaced the SSD and installed a non-Google Linux distro, and would like to limit the amount of swapping Firefox can do.

I had been planning to simply use cgroups' memory features to limit the amount of memory consumed by Firefox processes, but if I am to understand the article (which I admit I didn't read in full detail), I should also be able to tune swapping to limit the actual amount of swapping that takes place, avoiding a drastic uptick in SSD wear whenever open too many tabs.

That, and perhaps a Firefox extension that suspends background tabs in memory (which I've used before with a certain amount of effectiveness in the pre- WebExtension days).

<comment><USER26> Firefox has a couple options which may help you get by. I can't guarantee these will fix everything but they are worth experimenting with.

about:memory has various options, including a 'minimize memory usage' button and profiling tools.

about:preferences has Privacy & Security > Cached Web Content > Override automatic cache management (select and set at 500MB, 1GB, or whatever works best).

<comment><USER27> Yep, you can adjust the max amount of cache it will store in ram and disk(separately).

<comment><USER28> I don't see any option to separate that in the settings page, is it hidden behind a flag somewhere?

<comment><USER27> In about:config search for browser.cache.memory.capacity and browser.cache.disk.capacity.They are in kilobytes. To enable or disable any of those caches use browser.cache.disk.enable and browser.cache.memory.enable.

<comment><USER28> Thanks!

</comment></comment></comment></comment></comment></comment><comment><USER29> Swap was a great idea, but its time is gone. Swap doesn't make sense anymore, hard drives have not scaled and kept up with the improvements in RAM.

In the Pentium 1 era EDO RAM maxed out at 256MB/s and hard disk xfer was 10MB/s. Common RAM size was 16MB.

In today's era DDR4 maxes out at 32GB/s and hard disk xfer is 500 MB/s. Common RAM size is 16GB.

RAM xfer rate has grown is 320x. RAM capacity has grown 100x. Disk xfer rate has grown 50x.

Swap is no longer a useful tool.

<comment><USER30> The purpose of swap is not running things out of it. The purpose is shoveling unused data out of memory. And for that it doesn't need to be particularly fast.

Swap has a lot less purpose in a world without memory leaks and extraneous functions. But in practice it's quite good at getting several gigabytes of unnecessary data out of the way, so ram can be used properly.

Swap, well-used, should only take up a few percent of the drive's bandwidth.

<comment><USER31> > The purpose of swap is not running things out of it. The purpose is > shoveling unused data out of memory. And for that it doesn't need to be > particularly fast.

You can't detect a priori whether data is "unused." If you guess wrong a few times in a row, you get the familiar pattern where your Linux box is unresponsive to everything and needs to be bounced.

If you could detect whether data is rarely used, swap still isn't necessary. Applications can mmap() a file and use that region for "rarely used data" if such is known in advance.

Extraneous functions should be backed by the executable in the common case. In the JIT case, they probably won't be JITted anyway.

I still think the OOM killer is less intrusive than swapping to disk. It kills some, but not all, of the processes on the machine get killed. The system pretty reliably comes back to life in less time than it takes a human to diagnose the problem and bounce the system. As a bonus, no human needs to get involved.

<comment><USER30> > Applications can mmap() a file and use that region for "rarely used data" if > such is known in advance.

They could, but that's a lot like just making swap be manual.

> Extraneous functions should be backed by the executable in the common case.

I don't mean the code itself, I mean all the data it builds up for something that isn't needed.

<comment><USER31> > They could, but that's a lot like just making swap be manual.

Sure. Isn't that a good thing? Rarely-used data can be swapped out to storage allocated for the purpose, as you desire, and too-large working sets don't have to hose the machine.

> I don't mean the code itself, I mean all the data it builds up for something > that isn't needed.

I guess I don't often run programs that waste large amounts of memory for no reason. Either my working set fits or it's too large, and swap only matters if it's too large.

Is Java the typical beneficiary here?

</comment></comment></comment></comment></comment><comment><USER32> This is an informative and well written article, but seems incomplete in this day and age. In public cloud environments, network attached storage is far more prevalent, so the swap story may be different there (I honestly don't know though). Since the author works at Facebook, he probably lacks experience in this regard.

<comment><USER33> Every cloud provider I've worked with (okay, so AWS :P) gives you ephemeral local storage. Obvioulsy you don't swap onto a network drive.

<comment><USER32> Modern AWS instance types are EBS-only.

<comment><USER34> With the exception of the High IO types (I2/I3). They still get it and the newer instances get NVMe SSDs. In other words they are making it a feature of certain types that would benefit from it.

<comment><USER35> For example, F1 instances have NVMe local instance storage.

[https://aws.amazon.com/ec2/instance- types/f1/](https://aws.amazon.com/ec2/instance-types/f1/)

</comment></comment></comment></comment></comment><comment><USER36> Does hibernating via a swap file work reasonably well yet? I haven't had a chance to try this out yet, but that's the main reason I still have a swap partition on my laptop.

<comment><USER37> Well done mate - you are the first person to mention this here. It was also only briefly mentioned in the article.

Yes, hibernation does work well and it requires swap. Personally, I set a swap partition equal to RAM + 512MB on systems that I want to hibernate on.

Linux also supports swap files and this might be handy: [https://wiki.debian.org/Hibernation/Hibernate_Without_Swap_P...](https://wiki.debian.org/Hibernation/Hibernate_Without_Swap_Partition)

</comment></comment><comment><USER38> On windows I have found it necessary to disable swap to keep myself efficient. Many times Ive had applications decide to allocate massive amounts of memory and then it leads to my system slowing down with tons of swap activity. In nearly all cases, I didn't want my system to try its best to handle these massive memory requests but rather it should have just killed the offending application. Often in these failure scenarios the swap goes nuts and my computer becomes unresponsive that it takes a long time to even get to kill the bad actor.

Thus I disabled swap and I never had these unresponsive issues. I run with 32GB of ram so generally well behaved applications never run into memory issues.

Some applications that would cause issues would be too many VirtualBox instances that use more than available memory. A text editor that chocks trying to open a >1GB text file (looking at you, the new JS-based editors.)

<comment><USER39> Windows is terrible at swapping. As soon as you hit max RAM performance suffers a lot. Even if you never use inactive apps.

<comment><USER40> Yep. When I was using win7 with 2gb of ram, I was basically managing it manually - as soon as memory usage hit 90-95%,I was shutting apps down. If I let it swap, it would often hang completely.

</comment></comment></comment><comment><USER27> For many years now I use ram compression instead of swap for desktops/laptops. I particularly like zram but zswap is also great if you you are hitting hardware limits.

The difference with swap is that the computer doesn't get unresponsive, it just slows down a bit. And Ram compression still buys some time before OOM killer hits.

</comment><comment><USER41> Count me among the believers in running w/ swap. Here's all it takes to provide Linux with a little swap space:

      fallocate -l 8G /swapfile
      chmod 0600 /swapfile
      mkswap /swapfile
      swapon /swapfile

Add an entry in /etc/fstab & you're done. "This little trick" made all the difference on a compute cluster I managed, where each node contained 96G of RAM. It's much more pleasant to monitor swap usage than the OOMKiller and related kernel parameters.

</comment><comment><USER42> How long before desktop OSs manage memory like mobile ones, automatically shutting down background apps that aren't being used, so that the system remains responsive no matter what?

Or, worst case, if two tasks that need 8GB each are running on a machine with 8GB memory, kill one, let the other finish, and restart the first one. Or, less ambitious, freeze one, swap it out to disk, let the other finish, and only then resume the frozen app.

Desktop OSs are so primitive at memory management, forcing the complexity onto the user.

<comment><USER43> > How long before desktop OSs manage memory like mobile ones, automatically > shutting down background apps

Once desktop systems and applications support required APIs to handle saving state before being shut down.

<comment><USER44> > Once desktop systems and applications support required APIs to handle saving > state before being shut down.

SIGTERM and friends? :-)

If your application is just dropping state on the floor as a result of having an intentionally trappable signal being sent to it or its children, that seems like a bug.

</comment></comment></comment><comment><USER22> I feel like the real metric is like to see from swap usage is: how much time did I spend waiting to swap-in a page, how much extra cache pages it gave me, and what's the cache hit ratio. If the big purpose is to allow those extra few pages to be available, then either it's worth doing or not - there should be an objective way to look at this. Unfortunately only the second and third part is easily available. The first... maybe via systemtap?

<comment><USER44> > how much time did I spend waiting to swap-in a page

You can do this with eBPF/BCC by using funclatency ([https://github.com/iovisor/bcc/blob/master/tools/funclatency...](https://github.com/iovisor/bcc/blob/master/tools/funclatency.py)) to trace swap related kernel calls. It depends on exactly what you want, but take a look at mm/swap.c and you'll probably find a function which results in the semantics you want.

</comment></comment><comment><USER45> So, in other words, if you have enough memory for your workload that you won't run out, there's no benefit to having swap space (i.e. you've wasted money on memory you don't need).

But if you DO have swap space, there won't be a performance hit (at least not under Linux) because it will only swap out some rarely used pages and then sit there doing nothing.

So, in the general case, it's better to have it and not need it than need it and not have it.

<comment><USER33> > So, in other words, if you have enough memory for your workload that you > won't run out, there's no benefit to having swap space (i.e. you've wasted > money on memory you don't need).

No, that's the opposite. If you have enough memory for your workload that you won't run out, swapping lets you use more memory for disk cache (instead of keeping unaccessed anonymous pages in real ram).

Unless by "won't run out" you mean "never have to throw away a disk cache page", which seems very unrealistic.

<comment><USER46> > If you have enough memory for your workload that you won't run out, swapping > lets you use more memory for disk cache

Except that it doesn't happen in practice, on my systems anyway. If you have plenty of memory, you can keep all your programs in it _and_ as much as the system wants to cache and still not run out.

The theory says that swap effectively buys you some memory to spend on more important things (than what the system chooses to page out). So does buying more memory.

> Unless by "won't run out" you mean "never have to throw away a disk cache > page", which seems very unrealistic.

I have an instance of top running on my desktops & laptops all the time. I never see cache using up all of the memory.

                      total        used        free      shared  buff/cache   available
        Mem:           7848        1523        5827          53         497        6033

<comment><USER33> Are you talking desktop workflows here? If this is a service, why are you paying for RAM that never gets used? Get a smaller instance.

<comment><USER46> > Are you talking desktop workflows here?

Yes [I explicitly spelled out desktops & laptops] but it's been true on my servers too.

> If this is a service, why are you paying for RAM that never gets used? Get a > smaller instance.

Not all providers are so flexible. I may want a server with more CPU & traffic and some additional disk space. Going there gets me more RAM too. It turns out these services are constrained by the real hardware. If I'm getting a box with enough CPU for my needs, well they are not going to a shop to buy that special box for me, they use what they have and what they have comes with plenty of RAM too.

</comment></comment></comment></comment></comment><comment><USER47> In defence against swap on my personal computer:

-PCs have a lot of RAM now

-When you allocate that much memory it's usually a bug in your own code like a size_t that overflowed. I never saw programs I would actually want to use try to allocate that much

-When using swap instead of ram, everything becomes so slow that you're screwed anyway. The UI doesn't even respond fast enough to kill whatever tries to use all that memory.

-How common is a situation where you need more memory than your ram size yet less than ram+swap size in a useful way? Usually if something needs a lot, it's _really_ lot (and as mentioned above not desirable)

-Added complexity of making extra partition

-Added complexity if you want to use full disk encryption

-I do the opposite of using disk as ram: I put /tmp in a ramdisk of a few gigs

-Disks are slow and fast ssd's are expensive so you would't want to sacrifice their space (maybe if this changes some day...)

<comment><USER48> Exactly! Also, the person who goes from 32G of RAM to 256G of RAM is going to run without swap.

<comment><USER49> I have 4GiB swapfiles on the cluster nodes I manage, which have 512GiB RAM.

It's hardly used, around 300MiB at present, probably things like the mail daemon. It's been useful to have a very slow node, which I can SSH into (after 10 minutes) and kill a chosen process, rather than a dead/OOMed node. But I think the difference is marginal, and perhaps 512MiB would have been a more appropriate size for the partition.

(Swappiness is set to 1.)

<comment><USER48> We run without swap, and haven't noticed an issue. Did you see any practical benefit to running swap vs not?

<comment><USER49> Other than a couple of occasions where things have run slowly, giving me time to kill the process _I_ choose, I've not noticed any benefit.

Picking a random machine, there is 600MB of swap used. "top" shows where about 50MB is used (Hadoop daemons, systemd bits) but I don't know what the rest is. I guess it could backfire, since logind is swapped out, and I might want to log in on the serial console if the machine is very busy.

</comment></comment></comment></comment></comment><comment><USER50> The system should be tuned so that under excessive pressure from requests it starts to turn requests away before running out of RAM. Having a small ammount of swap will let you get closer to the limit of RAM use without risking OOM killing something and getting the system into an undefined state. Also you can swap a fair bit of the system you don't use and leave it there, giving you more RAM to use for processing requests.

</comment><comment><USER51> All of this boils down to "because buying more disk is cheaper than buying more RAM" and "avoid memory contention."

The author discusses the situation as if the quantity of RAM is fixed and swap can be added (or not). But that isn't the only possibility — you can also add more RAM (it's just expensive). For the same number of GB of RAM+swap vs just RAM, there is no reason to prefer the option with swap.

<comment><USER33> Sure, but why wouldn't you run some swap with your much bigger ram anyway?

In the end the core idea is: sometimes you have anonymous memory that is accessed so rarely that you'd rather have an extra disk cache page. If you assume that the kernel is not paging out memory that you actually use when not under pressure, swap doesn't hurt you.

<comment><USER46> > Sure, but why wouldn't you run some swap with your much bigger ram anyway?

If you don't need it, you don't need it. The other question is: how much swap exactly should I have? And why wouldn't I just add that much RAM instead?

> In the end the core idea is: sometimes you have anonymous memory that is > accessed so rarely that you'd rather have an extra disk cache page.

That's the theory. In practice I always have more than enough RAM for all the cached pages the system wants to cache. On my laptop right now (booted today), I have 500MB of cached pages and 5.8 gigabytes of free memory. On my server (booted 499 days ago) I have 700MB of cached pages and 6 gigabytes of free memory.

If I were running out of memory [be it for cache or applications], I'd prefer to have more RAM than add swap. Yes, I keep calling it emergency memory.

> If you assume that the kernel is not paging out memory that you actually use > when not under pressure, swap doesn't hurt you.

1) Bad assumption 2) it doesn't help you either, so why bother? Actually I might have a use for that disk space. In that case the swap just hurts.

<comment><USER33> >If you don't need it, you don't need it. The other question is: how much swap exactly should I have? And why wouldn't I just add that much RAM instead?

Because adding RAM costs money, adding swap space is a config setting.

It looks to me that your boxes are idle. Sure, if you're not doing any work, it doesn't matter...

>1) Bad assumption 2) it doesn't help you either, so why bother? Actually I might have a use for that disk space. In that case the swap just hurts.

1) Is it? Can you source that claim somehow? 2) It does help you, that's the point. It makes your disk reads faster 3) Your answer to needing memory is 'buy more ram', but you 'may have a use for that disk space'? Buy more disk.

<comment><USER46> > 1) Is it? Can you source that claim somehow?

Only my anecode. I've had systems become annoyingly sluggish because the OS decided I no longer need something and paged it out, even tough I had plenty of RAM. Turns out I needed that something.

> 2) It does help you, that's the point. It makes your disk reads faster

I just gave my numbers. The systems are not caching nearly as much as I have ram. These numbers come from systems I use every day; they are not idle.

> 3) Your answer to needing memory is 'buy more ram', but you 'may have a use > for that disk space'? Buy more disk.

Why are people so hell bent on telling me I should use swap that doesn't actually help me at all? Yes I buy as much as disk as I need, and I'm not putting unnecessary swapfiles or partitions on them. Yes I also buy as much RAM as I need.

The only real justification I see for swap here is that it's cheaper -- poor man's RAM. I call that emergency memory for when you can't have enough RAM. If I have enough memory, swap is completely pointless.

</comment></comment></comment></comment></comment><comment><USER52> My use case for swap: 10 dev environments on a cheap-ass AWS server. Where each environment is about 5 docker images.

They are slower, but to give every branch a fully usable test system is pretty awesome. No reason to pay through the butt for RAM for tier-1 dev environments. You can also have a premium dev environment for the develop branch on a different server.</comment>


Understanding caching in Postgres - <AUTHOR> http://madusudanan.com/blog/understanding-postgres-caching-in-depth <comment><USER2> One thing I've always wondered about: It seems that Postgres is designed to use the operating system's capabilities in terms of disk caching, I/O scheduling, Multitasking etc as much as possible, whereas most other databases take the opposite approach and try to bypass the operating system as much as possible, sometimes even to the point of bypassing the file system and doing raw disk I/O. AFAIK something like Oracle is almost an operating system by itself.

What lead to that design decision and what are the tradeoffs from it? Intuitively it seems like it'd be better to bypass the OS, but on the other hand Postgres does work incredibly well.

<comment><USER3> Generally speaking, bypassing the operating system I/O scheduler and cache will substantially increase the effective throughput of the database kernel. The reasons why are somewhat complex and technical (e.g. it is prerequisite to implementing a good approximation of Bélády's optimal algorithm) but this fact is not controversial. The reason for doing your own task scheduling has similar theoretical foundations in the abstract though it manifests differently.

The tradeoff is that it _greatly_ increases the complexity of maintaining portability and consistency because every operating system and file system behaves differently and offers different guarantees at this level of abstraction. PostgreSQL makes a priority of being highly portable, and has therefore chosen to avoid this particular code swamp and testing nightmare altogether thus far even though it comes at a significant cost to peak performance potential.

PostgreSQL is quite efficient and well designed for the design constraints they currently operate within but their primary value and philosophy is not peak performance per se.

<comment><USER2> Thanks for that - the tradeoff between performance and complexity in this case is clearer now.

It's interesting how many different approaches there are when stepping back and using some of the OS - for example where does MySQL/InnoDB fit in this?

It seems they are intending you to use their buffer pool rather than the OS's disk cache (as they advise to make the buffer pool take large percentage of RAM), as opposed to Postgres where they recommend having shared_buffers as ~20% of RAM and leaving the rest for the OS to use. Is InnoDB doing something different here or is it just because they have a better cache implementation?

</comment></comment></comment><comment><USER4> 1 CPU cycle is not really comparable to 1 s. A thougt that you can think of in 1 second can be extremely complex.

Nitpicking? neh dont think so.

<comment><USER5> I think the goal was to establish a base from which all the other comparisons could be reasonably drawn. Humans don't generally think in fractional seconds.

<comment><USER4> No but a human thought does not represent a numeric operator, it represents a very very complex state. It would take megabytes probably even perhaps gigabytes to represent a thought on some level.

<comment><USER6> It takes an awful lot of data to fully capture the instantaneous state of a superscalar processor, too. Instruction scheduling, register renaming, cache lookups, branch prediction, speculative execution, and so on.</comment></comment></comment></comment>


What is next when the hype around data science and deep learning fades away? - <AUTHOR> <comment><USER2> distributed ledger, replacement of intermediaries</comment>


Ride-hailing's NYC dominance, and the impact of DeleteUber - <AUTHOR> http://toddwschneider.com/posts/analyzing-1-1-billion-nyc-taxi-and-uber-trips-with-a-vengeance/#update-2017 <comment><USER2> 7 years ago you'd have to jump into a yellow cab before telling the rider if you were going from Manhattan to Brooklyn or Brooklyn to Brooklyn. If you told them before you got in, they'd speed off.

You used to have to walk to arterial roads in hopes of finding a yellow cab (in the outer boroughs). That is if you lived in neighborhoods that the article points out now skews towards Lyft. Well to do, post gentrification areas. If you didn't live in those neighborhoods, forget finding a yellow cab.

If you didn't find a yellow cab, you call a car service like Arecibo or Eastern. You say "73 Atlantic Ave, at Hicks st.", they say "10 minutes" and hang up. 25 minutes later a beat up town car with an illegal tint job pulls up, and you walk to the passenger door and knock on the window, or, if the window no longer works, open the rear passenger door -- but you don't get in. Instead you say the neighborhood you are going and the guy sort of waffles and says... 20. You say, forget it and go to close the door (but you don't), he says 18. You say 12, he says 15. You get in the car. Last week you paid 10 for the same ride.

The last time I called an UberT, a few years ago, the green taxi pulled up and someone ran over and threatened to fight me since they'd hailed the cab first. That's the last time I messed around with the old way of doing things and it sounds like a lot of New Yorkers feel the same way.

<comment><USER3> I had to make a throwaway as well just because I couldn't believe this comment. This is so utterly false. You're essentially claiming that the value of these services is all based on the innovation of the _hailing_ functionality, but there are countless apps that recreate this functionality _exactly_ whether for 'regular' taxis or separately licensed cars. If what you want is for a driver to come to your exact location very quickly, you can get that _anywhere_. This, as a business model alone, is like Dennis on 30 Rock with his basement coffee business. It's so easy an ubiquitious that it's not worth mentioning as a differentiator _at all_.

Separately from this, it ignores the fact that there are cost-based reasons for these kinds of behaviors. Drivers, for example, have to think about round- trip cost. If you ask me to drive you from Washington Sq. Park to Astoria, I have to think about what fare to charge you such that it covers my expenses on the longer trip _as well as_ the likely problem of not picking up a return fare that takes me back to a high-traffic area.

This is all discussed with much greater statistical depth in the Naked Capitalism articles on Uber, but the basic problem is that none of the ride hailing services have found a way to make this any cheaper, except through massive subsidy (to pay drivers to be the ride stock in low-profitability zones they others wouldn't be in). And even with an innovation like self- driving cars, it would only make a difference if it is not a commodity. If it is a commodity (which, IMO, is the overwhelmingly most likely outcome for self-driving cars), then either Ubers completely captures all relevant IP for end-to-end commercial solutions ahead of any competitiors, or else Uber is licensing out the cars just like everyone else, and for whatever cost reduction it buys them for getting rid of drivers, it just creates a new price war race to the bottom on who can charge the least for the self-driving based rides.

I'm just so tired of all the mental gymnastics defending them. It really is total ownership of all IP for commercial self-driving cars or bust for this business model.

And if it busts, you'll see that all the niceties you describe in this comment are mostly the product of either (a) massive investor subsidy, or (b) easy and cheap-to-copy innovations like a map and location-enabled hailing app.

<comment><USER2> > This is so utterly false. You're essentially claiming that the value of > these services is all based on the innovation of the _hailing_ > functionality, but there are countless apps that recreate this functionality > _exactly_ whether for 'regular' taxis or separately licensed cars.

There wasn't before Uber showed up. Last time I took a Juno, the driver asked me if he could text me a promo code so I could send it to me friends, that's when I stopped shopping around

> Separately from this, it ignores the fact that there are cost-based reasons > for these kinds of behaviors. Drivers, for example, have to think about > round-trip cost. If you ask me to drive you from Washington Sq. Park to > Astoria, I have to think about what fare to charge you such that it covers > my expenses on the longer trip _as well as_ the likely problem of not > picking up a return fare that takes me back to a high-traffic area.

It's illegal in NYC for a yellow cab to refuse a ride request to any of the five boroughs. As for the livery cabs, the pricing was inconsistent for the same trips, over and over. Doubt any of it had to do with anything other than maximizing fare, which is their prerogative, since there there are no checks in place for that kind of behavior.

> This is all discussed with much greater statistical depth in the Naked > Capitalism articles on Uber, but the basic problem is that none of the ride > hailing services have found a way to make this any cheaper, except through > massive subsidy...

I understand this. It doesn't invalidate the sea change in the experience of moving around NYC, particularly in the evening and late night.

> And if it busts, you'll see that all the niceties you describe in this > comment are mostly the product of either (a) massive investor subsidy, or > (b) easy and cheap-to-copy innovations like a map and location-enabled > hailing app.

Let's hope (b). I doubt these innovations would have spontaneously arose from the taxi commission or the livery cabs.

<comment><USER3> > It's illegal in NYC for a yellow cab to refuse a ride request to any of the > five boroughs...

Yes, of course. But the reason you don't see Uber drivers doing the same thing (or outright avoiding these unprofitable trips) is that they are subsidized to provide that unprofitable ride stock in areas where it's intrinsically not economical to provide ride stock. When you compel a yellow cab to do it, it means the prices have to be higher. This is why it makes less sense to compare the Uber experience to the yellow cab experience. It's like saying, "Would you prefer if a celebrity donated their first class Emirates seat to you, or a coach seat with United?" It ignores the mechanism by which the experience can be afforded to the end user (in Uber's case, huge subsidy, with no sign that the prices could be sustained absent the subsidy).

Overall I understand your comment and I realize you are aware of the counterpoints I'm making. The thing for me is that highlighting the good experience actually _is not a useful point of comparison_ with older, crappier cab experiences, specifically because everything we experience with Uber right now is inherently part of this huge ambient subsidy-based perception distortion field.

<comment><USER2> Ultimately no one did anything about the experience I brought up in the OP until Uber. This isn't an endorsement of Uber, more of a statement of fact. That anyone can now avoid all the old PITA of moving around NYC is good.

I generally don't consider the long term sustainability of the companies I purchase goods from. I do try to apply a vague and probably non-uniform ethical filter to them.

That said, I'm a software developer and my very generous salary is the indirect (via the demand curve) result of the destruction of manifold entrenched business models at the hands of VC subsidies. If I spent all day hailing yellow cabs it wouldn't make a difference.

<comment><USER3> > I generally don't consider the long term sustainability of the companies I > purchase goods from.

We could probably go on at length about the moral problems here, but likely not productive in a comment thread.

> That said, I'm a software developer and my very generous salary

I'm not sure how to read that. I am also a software engineer, and even very high salaries in wealthy urban zones are typically not 'generous' in the sense that good software labor creates a vast wealth surplus for the shareholders and ownership, to such a degree that the high salaries are still not very fair.

> the indirect (via the demand curve) result of the destruction of manifold > entrenched business models at the hands of VC subsidies.

Sure, but usually companies are expected to demonstrate evidence that the subsidy wouldn't be _permanent_ .. that there is evidence of a path to profitability.

The reason it's worthwhile to draw much more attention to the VC subsidy situation with Uber is that they have not been able to do that, and have instead generated unprecedented losses without making any progress towards profitability. It makes it incredibly different than typical cases when VC subsidy creates a short term runway to profitability. That is not what's going on.

<comment><USER4> Perhaps I shouldn't have been, but when Dara Khosrowshahi got put in as CEO, I was a bit surprised that part of his mandate wasn't to adjust fares to the point where is was a sustainable business and, if that turned out to be impossible, turn off the lights. I know things like self-driving get held up as some sort of magical pixie dust coming RSN, but close your eyes and hope a miracle happens doesn't seem the best approach to business strategy.

<comment><USER3> Rather than 'turning off the lights' Uber is surely planning to find some means by which they can use hype to drive their valuation higher in preparation for an IPO.

When a company in this situation goes to IPO, it is generally a tool for the existing investors (who know that their investment is not worth the perceived, hype-driven valuation) to externalize the losses out onto unwitting regular folks who passively come to own Uber stock through some random 401(k) manager or speculative day traders.

Uber's IPO will inevitably be a massive wealth transfer from regular folks to Uber investors, who will get good returns on the original investment (because they will be able to sell their stock at the hype-inflated prices), followed by a massive stock correction lowering the price to the true valuation that correctly prices in this subsidy effect and the strength or weakness of the business model.

After that correction, the price movement will reflect aggregated beliefs about whether Uber can innovate and find solutions for these problems. But the problem is with that initial externalization of losses from the IPO. Once original investors and early employees are paid out of the proceeds of that, they are free to leave and don't need to care about the long-term profitability of Uber, which in turn means that as long as they can keep levering up the valuation through hype now, they don't need to fix any of the fundamental problems (and, sadly, they are acting rationally, unless people find ways to ensure their portfolios do not come to include Uber after the IPO, and refuse to buy until they wait out the correction).

<comment><USER4> Your doubtless correct. To the degree there's a trade off between management and investors getting the max cash-out and Uber operating as a profitable business we all know which will take priority.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> So since this information was so hard to pull out of the article, I've summarized it according to the weekly ride-share dominance chart[0]. Pre #DeleteUber, Uber sat at about ~75% of the weekly rideshare dominance. Currently, Uber sits at about ~68.5% of weekly ride share dominance. While it sounds like there were about half a million people that requested their account be deleted, at least in NYC (remember these charts are JUST NYC ride hailing) the campaign has had a negligible impact.

Judging by the demographic of people that protest, and are protesting on twitter, I'd be more curious about judging the impact of #DeleteUber by looking at ride-hailing rides around epicenters of people younger than 30. Fortunately though, he covered this also, touching on Lyft's growth in Brooklyn. I'd also be curious about ride hailing stats around universities too.

[0] [http://cdn.toddwschneider.com/taxi/update2017/ridehail_marke...](http://cdn.toddwschneider.com/taxi/update2017/ridehail_market_share.png)

</comment><comment><USER6> This is one of the more interesting things that I learned from the article:

" _Uber's share of all ride-hailing trips has generally declined since 2015 as more competitors entered the NYC market, even as its total number of trips has increased dramatically._ "

That would have me worried, if I was an investor in Uber.

<comment><USER7> I think in general investors are aware that Uber will need self driving cars to stay relevant.

Transportation is going to become a monthly flat rate like a cable bill for local travel via autonomous vehicles and the winners will be the companies that own the fleets and get to charge the tolls.

Owning your own car will be a luxury item.

Owning a gasoline powered car will be an extremely niche luxury item.

Obviously this won't happen tomorrow but it's not hard to see the parallels to the horse and buggy industry in this transition.

New players will emerge to be the big names in this new industry and Uber investors hope it will be one of the winners.

<comment><USER4> That would seem to be a very urbanite view of the world. A lot of people have good reasons for owning their own personalized vehicle. (Gear, kids, mobile storage locker, etc.) So long as you live somewhere with enough space that storing a vehicle isn't expensive/difficult, given that most of the costs are in mileage, it's hard to see most people giving up ownership.

And, in any case, full door to door autonomy, in at least most locations, is likely a long ways off. Certainly outside of the relevant horizon for Uber investors.

<comment><USER8> Kids is a big thing -- needing specific car seats means taxis are out until your kids are 12.

That said the average car costs $34k in the US and lasts say 10 years, that's $3,400 a year. Maintenence, taxes, etc another $1500 at least, that's $5k/yr before you move an inch.

While automatic cars may not get rid of everything, many 2 or 3 car families, especially in the suburbs, may drop to 1.

<comment><USER9> Normal car expenses are around $0.55/mile for an average car. The average joe pays about $8,000-$8,500 annually for all car related expenses. Some pay more for fancy stuff or utility.

That's why suburbs exist. Financially, it's a much better deal for families on just about any axis.

<comment><USER8> With an average 13-15k miles a year, at 20-30mpg that's about 400-750 gallons a year, or $1000-$2000 a year in gas, leaving the vast majority of expense in deprecation and maintenence.

<comment><USER4> >leaving the vast majority of expense in deprecation and maintenence.

Depending on the car, gas is probably between about 20% and a third of a car's operating costs (which are typically in the 30 cent to 60 cent per mile range). Although there are a few vehicle costs, like excise taxes, that are more or less independent of distance driven, most car expenses will scale with mileage--especially in regions where rust isn't a major issue.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> In NYC, Juno is cheaper in most circumstances, and you get better drivers. Otherwise, Via or Lyft. There are so many options instead of cludgey old Uber.

<comment><USER11> Thankfully, Uber exists. Otherwise I'd have to download and register on all sorts of dinky city-specific apps any time I travel.

Uber, for all of its faults, is the only one that works in every city I've traveled to.

<comment><USER4> I won't say it's _vanishingly_ small but, relatively speaking, there aren't a huge number of people who have a problem with needing taxi services in a bunch of different cities. I travel about 100 days a year and I don't use these services very much and I don't think I've used Uber in a year or two. Worldwide availability of a brand just doesn't seem like a huge deal.

<comment><USER12> There are some places where brands like Uber is extremely valuable, like Manila, where taxis are very apt to rip off free-off-the plane foreigners. Heck, most of SE Asia (there are local services also, but I'd rather not track them down before each trip).

<comment><USER4> That may be the case but my point was that hopping off-the-plane in SE Asia and grabbing a cab is not such a routine occurrence for most Westerners as to be a viable business model for VC-backed companies.

<comment><USER8> Outside of america, how many people routinely travel by taxi/rideshare/etc. In my experience, most locals take public transport. The international businessperson (on expenses) is a large market -- look at the size of the market for hotels, airlines, etc.

The one place I see taxis in use as mass transit by normal people is Singapore, where they are mostly ubiquitous (in the centre) and cheap. Even then getting a taxi if you're a couple of miles out (say in Hougang, where I spent a month) is hard work - Uber wasn't perfect but it worked well enough.

<comment><USER12> China: taxis and ride sharing is extremely popular with the middle class. "Most locals" do take public transit, but given 10 million people in Beijing alone, some is still a large number.

Most developing Asian economies have thriving taxi industries where taking a taxi isn't considered a niche service/extreme luxury. Heck, taxis were never that popular in America, nowhere near how they were popular in China in even 1999.

<comment><USER8> I took a taxi in beijing a few years ago. Was in it about 40 mintues, traveled about a mile. Paid the driver a 20 Yuan note (about $3) and went into the hotel. Driver came running after me with 8yuan in change.

I assume taxis are still insanely cheap, and traffic is still insanely bad. Last time I was there I took the train and metro. I do the same in Moscow - why sit in traffic?

<comment><USER12> Have you seen Beijing's subway? I travelled to and from work off hours, so I would go to work around 6:30 and come home around 3:30-4, taking the taxi to and from each day was totally feasible and affordable. Even in peak traffic, you still might win in a taxi vs. a subway depending on where you are going. When the whole system melts down, sure, take the subway as a last resort.

<comment><USER4> I took Beijing's subway around quite a bit last summer. At least where I was going, it seemed preferable to sitting in traffic. Payment seemed easier to navigate as well compared to the prior time I visited. I'd have said it was a pretty decent subway system overall.

<comment><USER12> I think if you are exploring the city during rush hour, sure subway makes sense. If you have a fixed route you go between everyday and know when the traffic ebs and flows, taxis are much more convenient. If you are out at night after rush hour, then taxis are the obvious choice.

Beijing's subway system is crowded, some routes are super crowded (line 10, ugh), interchanges can be long and windy, and can also have their own long lines (e.g. transferring at dongzhimen during rush hour). There are a lot more lines definitely then when I started living there (2007, line 10 didn't even exist yet), but there are very good reasons didi is still extremely popular.

<comment><USER8> The system can be crowded, but not as crowded as Tokyo or even London IME (very limited experience of course)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> See, I guess I'm funny about cabs. For my lifestyle, I mostly ride the MTA - buses, subway, etc. I plan based on that cost and timing. When I've had an interview and needed to take a direct above ground route, I always take a yellow cab. I also make it very visible to the driver that I'm photographing the driver's number and name before I enter the vehicle. I don't do Uber/Lyft because I don't trust their consumer protections as much as I do those from the City or the T&LC.

Of course, I also have a prepaid phone & I have no desire to waste my data on hailing a cab. And I prefer to pay in cash where possible.</comment>


Show HN: Abstracting APIs with ES2015 Proxy – promisifying Chrome and Node - <AUTHOR> https://github.com/slikts/promiseproxy <comment><USER2> Nice job! I like how lightweight it is.

I've recently built this wrapper for chrome apis for ClojureScript developers: [https://github.com/binaryage/chromex](https://github.com/binaryage/chromex)

It converts callbacks to core.async channels (which is a popular clojurescript async primitive) also it uses chromium sources to generate whole api-index as clojure data structures and then uses macros to generate wrapping / marshalling of API calls. That way I can also retain API versioning and warn API clients when using wrong version / signatures.

<comment><AUTHOR> I wasn't anywhere near as fancy and just collected the callback methods from the Chrome docs with a snippet pasted in browser console.</comment></comment>


Ask HN: Deleted FB account before GDPR, now what? - <AUTHOR> Where can I request them do delete everything they got on me? <comment><USER2> Recreate your account using same email but with a different password, then file a formal erasure request on newly created account. That will result in deletion of both old and new accounts due to how Facebook's shadow profiling works.</comment>


Introducing: Podcast Feed Creation and Publishing - <AUTHOR> https://feed.press/blog/2016/08/15/introducing-podcast-feed-creation-publishing/ <comment><AUTHOR> Product Manager here from FeedPress. We've been quietly working on this for months. TLDR: to get setup for podcasting before, you had to add an RSS feed you already owned so we could fetch the contents and give you analytics. Now, FeedPress acts like a mini CMS. We create a new RSS feed for you and provide a post management system so you can create your show notes, upload your files, and publish straight to iTunes. Check out the announcement and you'll find a promo code at the bottom.</comment>


Amazon Mobile Redesign - <AUTHOR> https://www.dropbox.com/sh/wblhgjb1mvwsl4e/AACkB0ffaxaT9Ly6rI_tP6Gla?dl=0 <comment><USER2> What's the context? Is this a leak, a live production change, or a fan-made demo?

</comment><comment><USER3> Looks like it's real and it's live in US, but they might be A/B testing because not everyone is getting it</comment>


Show HN: Exercise from Home, Together - <AUTHOR> https://virtualgym.run/ <comment><AUTHOR> The basic idea is to connect people and provide a platform where they can train together, from home. The platform offers a collection of workouts with exercise illustrations. Users can create rooms and share the link with friends. Inside the virtual room, the users are connected via a peer to peer video chat and the exercises displayed are synchronized. Virtual rooms currently have no user limit (bandwidth of course :) ).

<comment><AUTHOR> Any feedbacks/suggestions are appreciated. Please be patient with the slowness of the site, we are using free servers and database, but with a bit of luck we will improve soon.

<comment><USER2> This seems silly, but the use of that emoji as your logo really turned me off. I have been trying to figure out why in order to give you better feedback, but I can't.

<comment><AUTHOR> We are still improving the platform and we will change the logo. Thank you for your feedback</comment></comment></comment></comment>


Former Wired Editor: We Could've Been Google - <AUTHOR> http://www.observer.com/2008/former-i-wired-i-editor-we-couldve-been-google <comment><USER2> Mr Former Editor, I knew Google. Google was a friend of mine. Mr Former Editor, you're no Google.</comment>


Checking In Is the Least Popular Smartphone Activity - <AUTHOR> http://mashable.com/2011/09/06/location-based-services-unpopular/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Mashable+%28Mashable%29

<comment><USER2> I've been around this LBS rodeo a couple of times and imho, it has more to do with people in your circles not carting about your activities and you knowing the futility of your check-in. There is a very quantifiable value derived from checking in, and only then it must address at least one of the capital vices, otherwise it's just cute, filler info that keep the heart beating.

Dens and Naveen have really ridden this horse into the ground and while I appreciate them for trying, the truth is that their idea(s) don't scale across social verticals.</comment>


Chess rating system provides the best predictor of World Cup success - <AUTHOR> http://kaggle.com/blog/2010/06/14/quants-pick-elo-ratings-as-the-best-predictor-of-world-cup-success/ <comment><USER2>
      Are today's crop better players?
      Or like the economy, are Elo ratings subject to bouts of inflation?

Players leaving the system is one of the major sources of rating inflation in chess. Even when rating changes of any single game is zero-sum, if a player stops playing chess after a career of net losses, all the remaining players in the world have more rating to divide up amongst themselves.

This is very obvious on FICS bughouse where new sock puppet accounts crop up all the time, net lose several games, and then leave forever.

Part of the problem is new players are usually given a starting rating around 1600 (depends on their provisional games), a "median" ranking of sorts, but in reality a new player is often far worse and donates rating as they slide down to their correct rating.

<comment><USER3> The last issue that you raised (new players donating ranking) has long made sense to me. However, I had not considered inflationary pressure due to players _leaving_ before. Is the inflation really from their departure, or is it purely explained by their entrance, and loss of points while still playing?

<comment><USER4> Players leaving is a problem only if they lose points and then leave without winning them back.

</comment></comment></comment><comment><USER5> This looks like complete arse to me. 30 international games can span a 5-6 year period, during which time a completely new crop of players can emerge. Where does form come into the equation?, England's performance the other night was dismal, while Germany looked like potential winners. I've no clue how Mexico made the top eight, perhaps because they're based in Central America, and the competition isn't that stiff? And winners of the previous world cup, Italy, in behind them at number 8??? ... get real!!

<comment><AUTHOR> Good point on the 30 international games.

</comment></comment><comment><USER3> Sweet. I love Elo points a lot (so much so that they are how my startup ranks U.S. colleges: <http://college.mychances.net/college-rankings.php> ).

The author poses the following question: "[...] like the economy, are Elo ratings subject to bouts of inflation?" Since the amount of points available depends on the number of individuals participating, Elo points should indeed be subject to inflation.

<comment><USER6> Cool site. I knew Harvard's yield was higher than the other schools, but I didn't expect Stanford and Yale to have higher yields so much higher than Princeton. I don't think this was true a few years ago when I was applying to schools.

How are you gathering information about where students were accepted? And where is Caltech in all this?

Also, you know that you can't just add the 25th/75th percentiles for individual SAT sections and come up with overall results, right?

<comment><USER3> Thanks. Re: Caltech - if I recall correctly, there wasn't enough data for it to produce results when I ran the algorithm last year. I suspect we'll have enough this year (I'll probably re-run the algorithm in a few weeks).

Basically the entire site is dedicated to gathering information from students, and then transforming that data into something useful for them. So I gather the info directly (over 250,000 college apps from 50,000 different people are in the system by now).

I honestly have no idea why I thought I could get away with adding together the 25th and 75th percentiles for SAT subscores on that page; it's not something I do elsewhere on the site. (I'm using "I" here since I'm sure this is my fault alone.) Thanks for pointing that out!

Edit: I forgot about Princeton. The Elo points can be a bit hard to decipher on the fly; for example, Yale should only beat Princeton in 59:41 fashion - not exactly a drubbing. Here's an Ivy+Stanford+MIT cross-admit comparison based on this data - [http://college.mychances.net/tools/college-choice- matrix.php...](http://college.mychances.net/tools/college-choice- matrix.php?list\[\]=342&list\[\]=544&list\[\]=3&list\[\]=1737&list\[\]=1519&list\[\]=165&list\[\]=1034&list\[\]=319&list\[\]=762&list\[\]=361)

</comment></comment></comment><comment><USER7> Table tennis uses a similar system. My wife used to play competitively, and people at the club would always ask each others' ratings when looking for a good match.

Of course, a single number can only tell you so much -- it's a very gross generalization of skill. A lower-score player can have a particular strength corresponding to another's weakness, thus leading to a win for the (generally) weaker player. I imagine that this is similarly true for soccer teams.

<comment><USER8> > _"Of course, a single number can only tell you so much -- it's a very gross > generalization of skill."_

In a system where new players can enter after competition has begun (and others have accumulated points), there's also a good chance that a new player will not yet have had enough time to accumulate a score that's in line with their actual skill, which can lead to some misleading matchups. Depending on the details of the ranking system and the ease of finding similarly-ranked partners, it can take a long time to converge to a "true" rating.

For example: I had friends in college who played competitive card games that had some sort of lifetime ranking (Bridge, perhaps?) Because they were young, they were often the lowest seed in tournaments. Despite above-average skill, they were normally eliminated by the very most skilled teams in the first round, and therefore had few opportunities to accumulate points against more equal competition. Once they finally broke through and won a couple of tournaments, there was a bit of a snowball effect as they moved out of the lowest seed.

</comment></comment><comment><USER9> A difference of two points in Elo is nothing; basically this corroborates the current betting odds that Brazil & Spain are the two favorites with small differences between them.

In any case football/soccer results have such a big variance that it doesn't really matter. I think in HK there was a paper no long ago that studied soccer competitions and sayed something like the best team in a soccer competition only wins it a 30% of the time.

<comment><USER10> A big variance is going to occur in any direct elimination competition (which the world cup is after the group stage).

(This has always bugged me regarding the baseball playoff system. It is a big boost for teams economically but for purity I would prefer to see just a NL and an AL league table).

</comment></comment><comment><USER11> So is there a way to compute the probability of one team beating another, given their Elo scores?

<comment><USER3> It depends on their exact formula. Roughly, though:

Probabilty of Team A beating Team B, given that Team A has 'A' Elo points and Team B has 'B' Elo points: 100/(1+10^((A-B)/400))</comment></comment>


Do We Need Zero Training Loss After Achieving Zero Training Error? - <AUTHOR> https://arxiv.org/abs/2002.08709 <comment><AUTHOR> This technique is called "flood loss," and it's one of the most underrated papers in ML due to its implementation simplicity: [https://twitter.com/theshawwn/status/1253505386172145666](https://twitter.com/theshawwn/status/1253505386172145666)

Loss getting too low?

      loss = abs(loss - x) + x

where x is a value like 0.2.

Presto, your loss is no longer <0.2.

We have some pretty shocking screenshots of before/after using this technique for our BigGAN training runs.

Before flood loss, no progress: [https://media.discordapp.net/attachments/696383989145010216/...](https://media.discordapp.net/attachments/696383989145010216/702832115162808340/individualImage.png)

After flood loss: [https://media.discordapp.net/attachments/696383989145010216/...](https://media.discordapp.net/attachments/696383989145010216/702831280634462228/individualImage.png)

Both of these are screenshots from around step ~3k, about an hour into training. As you can see, flood loss improved things immediately and dramatically.

Other people are reporting that it's stabilizing their runs as well, but that's hearsay for the moment.</comment>


Murray Gell-Mann Has Died - <AUTHOR> https://www.nytimes.com/2019/05/24/obituaries/murray-gell-mann-died-.html <comment><USER2> "The first thing that makes me respect Murray is that unlike all his contemporaries, including Feynman, Weinberg, Hawking, and all the other particle physicists, he saw that complexity is the next big problem. The kind of breakthroughs he made in the early 1960s in terms of impact on the world of science are not going to get made in that domain, they are going to get made in this domain. Murray recognized that, and has become more than just conversant with what's going on and with what the problems are." \- J. Doyne Farmer [0].

Last month, there was a neat discussion of Murray Gell-Man's idea on plectics: the study of simplicity and complexity [1]. He definitely influenced my views on systems theory. Thank you and rest in peace.

[0]: [https://www.edge.org/conversation/murray_gell_mann- chapter-1...](https://www.edge.org/conversation/murray_gell_mann- chapter-19-plectics).

[1]: [https://news.ycombinator.com/item?id=19760682](https://news.ycombinator.com/item?id=19760682).

<comment><USER3> I'm willing to bet that the Gelllman Low equation outshines his other work in the long term.

</comment></comment><comment><USER4> These video series with Gell-Mann are super interesting in my opinion -- [https://www.youtube.com/watch?v=3LU6kbao3vo](https://www.youtube.com/watch?v=3LU6kbao3vo)

</comment><comment><USER5> He gave an amusing talk at Google in 2007 about creativity[0], where I thought he talked about one of the important things in science is actually believing the data since it seems so outlandish. I thought it was in this talk, but skimming the video I can't seem to find it. The talk is quite amusing regardless.

[0] [https://www.youtube.com/watch?v=e7M2l-jzRG8](https://www.youtube.com/watch?v=e7M2l-jzRG8)

</comment><comment><USER6> NYT obituary: [https://www.nytimes.com/2019/05/24/obituaries/murray-gell- ma...](https://www.nytimes.com/2019/05/24/obituaries/murray-gell-mann- died-.html)

</comment><comment><USER7> Am I right in thinking he named the Quark - he was reading Finnigan's Wake at the time.

I wonder if he finished it...

</comment><comment><USER8> Embarrassing confession: For 20 years thanks to my shallow physics education (i.e. for engineering) I always thought Gell and Mann were two people, like Meyers and Briggs, and (appropriately) Dunning and Kruger.

</comment><comment><USER9> A true giant. The concept of quarks really captured my mind as a kid. It was one of the first things that I recall that made me realize there was a whole world of knowledge beyond my basic textbooks.

</comment><comment><USER10> his book "The Quark and the Jaguar" included a bit about the game Oware which is highly recommended

<comment><USER11> [https://archive.org/details/MurrayGellMannTheQuarkAndTheJagu...](https://archive.org/details/MurrayGellMannTheQuarkAndTheJaguarAdventuresInTheSimpleAndTheComplex1995St.MartinsGriffin.compressed)

<comment><USER12> How is this allowed on the Internet Archive? It was only published in the 90s, it is still under copyright. I thought the Internet Archive's publicly accessible book collection was supposed to only include content in the public domain (or otherwise freely licensed).

<comment><USER13> IA began to put up out-of-print items some time ago (old LPs for example). I'd guess that many books still under technical copyright can only be reprinted in small quantities (probably at a loss) and be hard to find. Seems a reasonable 'gentleman's agreement' that's working out for IA.

</comment></comment></comment></comment><comment><USER14> [https://en.wikipedia.org/wiki/Gell- Mann_amnesia_effect](https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect)

</comment><comment><USER15> A great man has left us. RIP.

<comment><USER16> I wonder how many such monumental "brand name" physicists are still alive.

<comment><USER17> We still have Witten, Weinberg, 't Hooft, Yang&Lee. Also Nobelists but (in my utterly prejudiced view) less monumental: Rubbia, Glashow, Wilczek, Thorne. Not Nobelists but arguably in the monumental category: Dyson, maybe Penrose.

<comment><USER18> For "brand recognition" surely Peter Higgs is at the top, along with maybe Dyson. I'm not qualified to opine on how his work compares to the others.

<comment><USER19> I don't think Higgs is really widely considered monumental in the same was as those other people. He had one good idea (at essentially the same time as Brout, Englert, Kibble, ...) which got his name attached. But I am a professional physicist and know none of his other work, in contrast to Dyson, Weinberg, Witten, t' Hooft, ...

<comment><USER20> Higgs is absolutely a more familiar name than the others, and stain too, whether you think his contributions were proportional to that recognition or not

<comment><USER19> That he has greater name recognition amongst lay people doesn't make him a monumental physicist.

</comment></comment></comment></comment></comment></comment></comment><comment> grayed-down A quarky guy with a love of simplicity. Thanks for the inspiration a long time ago. RIP

</comment><comment><USER21> Sorry to hear, my acquintance with him is via the Gell-Mann amnesia effect.

"Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know."

— Michael Crichton

<comment><USER22> I like that quote of Crichton's too, but since it has nothing to do with the actual Murray Gell-Mann or his life—only his name, which Crichton made use of in a particularly arbitrary way—I think we should treat it as off topic in this thread.

<comment><USER23> Lest anyone think that dang is being uncharitable here, Crichton himself admitted as much immediately before giving his oft-quoted definition [1]:

"Media carries with it a credibility that is totally undeserved. You have all experienced this, in what I call the Murray Gell-Mann Amnesia effect. (I call it by this name because I once discussed it with Murray Gell-Mann, and by dropping a famous name I imply greater importance to myself, and to the effect, than it would otherwise have.)"

</comment></comment></comment><comment><USER24> This makes me incredibly sad, as he is one of those people that has taught me over the years in all kinds of ways. One of the most recent, and really rather unrelated to his fields, is the Gell-Mann Amnesia effect: [https://en.wikipedia.org/wiki/Gell- Mann_amnesia_effect](https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect)

Which is amusing, but also profound. Fitting it is one of the last things he taught me (not directly, of course!).

</comment><comment><USER25> I think about him often, because of the infamous [https://en.wikipedia.org/wiki/Gell- Mann_amnesia_effect](https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect)</comment>