The UW Study on $15 an Hour Does Not Reflect My Life as a Minimum Wage Worker - <AUTHOR> http://www.thestranger.com/slog/2017/07/20/25299657/guest-editorial-the-uw-study-on-15-does-not-reflect-my-life-as-a-minimum-wage-worker <comment><USER2> The gist of this editorial is, "That study said that the minimum wage hike led to businesses laying people off, but that didn't happen where I work at Domino's, we're hiring." And since she brought it up, it does seem like for certain businesses, like pizza delivery, the cost of the hike (paying their employees more money) would be offset by the benefit (Seattle's poor people having more money to spend on pizza). A business with low-income workers but no low-income customers, e.g. a company that cleans yachts, would see the opposite.

And lo and behold, it turns out that one of the main criticisms of this study is that they excluded most large employers and chains (on the theory that if a chain restaurant in Seattle hires people or fires people, it's more likely to be caused by some external effect than by the wage hike). And since OP brought it up, it does seem like the chains that this study excluded would be likely to be exactly the sorts of employers who would see more benefit than cost. That's not evidence that the study is wrong, period, end of story, but it is intriguing, and I can't find any suggestion on the study's website that they attempted to account for this.

It's easy and satisfying to dismiss things out of hand for being anecdata, but sometimes you miss interesting perspectives in so doing.

<comment><USER3> The Weeds podcast by Vox had a recent episode that talked about this and the study in detail if anyone is interested. [https://www.vox.com/2017/7/14/15964284/the-weeds-seattle- min...](https://www.vox.com/2017/7/14/15964284/the-weeds-seattle-minimum-wage- podcast)

</comment></comment><comment><USER4> "My single data point disproves a rigorously-conducted study spanning millions of workers and thousands of employers."

Come _on_.

<comment><USER5> [https://evans.uw.edu/sites/default/files/NBER%20Working%20Pa...](https://evans.uw.edu/sites/default/files/NBER%20Working%20Paper.pdf)

While I don't doubt the veracity of some of the conclusions (more cost per hour means hours get cut) there does seem to be a few what I'd consider "contortions" to get the desired result. I say "desired result" because sometimes people who fund these things, yes, even (especially?) at universities have compelling interest.

For instance, they exclude all "multi-site" businesses from the study (Dominos for instance so study doesn't even apply to author in the first place). One has to wonder what the results would have looked like had they not done that.

Not arguing for against state/city minimum wages, don't have an opinion either way. Just skeptical.

<comment><USER4> The study was commissioned by the city of Seattle after the passage of its MW ordinance - there is every reason to believe that the "desired result" was "the MW raise is good and justified and has no net negatives". That's a part of the reason that the results of this study are so surprising.

The exclusion of multi-site businesses was because of the inability to distinguish a business's physical location from its account location, which would have confounded the results:

> The data identify business entities as UI account holders. Firms with > multiple locations have the option of establishing a separate account for > each location, or a common account. Geographic identification in the data is > at the account level. As such, we can uniquely identify business location > only for single-site firms and those multi-site firms opting for separate > accounts by location. We therefore exclude multi-site single-account > businesses from the analysis, referring henceforth to the remaining firms as > "single-site" businesses.

(This suggests that a locally owned and operated Domino's franchise would be classed as a single-site business and included in the study, while a non- franchised Walmart would be classed as a multi-site business and excluded.)

Furthermore, survey evidence suggests that multi-site businesses may be _more_ likely to contribute to the "negative" effects:

> Survey evidence, discussed below, indicates that multi-site employers were > if anything more likely to report staffing reductions in the wake of the > minimum wage increase...survey evidence collected in Seattle at the time of > the first minimum wage increase, and again one year later, increase suggests > that multi-location firms were in fact more likely to plan and implement > staff reductions.

Skepticism is good, but let's give the researchers credit here: they've done a pretty stellar job of mitigating confounding variables and justifying why they've made the choices they have in which data they chose to study.

<comment><USER5> Fair enough.

For the record I'm generally opposed to government induced market distortions (but understand they may sometimes be necessary). Not for ideological reasons but because practical reality and unforeseen side effects and the cure being worse than the disease and all. Municipalities dictating a minimum wage I'd think to be probably a bad idea.

Still, excluding nearly half the workers in question from a specific segment leaves the study much less credible than it could be otherwise.

Most Dominos, WalMarts etc. have been honed for labor efficiency for decades. I suspect they can't cut staff or hours much more while continuing to operate in which case workers are (temporarily at least until wage pain causes someone to invent a pizza store machine) better off. Maybe not so much "looser" mom and pop type businesses who this will cause to tighten up.

A final point is the value of extra time. So workers are $125 less well off at the end of the month. But they are working many less hours and no matter how you slice it they are making more money per hour. With more free hours they can take part time jobs, pursue education or training which increases future value, save money by doing things like cooking and their own childcare etc. etc. So the loss of $125 per month is at least partially offset by a certain amount of value.

</comment></comment></comment></comment><comment><USER6> I, for one, am divided on issues like this and all of the other social issues of our time. In my world-view, the cause is the state (don't take that to mean society, I mean literally the state apparatus) so all of these state-sponsored fixes amount to mitigation strategies at best, not solutions.

Even having said that, mitigation is better than continued suffering. To illustrate my point with an extreme example: making it illegal to kill your slave doesn't solve the underlying slavery issue, but hopefully it would stop some slaves from getting murdered, in the meantime.

There is an inherent danger in trying to work within the system to fix it, but we're talking about real people's lives.

I think it's still an open question whether or not raising the minimum wage is the best way to mitigate the problems of the working poor, but I don't think the sky will fall if we get it wrong–it's already wrong, amirite?

<comment><USER7> > In my world-view, the cause is the state

What does this even mean?

> To illustrate my point with an extreme example: making it illegal to kill > your slave doesn't solve the underlying slavery issue, but hopefully it > would stop some slaves from getting murdered, in the meantime.

The state outlawed slavery.

<comment><USER8> I thought the same thing. Private industry made slavery. It took the state to put an end to those abuses.

And the state is also trying to put an end to wage slavery abuses.

But most rebublicans appear to be ok with getting paid nothing and working slave hours.

So I'm happy to let them elect conservatives that cut labor laws, and lower minimum wage.

A nice cheap unregulated labor force in the heartland America will lower my cost of goods and shipping on the coast.

Republicans are like a voluntary slave force for the liberal coasts.

<comment><USER6> I'm not sure why you think that, but private industry did not make slavery in America. Large royal charters and monopolies granted by the crown created huge estates and large tracts of low value farmland that could only become valuable by the use of slave labor. Sure, it was private individuals who were granted this favor and given access to cheap forms of labor, but it was not a natural outgrowth of private industry. At best it was unfortunate side-effect of the state's benevolence, and at worst, a conspiracy by the state to create wealth at the expense of an exploited class of people.

At any rate, I think you misunderstand my stance, I'm not looking to cut labor laws or lower minimum wage. I'm looking to free the worker from the shackles of the system that the state has created for them. In the meantime, if we're looking to ease the pain and suffering of those forced to live and die in that system, I think we should consider it, but think long and hard about how and if it will actually bring relief, or will it have unforeseen consequences that will make the situation even worse? That's what we should focus on.

BTW, definitely not a republican, so I think the rest of your comment can be fairly ignored by me. :)

</comment></comment></comment></comment><comment><USER9> It's a complex issue but I think one of the main points from the opposition is that with increased wages will come decreased rates of employment.

I think this point really becomes an issue for people who believe that "if you work hard, you will get what you deserve (e.g., a middle to high wage)".

For them, the tradeoff is raising the wage is for a lazy worker vs. having a lazy worker be unemployed.

The point becomes less of an issue if you believe that there are hard working people making the best of their abilities who can only obtain low wage jobs.

Then it becomes an issue of whether you believe $15 should be the minimum for someone who works just as hard as you.

</comment><comment><USER10> It's important to keep in mind that Amazon's presence has raised the average income in the city to over $90k/year. Even making $15/hour, this individual relies heavily on public assistance.

I didn't read the study so I'm not sure if this outlier was included in the overall analysis.

</comment><comment><USER11> Minimum wage raises are great, if basic living expenses don't inflate to match it.

As the author states, $15 an hour isn't even enough to get her + kids off of Section 8 and Foodstamps. Minimum wage is supposed to be the minimum to afford basic living expenses, which clearly it is not.

The fix is not in continually raising the minimum wage. The fix will only come by putting a cap on rental prices, utilities, and food staples.

Universal income is a better step the right direction, but again it will only work if there is a cap on how much housing/utilities/food is raised.

<comment><USER5> Caps sound like a very good way to get less food, utilities and housing in general.

Until such a time when the robot overlords are put in charge of distribution freeing us from the tyranny of inefficient and cruel markets.

<comment><USER11> So just keep inflating wages? Great idea if we want to keep the poor in poverty and continue living a waste-filled and over consuming lifestyle.

<comment><USER12> Minimum wage increases dont inflate wages. Most get the raise but some get laid off.</comment></comment></comment></comment>


Stanislaw Lem's Futurism (2014) - <AUTHOR> http://nautil.us/issue/17/big-bangs/the-book-no-one-read <comment><USER2> Do watch "Solaris"[0], the 1972 USSR Andrei Tarkovsky[1] science fiction film inspired by the Lem book. Even if you'ree not into science fiction . It's magnificent.

[0] [https://en.wikipedia.org/wiki/Solaris_(1972_film)](https://en.wikipedia.org/wiki/Solaris_\(1972_film\)) [1] [https://en.wikipedia.org/wiki/Andrei_Tarkovsky](https://en.wikipedia.org/wiki/Andrei_Tarkovsky)

<comment><USER3> In what way do you find the film to be magnificient? I found it to be extremely boring. I also read the book, thinking that the movie might misrepresent it, but alas, the book was even more boring. Literally nothing happens. The molasses like plot really conveys the feelings of the researchers in the book. You spend ages studying the text with barely any payout.

<comment><USER4> Many things happen. None of them are stock superhero/villain Hollywood Explodium.

If you want your science fiction to spoon feed you thrills and wisecracks, neither Lem nor Tarkovsky are going to be a good fit for you.

Edit: see also -

[https://culture.pl/en/article/13-things-lem-predicted- about-...](https://culture.pl/en/article/13-things-lem-predicted-about-the- future-we-live-in)

<comment><USER5> > Many things happen. None of them are stock > superhero/villain Hollywood > Explodium.

One can find a particular movie or book boring without being a simpleton who needs "stock superhero/villain Hollywood Explodium", so please drop the snobbery.

</comment></comment></comment></comment><comment><USER6> Here's a short story of Lem's that was only recently rediscovered and translated:

[https://przekroj.pl/en/literature/the-hunt-stanislaw- lem](https://przekroj.pl/en/literature/the-hunt-stanislaw-lem)

<comment><USER7> Thanks for sharing this!

</comment></comment><comment><USER8> Good article but I'm mighty tired of this clichéd opening to seemingly everything I read.

>on a warm September evening in San Francisco, where I was relaxing

<comment><USER9> >In math, a "singularity" is a function that takes on an infinite value, usually to the detriment of an equation's sense and sensibility. In physics, the term usually refers...

The dictionary defines "singularity" as...

>...you will find a challenge, if not a beauty; and that you will proceed in your own way after all, since in casting off man, man will save himself."

In short, the future is a land of contrasts.

</comment></comment><comment><USER10> I just finished "Memoirs Found in a Bathtub":

"Why, the Trojan Horse marked the birth of cryptoequestrianism!"

<comment><USER11> "Hey, the Building, hey! What makes the Building stay? The Antibuilding makes it stay! Hey!"

I reread this periodically and every time am blown away at the paranoiac weirdness of that one. It's a nice balance of the 'zany' Lem (Futurological Congress) and the 'serious' Lem.

</comment></comment><comment><USER12> Discussed last year: [https://news.ycombinator.com/item?id=16896880](https://news.ycombinator.com/item?id=16896880)

</comment><comment><USER13> Read "The futurology congress". That's both plain crazy and so prescient, a wonder.</comment>


Teleforking a Process onto a Different Computer - <AUTHOR> https://thume.ca/2020/04/18/telefork-forking-a-process-onto-a-different-computer/ <comment><USER2> Seriously cool. That also reminds me of DragonFlyBSD's process checkpointing feature that offers suspend to disk. In Linux world there were many attempts, but AFAIK nothing simple and complete enough. To be fair I don't know if DF's implementation is that either.

[https://www.dragonflybsd.org/cgi/web- man?command=sys_checkpo...](https://www.dragonflybsd.org/cgi/web- man?command=sys_checkpoint&section=2)

[https://www.dragonflybsd.org/cgi/web- man?command=checkpoint&...](https://www.dragonflybsd.org/cgi/web- man?command=checkpoint&section=ANY)

<comment><USER3> This article[0] on LWN seems to suggest that Linux has no kernel support for checkpoint/restore because it's hard to implement (no arguments there). But hypervisors support checkpoint/restore for virtual machines, e.g. ESXi VMotion and KVM live migration, so it seems like these technical problems are solvable. Indeed all the benefits of VM migration seem to also apply to process migration (load balancing, service uptime, etc).

0\. [https://lwn.net/Articles/293575/](https://lwn.net/Articles/293575/)

<comment><USER4> There's also a project called CRIU that is used experimentally by Docker for container save/load: [https://www.criu.org/Main_Page](https://www.criu.org/Main_Page)

<comment><USER5> You were faster than me. Yes, CRIU does checkpointing in userspace on Linux.

They contributed a lot of patches to the Linux kernel to support this feature. So there isn't any support for checkpointing in the mainline kernel. Rather, there are many kernel features that allows building such a feature in userspace. For instance, you can set the PID of the next program you spawn on Linux thanks to CRIU, because CRIU needs to be able to restore processes with the same PIDs as when they were checkpointed [1].

CRIU is used by OpenVZ, which, if I remember correctly, are moving or moved away from a kernel-based approach.

[1] [http://efiop-notes.blogspot.com/2014/06/how-to-set-pid- using...](http://efiop-notes.blogspot.com/2014/06/how-to-set-pid-using- nslastpid.html)

<comment><USER3> This was immensely informative, thank you for that link. CRIU looks like an awesome project. I can't believe I haven't heard of them prior to today.

</comment></comment></comment></comment></comment><comment><USER6> This reminds me of OpenMOSIX, which implemented a good chunk of POSIX in a distributed fashion.

MPI also comes to mind, but it's more focused on the IPC mechanisms.

I always liked Plan 9's approach, where every CPU is just a file and you execute code by writing to that file, even if it's on a remote filesystem.

<comment><USER7> For people that want to try OpenMOSIX out, take a look at this site [http://dirk.eddelbuettel.com/quantian.html](http://dirk.eddelbuettel.com/quantian.html) He has a distro that is called Quantian, with a big collection of science tools added. Shame it's Sunday, I'll need to wait a week to pull it down and see how well it flys.

<comment><USER7> Found the developer, he passed on 'quantian 0.7.9.2 iso' leads to [http://sunsite.rwth- aachen.de:3080/ftp/pub/Linux/quantian/](http://sunsite.rwth- aachen.de:3080/ftp/pub/Linux/quantian/) This was the last release.

</comment></comment></comment><comment><USER8> What's old is new again -- I'm pretty sure QNX could do this in the 1990s.

QNX had a really cool way of doing inter-process communication over the LAN that worked as if it were local. Used it in my first lab job in 2001. You might not find it on the web, though. The API references were all (thick!) dead trees.

Edit: Looks like QNX4 couldn't fork over the LAN. It had a separate "spawn()" call that could operate across nodes.

[https://www.qnx.com/developers/docs/qnx_4.25_docs/qnx4/sysar...](https://www.qnx.com/developers/docs/qnx_4.25_docs/qnx4/sysarch/proc.html)

<comment><USER9> > I'm pretty sure QNX could do this in the 1990s.

Plan 9, SmallTalk

<comment><USER10> Erlang of course. All spawns and messages can be local or remote.

<comment><USER11> Any time I see something about remote processes, I immediately think, "Erlang could probably do this."

I think Erlang would have been the programming language of the 21st century... If only the syntax wasn't like line noise and a printer error code had a baby, and raised it to think like Lisp.

<comment><USER12> Elixir is pretty cool. You get all the power of Erlang wrapped up in a much more developer friendly language.

</comment></comment></comment></comment></comment><comment><USER13> It's nice to see people re-discover old school tech. In cluster computing this was generally called "application checkpointing"[1] and it's still in use in many different systems today. If you want to build this into your app for parallel computing you'd typically use PVM[2]/MPI[3]. SSI[4] clusters tried to simplify all this by making any process "telefork" and run on any node (based on a load balancing algorithm), but the most persistent and difficult challenge was getting shared memory and threading to work reliably.

It looks like CRIU support is bundled in kernels since 3.11[5], and works for me in Ubuntu 18.04, so you can basically do this now without custom apps.

[1] [https://en.wikipedia.org/wiki/Application_checkpointing](https://en.wikipedia.org/wiki/Application_checkpointing) [2] [https://en.wikipedia.org/wiki/Parallel_Virtual_Machine](https://en.wikipedia.org/wiki/Parallel_Virtual_Machine) [3] [https://en.wikipedia.org/wiki/Message_Passing_Interface](https://en.wikipedia.org/wiki/Message_Passing_Interface) [4] [https://en.wikipedia.org/wiki/Single_system_image](https://en.wikipedia.org/wiki/Single_system_image) [5] [https://en.wikipedia.org/wiki/CRIU#Use](https://en.wikipedia.org/wiki/CRIU#Use)

</comment><comment><USER14> Really cool idea! Thanks for providing so much detail in the post. I enjoyed it.

A somewhat related project is the PIOS operating system written 10 years ago but still used today to teach the operating systems class there. The OS has different goals than your project but it does support forking processes to different machines and then deterministically merging their results back into the parent process. Your post remind me of it. There's a handful of papers that talks about the different things they did with the OS, as well as their best paper award at OSDI 2010.

[https://dedis.cs.yale.edu/2010/det/](https://dedis.cs.yale.edu/2010/det/)

</comment><comment><USER15> Condor, a distributed computing environment, has done IO remoting (where all calls to IO on the target machine get sent back to the source) for several decades. The origin of Linux Containers was process migration.

I believe people have found other ways to do this, personally I think the ECS model (like k8s, but the cloud provider hosts the k8s environment) where the user packages up all the dependencies and clearly specifies the IO mechanisms through late biniding, makes a lot more sense for distributed computing.

<comment><USER16> I clicked through to mention Condor too... I first came across it in the 90's, and it seems like one of those obvious hacks that keeps being reinvented.

<comment><USER15> I was actually channeling the creator of Condor, Miron Livny, who has a history of going to talks about distributed computing and pointing out that "Condor already does that" for nearly everything that people try to tout as new and cool.

He's not wrong, but few people use Condor.

<comment><USER17> Few people outside academia, maybe? But inside it still seems to dominate in areas like physics computation. CERN uses a world wide Condor grid. LIGO too. It's excellent for sharing cycles for those slow-burn, highly parallel, massive data scale problems.

I spent more than a decade bringing Condor to semi-conductor, financial and biomedical institutions. It was always a fight to show them there was a better way to utilize their massive server farms that didn't require paying the LSF Tax. Without a shiny sales or marketing department, Condor was hard to pitch to IT departments.

Still, to this day, I see people doing things with "modern" platforms like Kubernetes and such and I chuckle. Had that in Condor 15 years ago in many cases. :)

<comment><USER18> > Still, to this day, I see people doing things with "modern" platforms like > Kubernetes and such and I chuckle. Had that in Condor 15 years ago in many > cases. :)

I'm reading the docs and it seems used mostly for solving long running math problems like protein folding or seti at home?

Can it be used for scaling a website too? I think that's k8s "killer" feature heh.

<comment><USER15> Most systems like Condor have a concept that a job or task is something that "comes up, runs for a while, writes to log files, and then exits on its own, or after a deadline". I've talked to the various batch queue providers and I don't think they really consider "services" (like a webserver, app server, rpc server, or whatever) in their purview.

In fact, that was what struck me the most when I started at Google (a looong time ago): at first I thought of borg like a batch queue system, but really it's more of a "Service-keeper-upper" that does resource management and batch jobs are just sort one type of job (webserving, etc are examples of "Service jobs") laid on top of the RM system.

Over time I've really come to prefer the google approach, for example when a batch job is running, it still listens on a web port, serves up a status page, and numerous other introspection pages that are great for debugging.

TBH I haven't read the Condor, PBS, LSF manuals in a while so it's very well possible they handle service jobs and the associated problems like dynamic port management, task discovery, RPC balancing, etc.

<comment><USER17> But in a world where you're continuously deploying on a cadence that's incredibly quick, how do things differ? I contend the batch and online worlds start to get pretty blurry at this stage. We're not in a world where bragging about uptime on our webserver in years is a thing any more.

I was routinely using Condor in semi-conductor R&D and running batches of jobs where each job was running for many days -- that's probably far longer than any single instance of a service exists at Google in this day and age, right?

None of the batch stuff does the networking management though. No port mapping, no service discovery registration, no load balancer integration, etc. That's Kubernetes sugar they lack. But...has never struck me as overly hard to add, especially if you use Condor's Docker runner facilities.

Edit: I should say that I don't _really_ think you could swap out Kubernetes for Condor. Not easily. But it's always been in my long list of weekend projects to see what running an cluster of online services would be like on Condor. I don't think it'd be awful or all that hard.

The other killer Condor tech is their database technology. the double- evaluation approach of ClassAd is so fantastic for non-homogenous environments. Where loads have needs and computational nodes have needs and everyone can end up mostly happy.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> That goes back to the 1980s, with UCLA Locus. This was a distributed UNIX-like system. You could launch a process on another machine and keep I/O and pipes connected. Even on a machine with a different CPU architecture. They even shared file position between tasks across the network. Locus was eventually part of an IBM product.

A big part of the problem is "fork", which is a primitive designed to work on a PDP-11 with very limited memory. The way "fork" originally worked was to swap out the process, and instead of discarding the in-memory copy, duplicate the process table entry for it, making the swapped-out version and the in- memory version separate processes. This copied code, data, and the process header with the file info. This is a strange way to launch a new process, but it was really easy to implement in early Unix.

Most other systems had some variant on "run" \- launch and run the indicated image. That distributes much better.

<comment><USER20> No need to use the past tense — CreateProcess is the primitive in Windows NT. (It's been 20 years...can we just call that Windows now?)

<comment><USER21> posix_spawn is a thing as well.

<comment><USER22> posix_spawn is horrendous

</comment></comment></comment></comment><comment><USER23> _This can let you stream in new pages of memory only as they are accessed by the program, allowing you to teleport processes with lower latency since they can start running basically right away._

That's what "live migration" does; it can be done with an entire VM: [https://en.wikipedia.org/wiki/Live_migration](https://en.wikipedia.org/wiki/Live_migration)

<comment><USER24> Hey what's the best way to contact you? Have a question. If possible my email is in my profile.

<comment><USER25> [https://www.usenix.org/node/170864](https://www.usenix.org/node/170864)

</comment></comment></comment><comment><USER26> Telescript [0] is based on this idea, although at a higher level. I wish we could just build Actor-based operating systems and then we wouldn't need to keep reinventing flexible distributed computation, but alas...[1]

[0] [https://en.wikipedia.org/wiki/Telescript_(programming_langua...](https://en.wikipedia.org/wiki/Telescript_\(programming_language\))

[1] Yes I know Erlang exists. I wish more people would use it.

<comment><USER27> We'll keep poorly reinventing distributed computing features until we have a real distributed operating system. We're actually not that far off from it, but good luck convincing a mainline kernel dev to accept your patches.

</comment></comment><comment><USER28> I think the problem with a lot of these ideas is that the value of fork() is only marginally higher than the value of starting a fresh process with arguments on a remote machine. The complexity of moving a full process to another machine is 10 times higher than just starting a new process on the remote machine with all the binaries present already.

Quite frankly, vfork only exists and gets used because it's so damned cheap to copy the pagetable entries and use copy-on-write, to save RAM. Take away the cheapness by copying the whole address space over a network, adding slowness, and nobody will be interested any more.

And both techniques are inferior to having a standing service on the remote machine that can accept an RPC and begin doing useful work in under 10 microseconds.

RPC is how we launch mapshards at Google with a worker process that is a long- running server and it just receives a job spec over the network and can execute right away against the job spec.

</comment><comment><USER29> Also of interest might be Sprite- a Berkeley research os developed "back in the day" by Ken Shirriff And others. It boasted a lot of innovations like a logging filesystem (not just metadata) and a distributed process model and filesystem allowing for live migrations between nodes. [https://www2.eecs.berkeley.edu/Research/Projects/CS/sprite/s...](https://www2.eecs.berkeley.edu/Research/Projects/CS/sprite/sprite.html)

<comment><USER30> To be clear, John Ousterhout was the creator of the Sprite operating system. I was one of the grad students working on it.

</comment></comment><comment><USER31> In essence this is manually implementing forking — spawning a new process and copying the bytes over without getting the kernel to help you, except over a network too.

It reminds me a bit of when I wanted to parallelise the PHP test suite but didn't want to (couldn't?) use fork(), yet I didn't want to substantially rewrite the code to be amenable to cleanly re-initialising the state in the right way. But conveniently, this program used mostly global variables, and you can access global variables in PHP as one magic big associative array called $GLOBALS. So I moved most of the program's code into two functions (mostly just adding the enclosing function declaration syntax and indentation, plus `global` imports), made the program re-invoke itself NPROCS times mid- way, sending its children `serialize($GLOBALS)` over a loopback TCP connection, then had the spawned children detect an environment variable to receieve the serialized array over TCP, unserialize() it and copy it into `$GLOBALS`, and call the second function… lo and behold, it worked perfectly. :D (Of course I needed to make some other changes to make it useful, but they were also similar small incisions that tried to avoid refactoring the code as much as possible.)

PHP's test suite uses this horrible hack to this day. It's… easier than rewriting the legacy code…

<comment><AUTHOR> Indeed! I was talking to someone about their attempt to fork a Graal Java process and recreate all the compiler and GC threads, and I said if I had that task I'd be tempted to just use my new knowledge to implement a fork that also recreated those threads rather than trying to understand how to shut them down and restore them properly.

</comment></comment><comment><USER32> I implemented something similar ten years ago for a class project: [https://youtu.be/0am-5noTrWk](https://youtu.be/0am-5noTrWk)

</comment><comment><USER33> See [https://criu.org/Live_migration](https://criu.org/Live_migration)

</comment><comment><USER34> This reminds me a little bit of the idea of 'Single System Image'[1] computing.

The idea, in abstract, is that you login to an environment where you can list running processes, perform filesystem I/O, list and create network connections, etc -- and any and all of these are in fact running across a cluster of distributed machines.

(in a trivial case that cluster might be a single machine, in which case it's essentially no different to logging in to a standalone server)

The wikipedia page referenced has a good description and a list of implementations; sadly the set of {has-recent-release && is-open-source && supports-process-migration} seems empty.

[1] - [https://en.wikipedia.org/wiki/Single_system_image](https://en.wikipedia.org/wiki/Single_system_image)

<comment><USER35> That was the original concept that led Ian Murdock and John Hartman to found Progeny. The idea was that overnight, while no one was working at their desks, companies could reboot their Windows boxes into a SSI network of Linux nodes to run parallel compute tasks.

Roughly, anyway, I got the sales pitch 20 years ago so my memories are fuzzy. I wasn't remotely sold on it but was so anxious to work for a Linux R&D company in Indianapolis of all places that I accepted the job anyway.

Sadly we didn't get far on the concept before the dot-com crash. Absent more venture capital we pivoted to focus on something we could sell, Progeny Linux, and tried to turn that into a managed platform for companies who wanted to run Linux on their appliances.

</comment></comment><comment><USER36> Bonus points if you can effectively implement the "copy on write" ability of the linux kernel to only send over pages to the remote machine that are changed either in the local or remote fork, or read in the remote fork.

A rsync-like diff algorithm might also substantially reduce copied pages if the same or a similar process is teleforked multiple times.

Many processes have a lot of memory which is never read or written, and there's no reason that should be moved, or at least no reason it should be moved quickly.

Using that, you ought to be able to resume the remote fork in milliseconds rather than seconds.

userfaultfd() or mapping everything to files on a FUSE filesystem both look like promising implementation options.

<comment><USER37> If you just pull things on demand, you're going to get a lot of round-trip- time penalties to page things in.

I think you should still be pushing the memory as fast as you can, but maybe you start the child while this is still in progress, and prioritize sending stuff the child asks for (reorder to send that stuff "next"), if you've not already sent it.

<comment><AUTHOR> Yah that is indeed a super important optimization for avoiding round trips. CRIU does this and calls it "pre-paging", their wiki also mentions that they adapt their page streaming to try to pre-stream pages around pages that have been faulted: [https://en.wikipedia.org/wiki/Live_migration#Post- copy_memor...](https://en.wikipedia.org/wiki/Live_migration#Post- copy_memory_migration)

edit: lol I didn't realized that isn't CRIU's wiki since they just linked to a Wikipedia page and both use WikiMedia software. This is the actual CRIU wiki page, and it's way harder to tell if they do this, although I suspect they do and it's in the "copy images" step of the diagram [https://criu.org/Userfaultfd](https://criu.org/Userfaultfd)

</comment></comment></comment><comment><USER38> "Somebody else has had this problem."

Don't get me wrong, this is great hacking and great fun. And this is a good point:

> I think this stuff is really cool because it's an instance of one of my > favourite techniques, which is diving in to find a lesser-known layer of > abstraction that makes something that seems nigh-impossible actually not > that much work. Teleporting a computation may seem impossible, or like it > would require techniques like serializing all your state, copying a binary > executable to the remote machine, and running it there with special command > line flags to reload the state.

</comment><comment><USER39> There was a lot of work on mobile agents 20 years ago, Java programs that could jump from machine to machine over the network and continue executing wherever they landed. The field stagnated because there were some really difficult security problems (how can you trust the code to execute on your machine? How can the code trust whatever machine it lands on and use it's services?). I think later work resolved the security issues but the field has not re-surged. Might be a good place to start to see what the issues and risks of mobile task execution are.

</comment><comment><USER40> It's touched on at the very end, but this kind of work is somewhat similar to what the kernel needs to do on a fork or context switch, so you can really figure out what state you need to keep track of from there. Once you have that, scheduling one of these network processes isn't really all that different than scheduling a normal process, except the of course syscalls on the remote machine will possibly go to a kernel that doesn't know what to do with them.

</comment><comment> lachlan-sneff Wow, this is really interesting. I bet that there's a way of doing this robustly by streaming wasm modules instead of full executables to every server in the cluster.

</comment><comment><USER41> Very cool :). Apart from Plan9 that many people already talked about here, it also made me think of Emacs `unexec` [0].

[0] [http://git.savannah.gnu.org/cgit/emacs.git/tree/src/unexelf....](http://git.savannah.gnu.org/cgit/emacs.git/tree/src/unexelf.c)

</comment><comment><USER42> There's been a bunch of interesting work done on this over the years. Here's a literature survey on the topic: [https://dl.acm.org/doi/abs/10.1145/367701.367728](https://dl.acm.org/doi/abs/10.1145/367701.367728)

</comment><comment><USER43> It's a really nice idea. But when reading it i came to the conclusion that web workers are a really genious idea which could be used for c-like software equally good: Everything which should run on a different server is an extra executable, so this executable would be shipped to the destination, started and then the two processes can talk by message passing. This concept is so generic there could be dozens of "schedulers" to start a process on a remote location, like ssh connect, start a cloud vm, ...

</comment><comment><USER21> The page states that CRIU requires kernel patches, but other sources say that the kernel code for CRIU is already in the mainline kernel. What's up with that?

<comment><USER44> CRIU can use two mechanisms to detect page changes. One is the soft-dirty kernel feature which is mainlined and can be accessed via /proc/PID/pagemap [1]. The other is userfaultfd which is only partially merged in the newest kernel. userfaultfd lacks write detection which the article mentioned. My understanding is that using pagemap requires the entire process to be frozen while it is scanned for memory changes and the memory is copied while uffd allows for a more streaming/on-demand approach that doesn't require stopping the entire process.

[1] [https://www.kernel.org/doc/Documentation/vm/soft- dirty.txt](https://www.kernel.org/doc/Documentation/vm/soft-dirty.txt)

</comment></comment><comment><USER45> I could imagine having a build system which produces a process as an artifact and then just forks it in the cloud without distributing those pesky archives!

<comment><USER46> that... is a really fucking cool idea. i mean, it's of limited utility in the sense that builds really should generally work most of the time & be reproducible, so even though there is a bit of wasted effort, bandwidth, etc. in running the same build in multiple places at once, it's not like the _worst_ thing in the world... but it just feels like there have to be cool / "useful" scenarios where you want to distribute a _thing_ , but not the environment, etc. required to _build_ the thing. oh wait, that already exists: it's called "a binary"... lol. iono.

i mean i guess if you have a process byte-for-byte frozen in time, you're a bit more certain (provably so? maybe?) maybe that when it is resurrected on some remote host that remote process and local process (as of t_freeze) are in ~"exactly" the same state?

or if you continuously snapshotted a process while it was running, and then when it crashes you (hopefully) have a snapshot from right before when it crashed that can be replayed, modified, etc. at will? but that's a bit of a stretch, and there are simpler ways of accomplishing that for the most part...

the only other thing i could think of is that this could allow you to run completely diskless servers, because you can just beam all the processes in, without ever having to "install" anything, beyond copying all the shared libs, etc. which i guess could get the "1 have one of these and i want 100 of them, in the cloud" latency down by an order of magnitude or two maybe.

</comment></comment><comment><USER47> What I'd love it's bingding easily remote directories as local. Not NFS, but a braindead 9p. If I don't have a tool, I'd love to have a bind-mount of a directory from a stranger, and run a binarye from within (or piping it) without he being able to trace the I/O.

If the remote FS is a diff arch, I'd should be able to run the same binary remotely as a fallback option, seamless.

</comment><comment><USER48> I wonder whether the effort by the syzkaller people (@dyukov) could help with the actual description of all the syscalls (that the author says people gave up on for now, because too complex), since they need them to be able to fuzz efficiently...

</comment><comment><USER49> Condor did this in the early 90s.

</comment><comment><USER50> This is similar to what Incredibuild does. It distributes compile and compute jobs across a network, effectively sandboxing the remote process and forwarding all filesystem calls back to the initiating agent.

</comment><comment><USER51> This is one of the best side projects I've ever seen, hands down.

<comment><AUTHOR> <3

</comment></comment><comment><USER52> I think Unison [0] is going to make this trivial

[0] [https://www.unisonweb.org/](https://www.unisonweb.org/)

</comment><comment><USER53> There is some company using CRIU to implement general purpose process live migration (not specifically VMs).

</comment><comment><USER54> side note: take a look at this guy's other blog posts, they're all very good

</comment><comment><USER55> This makes me think about Urbit[0] OS — Urbit OS represents the entire OS state in a simple tree, this would be very simple to implement.

[0]: [https://urbit.org/](https://urbit.org/)

</comment><comment><USER56> I love how people reimplement Plan 9 (poorly).

</comment><comment><USER57> Amazing work.

<comment><USER58> Yep. Reminds me of u/xal praising u/trishume: [https://news.ycombinator.com/item?id=20490964](https://news.ycombinator.com/item?id=20490964)

</comment></comment><comment><USER51> THIS IS REVOLUTIONARY!

<comment><USER28> Hardly. See my rebuttal above.

</comment></comment><comment><USER59> teleforkbomb..

</comment><comment><USER60> hoard() would be a better name.

<comment><USER3> I don't understand, could you explain? `tfork(2)` seems more in line with Linux naming conventions.

<comment><USER60> Telefork as a verb does not map to a real word concept. Hoard fits better, as in hoarding another computer for your process.

<comment><USER38> Ha! I thought you had misspelled _horde!_ ;-)

<comment><USER61> I think you mishurd.</comment></comment></comment></comment></comment>


Show HN: X-File – The secure plain text format - <AUTHOR> X-File is a way to share text that can be read and also contain portions of that text which are kept as secret. In addition, the footer contains a cryptographic signature to verify that the content has not been tampered.
The main difference compared to a normal SHA1 signature is that the encryption used for the footer can randomly use one of the many encryption algorithms to store the secret data and thus make very difficult for brute forcing the key that unlocks the data.
The code and docs can be read at http://x-file.xyz/
Just let me know your thoughts. Thanks. <comment><USER2> Looks intriguing and interesting.

You could make the use case more explicit, in, how would people use it: * the publisher; having the original version of the text, and releasing a censored version; * a casual reader; reading and verifying the released text was not altered (without having access to keywords? at all?) - so there's a matter of trust to add here as well? * a reader with the key to verify and reveal the keywords</comment>


Pension fund puts $50M in blockchain fund - <AUTHOR> https://www.coindesk.com/pension-funds-put-50-million-into-morgan-creeks-new-blockchain-fund <comment><USER2> Wow, pensions use to stick to fixed income to avoid dropping risk on the people depending on them. This is quite the pendulum swing.

This has got to just be a hail Mary from the pension's perspectives and that probably says a lot about their solvency.</comment>


Ask HN: For memorising books, do you prefer a notepad or annotating the margins? - <AUTHOR> Which one have you found to be more effective when reading/memorising books? I've found that the notes I right are only effective as long as I'm engaged by the material. <comment><USER2> Keeping a notepad and taking time to write down impressions after a reading session (or after finishing a section/chapter/book) has been helpful in retaining key takeaways from texts. It's also surprisingly enjoyable to write about what impact non-fiction had on you as well so I definitely think this works for all sorts of reading. And of course if something strikes me in the middle of reading, I've got a notebook to in as well.

Besides that, and somewhat related to your question, starting/joining a book club was one of the more enjoyable things I did in my reading life. Get a group of people together and spend some time talking about what you're reading! In my case, we weren't even reading the same books, it was just another place to talk through things that really hit home while we were reading.

edit: and of course I come across an article about a silent book club here on HN right after writing this. Book clubs are for everyone but I think the format here can help broaden the audience [https://news.ycombinator.com/item?id=22023996](https://news.ycombinator.com/item?id=22023996)

<comment><AUTHOR> Thanks for the advice.

</comment></comment><comment><USER3> Annotating in the margins. No reason to keep the notes and source material separate if it can be avoided.

Also use a pen for underlining. That way you don't have to switch utensils if using a highlighter when you write in the margins.

</comment><comment><USER4> I read everything on a kindle, use kindle highlights, and then when I am done with the book (or even better, while im reading it) dump the highlights into Anki and create different cards from them -- this method has worked well for me for the past 3 years. I plan on doing it for the rest of my life.

<comment><USER5> Could I ask how you dump the highlights to an external service?

<comment><USER6> I guess he is using a technique similar to those posted in this SO post. [1]

Google tells me there are some third party libs/api's to do this, but nothing official, ISFICT.

[1] [https://stackoverflow.com/questions/5374045/how-to-access- ki...](https://stackoverflow.com/questions/5374045/how-to-access-kindle- highlights-api)

</comment></comment></comment><comment><USER7> Using an SRS like Anki will be more effective than any note-taking system, assuming that you do the cards everyday.

<comment><USER8> Which one? Anki or AnkiApps?

<comment><USER7> I use this one:

[https://apps.ankiweb.net/](https://apps.ankiweb.net/)

</comment></comment></comment><comment><USER9> Are you trying to memorize the book?

Or are you trying to learn to apply the mental models?

<comment><AUTHOR> I think the goal is just to retain the information for longer. I often have to revisit non-fiction just to remind myself what the lessons were, which is time I'd rather spend learning about new subjects.

</comment></comment><comment><USER10> Notes on paper --> Create/upload on Anki (SRS). Then open the Anki app daily.</comment>


Why You Should Stop Working at Noon Everyday - <AUTHOR> http://www.seanogle.com/entrepreneurship/stop-working

<comment><USER2> Interesting experiment - we implemented 90 minute sprints, 3x a day for our workdays at my first startup. Thinking was that as former soccer players, a 90 minute game was our peak performance; our bodies and minds were pretty conditioned to 90 minute stretches. It worked really well!</comment>


Bill introduced directing NASA to establish a moon base - <AUTHOR> http://arstechnica.com/science/news/2011/04/bill-introduced-directing-nasa-to-establish-a-moon-base.ars <comment><USER2> High-speed rail:

China: 5000 miles

US: 0 miles

There's only so much money. The US needs to build infrastructure, and other things that will grow its economy.

<comment><USER3> Aw geez, not another high-speed-rail argument.

The US does not have, and will never have, an effective high-speed-rail network. Nor does it really need one. You can already get from every city to every other city in a slow convenient way (car) or a fast inconvenient way (plane), and there's no real need for a third method intermediate in speed. Have you ever been at a total loss to figure out how to get from one city to another conveniently? I haven't.

The Chinese government can build high speed rail, cuz it doesn't mind ripping up the houses of hundreds of thousands of people to build dead-straight rail lines into the middle of cities. The US, at this stage, has far too much investment in its cities to rip 'em up just so that I can get from San Francisco to Los Angeles in four hours (instead of seven hours, or one hour).

<comment><USER4> Yes: Seattle/Portland/Vancouver BC via plane take only about 45 minutes of actual flight time, but 2 hours or so of hassle going through airport nonsense (more if you cross the border).

The drive is two hours, but a waste of gas and time if you're only going for a weekend. Highspeed rail is the perfect intermediate and at a reasonable price (read: $100 or so round trip) would be well more than enough reason to ditch my car for basically all trips greater than 100 miles.

Just because there are some places in the USA where highspeed rail doesn't make sense doesn't mean that argument extends to the whole country.

<comment><USER3> Actually you're right, there may well be parts of the US where high-speed rail makes sense. On a project-by-project basis it's certainly an option worth considering.

What I take issue with is the idea that because _other_ countries have it that the US needs it too. That's a silly argument. Other countries have medieval castles, ubiquitous bidets and abundant kangaroos, but the US doesn't need those either.

(And in fact, only four countries in the world have a high-speed rail network worth a damn -- that's France, Germany, Japan and China).

<comment><USER4> If I'm honest: I mostly want high speed rail because it's __badass __:-) I've been on HSR in France, Japan and China and always feel like I'm in the future.

</comment></comment></comment></comment></comment><comment><USER5> "As its text notes, a return to the Moon has been a Congressional priority several times before; that didn't stop Obama from dismissing it with "We've been there.""

We've been to Oregon too, with Lewis and Clark. Fortunately, we went back.

<comment><USER3> Would we have been back to Oregon, though, if it turned out _not_ to be a lush temperate land of conifer forests, rivers and flat arable soil, but a blazing hot/freezing cold dangerously-irradiated vacuum composed of a mix of rocks and much smaller rocks?

<comment><USER6> well... we did go back to New Mexico, Utah and the other desert states...

<comment><USER7> I've actually seen arguments that we shouldn't really have settlements in those areas -- more lush states aren't short of space, and it takes a large amount of infrastructure to make it work.

<comment><USER8> You could say the same thing about Los Angeles (massive water requirements over and above what native supply provides) but I don't see us abandoning that particular location anytime soon either.

(Edit: clarified LA to Los Angeles)

<comment><USER3> I think this argument is silly in the context of the American southwest, but may very well make a lot of sense for the goddamn moon.

</comment></comment></comment></comment></comment></comment><comment><USER8> Now I like the idea of a moon base and space research as much as the next engineer (and we here at HN seem pretty friendly to it) but I have the general impression that without commensurate investment, this is on par with saying "NASA is directed to clone unicorns"

<comment><USER3> Pretty much, though not exactly.

NASA is, after all, made up of a vast number of people competing to get their own favourite projects funded. If a directive comes down from on high that NASA's primary goal is to build a moonbase then it means that projects relevant to this goal start getting funded (and that projects irrelevant to this goal get a hasty rewrite to make 'em sound like they're relevant to this goal).

</comment></comment><comment><USER9> We should be putting funding into a space elevator. Moving things into space with rockets will be always be incredibly expensive and inefficient. If a space elevator were built, moon (and mars) bases would easily be built, almost as an afterthought. ("Oh yeah, I guess it is easy to do this now...")

Currently, all of our space vehicles are also built to withstand atmospheric exit & entry. With a space elevator, we could afford to have elevator -> moon or earth-elevator -> mars-moon/station transports that would be much easier to build.

If there is any foolish government project I am willing to put my tax dollars into, it's this.

</comment><comment><USER10> Why again do we want to build a base on a big rock? And why do we fund NASA for $20 billion/year? Especially in a struggling economy? Why doesn't some other country step up to do this if it's so important? It seems the US spends money on everybody except itself. Yet, we're the target for every criticism, while others stand practically idle.</comment>


MIT's charm school for geeks turns 20 - <AUTHOR> http://news.msn.com/pop-culture/mits-charm-school-teaches-bookworms-about-social-graces

<comment><USER2> From the article: "But how do you take someone who's grown up in the world of pocket protectors..."

Does anyone actually use a pocket protector anymore? That is such a lame cliche to describe geeks. It's like saying "...someone who's grown up in the world of slide rules..."

More accurate would be something like "...the world of algorithms and Maxwell's equations..." or "...the world of integrals and epsilon-delta..."

<comment><USER3> I know some professional engineers who wear pocket protectors. Not students, though.

<comment><USER4> I'm wondering where you would actually _buy_ a pocket protector? A quick Google search brings up plenty of ironic tshirts and novelties, but no actual pocket protectors.

(And no, I don't actually want one. I'm just curious.)

<comment><USER2> Most people didn't buy them in the old days. Typically they were given away as promotional items. They are actually pretty useful beyond their obvious function of keeping your pockets from being destroyed by ink leaks and sharp points. For example, you could switch a set of writing instruments you use everyday from shirt to shirt without having to disassemble and reassemble it. But even the nerdiest these days realize that the only surer way to look like a total social misfit without involving body odors is to break your plastic- rimmed glasses at the bridge and repair them with tape.

</comment></comment></comment></comment><comment><USER5> I was an engineering student at a top 20 university who did something similar to this. Southern school, manners were important. The university put on an etiquette dinner to help engineering students grasp the finer points of dining.

To this day, I reference that one dinner at least once a month.

</comment><comment><USER6> I would watch that show.

</comment><comment><USER7> Wish we had something like this at Georgia Tech</comment>


Google's Decade - <AUTHOR> http://www.technologyreview.com/blog/guest/22128/

<comment><USER2> You have to wonder what would have happened if Yahoo had bought Google back in 1999. Would Yahoo have had the success Google has had in the last decade? Would we ever have had access to the powerful web tools that Google is providing us with right now? And ore importantly would any other company have managed to transform the Web like Google has? One thing's for sure though - Yahoo and Microsoft would not be scratching their heads over how to stay afloat and compete in this market.

<comment><USER3> I think other companies would have appeared to fill the void, it just would have taken longer.</comment></comment>


New UNIX implementation (1983) - <AUTHOR> https://groups.google.com/forum/#!msg/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J <comment><USER2> A fascinating contrast to the announcment of Linux:

[http://www.thelinuxdaily.com/2010/04/the-first-linux- announc...](http://www.thelinuxdaily.com/2010/04/the-first-linux-announcement- from-linus-torvalds/)

RMS starts with a grandiose vision, which (at the time of writing) he hadn't even begun. He then asks for time, money, and equipment, before the end of the first paragraph. The very first thing promised, a kernel, has never been effectively delivered.

Linus starts with a modest disclaimer, then asks for feedback as to what other potential users might want most.

Night and day. Yet, could we have had one without the other?

<comment><USER3> I find it fascinating that RMS made his pronouncement in '83 and Linus in '91 (8 years later). At the time (and I was at Sun at that time) 'GNU' was still pretty much just a concept, all of the effort from 1983 -> 1991 was in C (gcc), binutils, and emacs. USL was on the verge of suing BSDi for their BSD project (AT&T had been lobbing threats at the GNU effort for years, especially their use of the word 'UNIX' in the description of their acronym)

Post lawsuit RMS wrote his manifesto and the GPL was born, and the various BSD flavors of UNIX were the only UNIX kernels that had had their software provenance litigated.

Into that Linus wrote his entirely new kernel which was 'unix like' but not UNIX at all on the inside (generally user land felt like UNIX because MINIX felt like UNIX).

And because Linux had a pretty complete history from birth to present (making tracing software ownership possible) RMS annexed it as the GNU "kernel" trying to get everyone to call it GNU/Linux for a while.

[1] [http://en.wikipedia.org/wiki/USL_v._BSDi](http://en.wikipedia.org/wiki/USL_v._BSDi)

<comment><USER4> _RMS annexed it as the GNU "kernel" trying to get everyone to call it GNU/Linux for a while._

As dllthomas said, RMS never wanted the kernel to be called "GNU/Linux", he wanted (wants) the Linux kernel with the GNU userland to be called "GNU/Linux".

I see that you an link to the Wikipedia page on the naming controversy. "Operating system", at least as RMS uses it, doesn't refer to the kernel, it refers to the kernel+userland.

<comment><USER3> And to dllthomas' point, the controversy is the 'it'. Naming things is a pain, and naming computer things is a bigger pain. That was the root of the controversy.

Today most people who use a Linux distribution on their machine will say they run Linux or "distroname" (like Ubuntu or CentOS) on their machine. They don't say they run Gnome 2/Linux Kernel 3.x/gcc 4/gnu utils 2.8. By RMS' reasoning (and he is very precise in this) the "Operating System" is not "Linux" it is "GNU/Linux". They even added a field for it in uname(1) (-o).

The "Operating System" is a combination of the kernel, the editors, and user tools, and the compilers. Go back and read his original announcement, this is his definition:

 _" To begin with, GNU will be a kernel plus all the utilities needed to write and run C programs: editor, shell, C compiler, linker, assembler, and a few other things. After this we will add a text formatter, a YACC, an Empire game, a spreadsheet, and hundreds of other things. We hope to supply, eventually, everything useful that normally comes with a Unix system, and anything else useful, including on-line and hardcopy documentation."_

By that original definition, we shouldn't even bother calling it GNU/Linux should we? Its just GNU. I expect taking that position though would be quite unpopular.

As the Wikipedia page alludes to, there were at least two camps at the time. Many, like myself who 'grew up' on the UNIX side of the house, the Naming Hierarchy was 'kernel->[userland]->window system' So it was SunOS->SunTools or SunOS->X/News. If you called something a "UNIX" system it meant that the _kernel_ was based on the design architecture of Thompon, Ritchey, et al from Bell Labs. SunOS, IRIX, SystemV, and Unicos were all "UNIX" systems even though their userland code and window systems (if they had one) varied. Even in RMS' original statement of purpose, the first component of GNU was a _kernel_ , and some other bits.

So to someone with the same background as I've had you could call it Linux/GNU (hierarchy preserved) or "Ubuntu" based on Linux with GNU tools. But Linux as the kernel architecture is at the top of the naming chain, not GNU.

For folks who came up from the DOS/PC side of things the 'window system' was the primary naming tool (in this case Windows). It was PC-DOS/Windows for a while but Microsoft decided to go all out and make "Windows" the brand/trademark they hung their hat on, and we got Windows/DOS, Windows/NT, Windows 98, Windows XP, Windows Vista, Etc.

As dllthomas illustrates, the interpretation of the 'it' part rooted the disagreement, and which side of the disagreement one resonated with seemed strongly correlated with ones early exposure to naming conventions.

My belief is that the _emotion_ in the discussion came from egos, or people feeling person A or person B's contributions or impact were over or under valued. We can see some of that in the comments here where people emotionally jump to a conclusion that I'm down on RMS (I'm not), or even partisan in this debate (I'm not that either). There is ample documentation that Linus' goal with Linux was to write a new OS from scratch kind of like (but better than :-) MINIX. And there is ample documentation that RMS' goal was to create an operating environment that was unencumbered by onerous restrictions. It was fortuitous the Linus' work was so successful, and it was fortuitous that Linus could leverage the work done in the preceding 9 years in making tools available for him to use.

But it was also clear that Linus wasn't specifically supporting the GNU project by writing Linux (although he subscribed to the philosophy). When RMS started claiming as part of the project it set up this little tempest. Perhaps the term 'annex' is too emotionally charged to be used for that action.

<comment><USER5> FWIW, I frequently refer to the system I run as "Linux" \- with the fact that there's a sizable amount of GNU code pretty much assumed by the fact that I'm running a Linux kernel and it's a server or desktop and not an embedded device or something. I just think the case for "GNU/Linux" is better than it is frequently made out to be, and (as mentioned) disagreed about Stallman's position on the issue.

</comment></comment></comment></comment></comment><comment><USER6> "Individual programmers can contribute by writing a compatible duplicate of some Unix utility and giving it to me. For most projects, such part-time distributed work would be very hard to coordinate; the independently-written parts would not work together. But for the particular task of replacing Unix, this problem is absent. Most interface specifications are fixed by Unix compatibility. If each contribution works with the rest of Unix, it will probably work with the rest of GNU."

This stood out to me. Back in 1983 online collaboration was unheard of, and it was only the incredibly modular nature of Unix which made the project seem at all plausible.

<comment> fsck--off > Back in 1983 online collaboration was unheard of

Not really. Stallman mentioned Chaosnet which the AI Lab had used for, among other things, internal mail (not yet called "e-mails") between developers about changes to programs. He also mentions UUCP, which was the godawful Unix way of sharing files (and which could be used for mail purposes).

</comment></comment><comment> fsck--off

      "I am Richard Stallman, inventor of the original
       much-imitated EMACS editor."

Stallman may have significantly improved Emacs, but he isn't the inventor. Guy Steele and David Moon are. Stallman only took over development after it had become the standard AI text editor. Stallman wasn't even the first one to implement Emacs in Lisp; Dan Weinreb did it first. "Inventor of the original" makes it sound like Emacs was his original idea.

[http://web.archive.org/web/20110719154038/http://danweinreb....](http://web.archive.org/web/20110719154038/http://danweinreb.org/blog/rebuttal- to-stallmans-story-about-the-formation-of-symbolics-and-lmi)

<comment><USER7> I really dislike the term invent when applied to creating software. Emacs was never invented, it was written (or if you prefer, coded). Text editors were invented long before Emacs. Just iterating and improving on a concept doesn't make you inventor.

<comment><USER8> To be fair: "text editors" really hadn't been invented "long before emacs". At the time (the early 1970's) editing text was a subject of much research and experimentation. Differing paradigms were being tried on new and exciting hardware (the glass tty). Screen editors, as they came to be known, were a brand new curiosity -- they were equally disruptive (if not more so) as the "web application" or "capacitive touchscreen interface" would be decades later.

And emacs was one of the very first screen editors. It invented lots of the stuff that would later seem "obvious".

So no: I think it's entirely appropriate to say that emacs was "invented" in the same way the browser was.

<comment><USER9> It's before my time, but my impression was that both TECO and EMACS originated as teletype line editors (as with ed and ex, the direct predecessors to vi), and were only later adapted to fancy new screen terminals. Am I mistaken?

<comment><USER8> I likewise was never a user. But the EMACS package of TECO macros was intended specifically to enable screen editting on terminals as I understand it (though surely there was overlap). Basically EMACS:TECO as vi:ex.

</comment></comment></comment></comment></comment><comment><USER10> You can also read it here, complete with an old-school font: firstname.lastname@example.com

This site replays old usenet posts. It was cool to wait for the announcement to pop up on net.unix-wizards.

Happy 30th birthday, GNU!

There is a GNU hackathon at MIT this weekend, for those that don't know already and might be interested: [https://gnu.org/gnu30/](https://gnu.org/gnu30/)

</comment><comment><USER11> Hopefully in 30 years people will be able to read such seminal messages. Now that a lot of discussion is happening on proprietary platforms without a standard it may not be the case.

<comment><USER12> It's already the case with some stuff from the '90s: there was important stuff announced and discussed on CompuServe and AOL forums that has not been publicly archived. There are some personal archives out there, but they are pretty spotty. Often the only surviving (or at least accessible) record comes in the form of occasional quotes in other venues, like an academic paper from the 1990s that included a blockquote and a citation to a now-unreachable network address.

</comment></comment><comment><USER13> Both C and Lisp will be available as system programming languages.

How I wish this became true, it's not too late right?

<comment><USER14> I wonder GNU Guile fills this requirement. It exists, but isn't common. I've never personally encountered it. Python or Lua seem much more prevalent as extension languages on a typical GNU/Linux system.

[http://en.wikipedia.org/wiki/GNU_Guile](http://en.wikipedia.org/wiki/GNU_Guile)

<comment><USER15> FWIW, it's starting to look like the long-considered plan to move Emacs to Guile may actually become a reality:

[http://article.gmane.org/gmane.emacs.devel/162042](http://article.gmane.org/gmane.emacs.devel/162042)

This has been a vague goal since the mid-'90s, and given the popularity of elisp etc, would probably be a nice boost for Guile, but the gulf between neat-idea and messy reality has always seemed so vast that I never thought it would actually happen.

[I haven't used Guile since the '90s... it was a bit of a disaster back then, but from what I hear, it's improved immensely (i.e., largely been rewritten) since.]

</comment></comment></comment><comment><USER16> Can't believe "on-line" and "snail mail" were terms being used in 1983!

<comment><USER17> I wouldn't be surprised if he invented snail-mail

Dude was proverbially "leet as fuck" back in that day.

<comment><USER18> 6|\||_| h4x0r5

</comment></comment></comment><comment><USER5> Hm, this may be a bit of history I'm missing: what was meant, in 1983, by "an Empire game"?

Edit: Apparently this: [http://en.wikipedia.org/wiki/Empire_Classic_%28video_game%29](http://en.wikipedia.org/wiki/Empire_Classic_%28video_game%29)

<comment><USER19> Or this: [http://en.wikipedia.org/wiki/Classic_Empire_(computer_game)](http://en.wikipedia.org/wiki/Classic_Empire_\(computer_game\))

<comment><USER17> Oh wow. I didn't realize the game Strategic Conquest was pretty much a copy of Empire, for Mac.

[http://en.wikipedia.org/wiki/Strategic_Conquest](http://en.wikipedia.org/wiki/Strategic_Conquest)

</comment></comment></comment><comment><USER20> Thanks Stallman for never give up of his dreams.

</comment><comment><USER21> The GNU system announcement is also on gnu.org, if you're not too kean on Google Groups.

<[https://www.gnu.org/gnu/initial- announcement.html>](https://www.gnu.org/gnu/initial-announcement.html>)

</comment><comment><USER22> Now, imagine if Oracle had its way and APIs were copyrightable.

</comment><comment><USER23> "GNU will be able to run Unix programs, but will not be identical to Unix. We will make all improvements that are convenient, based on our experience with other operating systems. In particular, we plan to have longer filenames, file version numbers, a crashproof file system, filename completion perhaps, terminal-independent display support, [...]

Linux has, eventually, started to fulfil the promise: technologies like cgroups, dm, uevents, kdbus, alsa..., and the respective userspace: systemd, dmraid, lvm, udev, pulseaudio, show that GNU/Linux is not UNIX but better in some respects.

<comment><USER5> Genetic unices have progressed as well...

<comment><USER23> I agree. And I think that FreeBSD and DragonFly stand over the rest in this respect, both have incredible technology in their guts. Obviously OpenBSD has done wonders teaching people that you can write sane, safe and audited code that works and NetBSD... Oh well, stagnant despite the really good technology that shows up in there frequently, but runs on toasters!

</comment></comment></comment><comment><USER24> Thank you, Mr. Stallman.

</comment><comment><USER25> Innocent question: what happened to GNU? Lotsa good tools available, but as Minix spawned Linux which has a good chunk of the world, GNU as an OS seems but a legend.

<comment><USER26> Their OS is called GNU Hurd. [1] Apparently, they are still making progress, but it must be hard to catch up with Linux.

[1] [http://en.wikipedia.org/wiki/GNU_Hurd](http://en.wikipedia.org/wiki/GNU_Hurd)

<comment><USER27> Especially hard to catch up with Linux when Linux began development in 1991 and Hurd began in 1983. Linux is only -8 years ahead of Hurd in development, you'd think Hurd could have caught up by now.

<comment><USER28> Well, it is only so, given the amount of money and resources IBM, Intel and friends put into Linux.

How far would be Linux without those investments?

<comment><USER29> The linux of 1998 is still light years ahead of the hurd of today. That was before all those IBM and intel investments.

<comment><USER30> Light years ahead in what sense? I think it's an anachronism from a theoretical standpoint. I'd like to see more people run with newer ideas from Plan 9 / Inferno / Hurd / all the other operating systems that aren't a monolithic 1970s design.

<comment><USER31> > Light years ahead in what sense?

Usability for pretty much any of the purposes for which one uses a computer operating system other than exploring theoretical ideas about how to architect a computer operating system.

> I'd like to see more people run with newer ideas from Plan 9 / Inferno / > Hurd / all the other operating systems that aren't a monolithic 1970s > design.

So would I. That doesn't make any Hurd more ready for any other use _other than_ exploring newer OS ideas that Linux is.

Or even the most ready for such uses (or even the exploring OS architecture use) of the "Plan 9 / Inferno / Hurd" set.

</comment></comment></comment></comment></comment></comment></comment><comment><USER32> For those interested in the history of open-source "UNIX" clones but who weren't in the industry at the time, I stumbled across this last night and thought it worth sharing: [http://www.cs.vu.nl/~ast/brown/](http://www.cs.vu.nl/~ast/brown/). Read the "original comment", "follow up", "code comparison", and "rebuttal".

</comment><comment><USER33> "Arpanet mail: email@example.com

Usenet: ...!mit-eddie!RMS@OZ ...!mit-vax!RMS@OZ

US Snail: Richard Stallman 166 Prospect St Cambridge, MA 02139"

Amazing. The use of words like "US-Snail" and all. Looks far cooler and geekier than what we have now usually:

        xyz
        firstname.lastname@example.com
        @xyztwitterhandle

</comment><comment><USER34> Delivery to the following recipient failed permanently:

         firstname.lastname@example.com

:(

</comment><comment><USER35> Perhaps a novice question coming from a mostly Windows user: an OS I thought handled processes, hardware, etc. It can of course be handy to have a linker/editor/compiler shipped with it too, but why is it considered part of the OS? Is gcc/vi/emacs more a "part" of GNU/Linux than notepad is a part of windows; i.e. just a convenience? Or is linux unusable without a complete c-based tool chain? I know the term "OS" is a fuzzy one, which is probably what causes my confusion.

<comment><USER36> My personal opinion, which might not line up with anyone elses: GCC is a 'part' of GNU/Linux because of the goals/aims of GNU/Linux, which is to have a freely modifiable and distributable operating system. If you put yourself in the time when this project was started, there was no free compiler, they simply didn't have one. But in order to meet the goals of the project, they would need one (you might not need one to run it, but you sure do need one to develop it), so it's therefore a very core part of the project.

You don't need a c-based tool chain installed to run a GNU/Linux system or make it usable for a non-dev though, no, Ubuntu doesn't have it by default for example.

</comment></comment><comment><USER37> Weird coincidence as how Google use Sept. 27th as their birthdate...

</comment><comment><USER38> I'd assumed from reading the UNIX haters' handbook, that everyone who had used a Lisp machine would automatically hate UNIX :)

<comment><USER39> RMS wrote: > To begin with, GNU will be a kernel

Right. Not so much. That was exactly the last thing to happen as it turned out. He and everyone else in the effort worked on the tools and apps.

By contrast: > I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) > and things seem to work. This implies that I'll get something practical within a few months

Anyhow, torvalds had already put in the work on the kernel and wanted feedback.

Torvalds had a kernel and no tools. RMS had tools but no kernel, so the inevitable happened and they were wed. Torvalds even chose the RMS version of a marriage license.

The marriage is fruitful, but RMS is frigid and bitchy the whole time, solely over the name of the child -- even though the child has matured and gone on to a brilliant career they can both be proud of.

This would also be a good time to note that Torvalds is not the one who chose the name of Linux -- usenet chose it.

I have great respect for RMS and admire his courage (and self-discipline) very much, but I really wish he'd let that thing about the kid's name go. Just let it go already Richard.

<comment><USER5> Well no, Torvalds had a kernel and all the tools GNU had created.

And Linux is (entirely legitimately) the name of the kernel - the dispute[1] is over the name of systems running a bunch of GNU code on top of a Linux kernel.

[1]: Elsewhere it has been contended that RMS has pushed for the name to apply to the kernel itself as well. I don't believe that was ever the case (though will certainly update my beliefs if presented with evidence) but am far more confident that it is not his current position.</comment></comment></comment>


UK House of Lords Report on AI - <AUTHOR> https://www.parliament.uk/business/committees/committees-a-z/lords-select/ai-committee/news-parliament-2017/ai-report-published/ <comment><AUTHOR> TL;DR version at [https://social.shorthand.com/LordsAICom/32KXpihQLj/ai-in- the...](https://social.shorthand.com/LordsAICom/32KXpihQLj/ai-in-the-uk)</comment>


Ask HN: Working on a Project to Learn - <AUTHOR>

Hey HN, I am currently an undergraduate in CS and I have a bit of free time that I'd like to spend programming something. I learn best by working on some kind of project but don't know what to program. I'm not looking for a "resume builder" but if I could say "I made this for fun in my spare time and learned a ton" that would be a bonus. I am open to suggestions regarding language and platform. So HN, let me hear your ideas of what I can work on so I can learn!<p>Edit: Like I said, I am open to suggestions and have no preference in regards to web apps or anything else. <comment><USER2> You don't mention what type of applications you're interested in, Web apps or something else.

The most fun programming i ever did was for my Digital Image Processing class, mainly because the output was so visual. You might try something visual, graphics(a game perhaps), image processing(non photo realistic rendering of a pic).... you get the picture :-)

Edit: you might even do some visualization of data, using processing.js for example. This would not require much time and you can even share them as a web app.

<comment><AUTHOR> Alright, thank you for the input. I'll take it into consideration.

</comment></comment><comment><USER3> My email address is in my profile if you want to email me. What languages do you know? Are you interested in web based projects? What other projects have you been doing? What other experience (besides programming) do you have?

<comment><AUTHOR> Hmm, I can't seem to see the email in your profile.

<comment><USER3> Oh, sorry. Please check again. It is in the About section. You need to convert it from hex to ascii to see it.

</comment></comment></comment><comment><USER4> Write an interpreter/compiler for your own little language. With your own garbage collector.

<comment><AUTHOR> Heh, that's an awesome suggestion. Do you have any resources that would be best for this? Probably the Dragon Book?

<comment><USER4> I found this to be a great introduction on how to write a compiler: [http://www.cs.indiana.edu/~aghuloum/compilers- tutorial-2006-...](http://www.cs.indiana.edu/~aghuloum/compilers- tutorial-2006-09-16.pdf)

It starts with a compiler for a language that only accepts integers and gradually extends it to support complex expressions, functions, closures, conditionals etc. Since the compiler is written in scheme and the target language has a lispy syntax, it completely skips the parsing step (using scheme's builtin reader). I really enjoyed this fact, since I find parsing to be the most boring part of a compiler. The fun comes after ;)

Another interesting read is [http://repository.readscheme.org/ftp/papers/ai- lab-pubs/AIM-...](http://repository.readscheme.org/ftp/papers/ai-lab- pubs/AIM-453.pdf)

My first language implementation was an interpreter (interpreters are easier to write, although compilers are more satisfing when you start to benchmark the speed of the generated programs) written in C for a dialect of lisp. I chose lisp because it was easier to parse. The program was converted in an AST (abstract syntax tree) that was then directly evaluated. At first, there was no GC, no support for direct recursion, no closures etc. I started with something small and easy to implement and then incrementally extended it.

Golden rule when writing your first language implementation: keep your language simple. Keep the implementation straightforward to understand, even if it means that it is slow. Don't bother adding support for debugging or meaningful error messages. You can add those things later after you've grasped better how an interpreter/compiler works.

<comment><AUTHOR> Hmm, that's interesting. The first example doesn't seem to work. I just get "#<unknown 0x00000007>". I wonder if it has to do with a difference with gcc.

</comment></comment></comment></comment><comment><USER5> I would suggest finding an open source project you like and contributing to it.</comment>


Bitsquatting: DNS Hijacking without exploitation - <AUTHOR> http://dinaburg.org/bitsquatting.html <comment><USER2> A serious amount of Redis crash reports are due to memory errors (we ask to test memory after crashes, since real segfaults are very rare, not everybody tests, tests may lead to false negative, so the problem is bigger than the one we observe).

This experiment also seems to show that there is a lot of corrupted memory out there.

There is a simple fix, I wonder why it is not used:

1) Add a feature at operating system level that one time every second tests N memory pages, at random. No observable performance degradation.

2) Report the problem to the user when it is found.

3) Mark the page as faked up and don't use it, ever.

So you have memory tested basically for free, users aware of their hardware errors, and a lot less consequences for those errors.

<comment><USER3> Guild Wars did this in user mode in the background while playing - it did background testing of the user's CPU, RAM, and GPU. Machines that failed the tests were flagged so that their crash reports got bucketed separately (saving us the time of trying to understand impossible crashes), and it popped up a message telling the user their computer was broken.

So even if the OS should be doing this for you, for long-running processes you could do it yourself in user mode. (I don't know if it's worth the effort, though.)

<comment><USER2> That's exactly my plan with Redis, and it is awesome to discover that it was used with success in the past! But I've a problem given that I can't access memory at a lower level, that is, when to test and what?

I've the following possibilities basically:

1) Test on malloc(), with a given probability, and perhaps only up to N bytes of the allocation, for latency concerns.

2) Do the same also at free() time.

3) From the time to time test just allocating a new piece of memory with malloc of a fixed size, test it, and free it again.

"3" is the one with the minimum overhead, but probably 1+2 have a bigger percentage of hitting all the pages, eventually...

I don't have a broken memory module to test different strategies, I wonder if there is a kenrel module that simulates broken mem.

Note that Redis can already test the computer memory with 'redis-server --test-memory' but of course this requires user intervention.

</comment></comment></comment><comment><AUTHOR> TL;DR: every day, there are hundreds of wrong URL requests being done due to memory failures in the computers. Due to hardware problem, the computer can connect e.g. to microsmft.com instead of microsoft.com. The data gathered by the researcher suggests those kind of bugs happen also in web caches etc. thus increasing the number of affected users.

In practice, the privacy problems coming from this are rather limited (unless you send private stuff in URL), since in majority of cases, you'll not be sending domain cookies for the original domain if it resolved to the bitsquat domain early. Anyway, it's still probably a thing not thoroughly thought of on a daily basis regarding security.

<comment><AUTHOR> There are follow-up blog articles on the author's page. This one is also interesting: [http://blog.dinaburg.org/2012/10/a-preview-of- bitsquatting-p...](http://blog.dinaburg.org/2012/10/a-preview-of-bitsquatting- pcaps.html)

The original link was found at h-online.

</comment></comment><comment><USER4> I did this experiment by bitsquatting all the domains around cloudfront.net after hearing about it from defcon. It works. You basically have the opportunity to replace the javascript of tons of sites. I simply served 404s. What was really interesting to me was the varied places where the corruption occurs. Some of the requests even have the correct Host header. Now you know why the old PC was so flaky!

<comment><AUTHOR> I've started thinking about all those banks [1] and other pages serving like/tweet buttons on the login page.

Or pages including Google Analytics. If the described behaviors really take place, given the massive scale of deployment of Google Analytics, Statcounter, FB buttons, jQuery includes from CDNs, you should be able to do arbitrary JS injections to a non-trivial number of users (though very random).

[1] [http://my.opera.com/hallvors/blog/2012/05/11/social-media- ba...](http://my.opera.com/hallvors/blog/2012/05/11/social-media-banking)

</comment></comment><comment><USER5> It's pretty amazing to see so many bit errors make it through DNS resolution without the client machines crashing instantly; imagine if the bit errors were introduced in RAM containing code or data structures with memory pointers, instead of a domain name!

Thinking about it, I guess such crashes are actually a much larger number.

<comment><USER6> Everyone who's worked with computers for any length of time has seen inexplicable, unreproducible crashes, lockups or reboots.

We're just conditioned to ignore them if they're neither frequent nor reproducible.

</comment></comment><comment><USER7> Hi,

Author of the article here. Was happily surprised to find it linked on the front page of HN.

I can try to answer any questions that you may have have.

I'm also in the midst of writing another blog post, this time talking about the bit-error distribution in the DNS query type field. Spoiler: its not uniform.

<comment><USER7> Part 3 is now up: [http://blog.dinaburg.org/2012/11/bitsquatting-pcap- analysis-...](http://blog.dinaburg.org/2012/11/bitsquatting-pcap-analysis- part-3-bit.html)

<comment><USER8> The part about the flip being mostly on a single bit reminded me of [1], I wonder if we're seeing the same cause in two different ways?

[1] [http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that- so...](http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes- couldnt.html)

</comment></comment></comment><comment><USER9> _HTTP 1.1 includes a header field called Host._

Technically, this statement is correct, but don't overlook the fact that while the Host header is optional in HTTP 1.0, most HTTP 1.0 clients will include it out of necessity. It's nearly impossible to guarantee you'll get the correct resource without a Host header, these days.

<comment><USER10> Are there HTTP 1.0 clients still in use? I can't imagine why a maintained program wouldn't be using 1.1.

</comment></comment><comment><USER11> Very fascinating... if a very narrow scenario with a very low probability resulted in this, how is it that these errors are not apparent during other computer activities ?

<comment><USER5> They probably are, but people are used to bluescreens and unexpected app crashes and think nothing of it.

<comment><USER9> I dunno. Something about this analysis bothers me. The basic premise is that a string the length of a domain name routinely, albeit rarely, gets corrupted by one bit, causing errant DNS lookups. Then it's reasonable to assume that a longer string is even more likely to contain corruption. But if that's so, why do I almost never see any evidence of bit corruption in my web server logs? Surely the same corruption would affect other parts of the URL, and the probability should be greater due to the length. But I can't find a single example in my logs that can't be explained by human error (typos by users or developers). If bit corruption is so overwhelmingly prevalent in hostnames, but not URLs or other identifiers, I suspect it's due to a software bug somewhere.

<comment><USER12> Presumably your web server doesn't serve quite as much traffic as fbcdn.net. The odds of such a bitflip happening are vanishingly low, so you need an incredibly large amount of traffic before you'll see such errors occurring.

<comment><USER13> In my understanding of network communication and data transmission this should be impossible. All payload data and encapsulated header data etc is subject to checksums, hash's, variable encoding schemes on the wire, parity balancing, redundant bit insertion (hamming) etc. The result of which will always signal an errors presence. Even if the bit flip occurs in Primary memory surely the OS's memory management subsystem's would detect the corruption.

So for a bit flip not to be detected and remedied before the execution of an errant DNS lookup seems odd. Although I could be wrong (just a final year CS student).

EDIT: Just watched the video, originally classified it as TLDW, seems plausible.

<comment><USER14> Note that no error detection code is able to detect all errors; it just lowers the probability of an error passing undetected even further. (CRCs are pretty robust in that they always detect sequences of errors with length <= N, with N depending on the particular algorithm.) With a large enough sample size, you _will_ hit errors.

In this case it is probably memory corruption. The OS won't be able to detect such a thing unless the memory has ECC (relatively uncommon these days). It could theoretically detect it if the memory pages were checksummed and periodically verified against the checksum, but afaik no OS does so.

</comment></comment></comment></comment></comment></comment><comment><USER15> I wonder if the distribution of hits to these domains follows the popularity of the underlying "correct" domain, which is what you'd expect if random bit errors were causing these hits and would help corroborate his claim.

<comment><USER5> I'm wondering if the string length or alignment could affect it, too. For example, if memory is allocated in 16byte chunks, longer domain names might have a larger chance for having a bit flipped in the active part of the string (instead of the padding). Just speculating wildly here.

<comment><USER16> There's no padding in DNS requests (having written my own DNS decoding routines). There's also very little that can change in a DNS packet that won't cause an error---basically, the only thing that can change without causing a DNS decoding error are text-related fields (say, the payload of a TXT or SPF record type) and even then, given the restrictions on character sets in DNS host names (and the crazy compression scheme used for domain names), it's actually surprising to see bit-6 errors, as that bit should cause more invalid domain names than not.

Edited because I thought bit-6 errors would flip letter case (upper to lower, lower to upper) when it's bit-5 that will do that.

<comment><USER15> I'm a little skeptical of this whole thing. Obviously, one other thing about bit errors in DNS packets is that they need to not break label compression.

<comment><USER5> Do you mean the whole article, or just the parent post? Because we're probably talking about bit errors while the domain name is still just a string in application memory, not when it's being assembled to a DNS packet or in transit. So I don't see how label compression comes into play.

</comment></comment></comment></comment></comment><comment><USER14> This reminds me of this (slightly old) article, wherein the authors observe that about 1 in every 30,000 TCP packets fails the TCP checksum, even though the actual errors should have been caught by the link-level CRC. They go on to speculate about possible causes, which include memory corruption at the hosts.

A good read, and it seems to be an instance of the same problem.

"When The CRC and TCP Checksum Disagree" - [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.7...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.7611)

</comment><comment><USER17> Are those really memory corruptions and not, say, damaged cables or bugs in network infrastructure equipment? Or, just typos in scripts (both server and client side)?

<comment><USER2> Other bit-flipping issues should be usually trapped by checksums. Typos in scripts is possible but then you should get a massive amount of requests from the same IP and should be easy to notice.

</comment></comment><comment><USER18> Yet, ECC on consumer hardware ranges from exotic to non-existent.</comment>


Lisp Idioms - <AUTHOR> http://cybertiggyr.com/gene/lid/ <comment><USER2> Gene Michael Stover wrote about Common Lisp as someone who hardly ever looked at anyone else's idiomatic use of the language. I was pretty bummed that he got published in Dr. Dobbs with some pretty awful CL code.

</comment><comment><USER3> Might be useful to mention that these are mostly specific to CL.

</comment><comment><USER4> If you happen to know the sequence is a list then might as well (apply #'min '(1 2 3 4 5)) instead of (reduce #'min '(1 2 3 4 5)).

I'm sure the author is aware of that, but a reader might get the impression that #'min can only take two arguments.

<comment><USER5> Apply has the problem in that Common Lisp allows compilers to restrict the number of arguments passed to a function to a small value (specified by call- arguments-limit: [http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...](http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/convar_call- a_uments-limit.html)).

<comment><USER6> I'm not well versed in CLHS legalese, but don't &rest arguments count as one argument? One would suspect so since &rest is allowed to be not copied.

<comment><USER7> Even if it is you still have to cons up all those arguments whereas reduce doesn't.

</comment></comment></comment></comment><comment><USER8> His first accumulate misses what is to me the most obvious way:

(defun accumulate-f (n) (let* ((end (list (next))) (n (1- n)) (start end)) (dotimes (_ n start) (rplacd end (cons (next) nil)) (setq end (cdr end)))))</comment>


Foundations of Physical Law - <AUTHOR> http://www.liv.ac.uk/physical-sciences/events/fpl/ <comment><AUTHOR> Video lectures starting here: [https://www.youtube.com/watch?v=W2XdhzCORbo](https://www.youtube.com/watch?v=W2XdhzCORbo)</comment>


Factors you must consider before buying SaaS for your business - <AUTHOR> https://myoperator.co/factors-consider-buying-cloud-software/ <comment><USER2> Great post! Will definitely help my startup.

<comment><AUTHOR> Thank you kanishkr. Can you think of any other factor that can be taken into consideration.</comment></comment>


Deterministic Parallel Ruby - <AUTHOR> http://www.cs.rochester.edu/u/scott/papers/2014_PLDI_DPR.pdf

<comment><AUTHOR> This is a fork of JRuby that allows you to build programs that are parallel but are also deterministic, avoiding one part of the difficulty in writing parallel programs.

Source code: [https://github.com/RB-DPR/RB-DPR](https://github.com/RB-DPR/RB- DPR).

<comment><USER2> So basically this will provide a nice DSL to deal with shared state?

<comment><AUTHOR> I'd say instead it provides a guarantee of the safety of your existing shared sate. Normally you write a shared state program and you just have to convince yourself it's correct. This enforces that correctness.</comment></comment></comment>


Hacker News Tokyo Japan Meetup #7, Friday 26th of August - <AUTHOR> http://www.makeleaps.com/blog/en/2011/08/english-hacker-news-tokyo-japan-meetup-7-friday-26th-of-august/

<comment><USER2> Registered. I'll be coming up from Osaka, but I'm feeling a little poor at the moment. If anyone happens to have a floor on loan I would I would be most appreciative :); my email is email@example.com. Looking forward to meeting up with the Japan crowd for the first time.

<comment><USER3> You've got mail.

</comment></comment><comment><USER4> See you all there.

</comment><comment><USER5> Nice, it's been a while.

(Anyone looking to celebrate Ramendan[1] pre-game, let me know.)

[1] <http://ramendan.com>

</comment><comment><USER6> Bummer, going to miss this by just a couple of days. I'm in Singapore right now.

</comment><comment><USER7> Damn, I can't go. I have very few leave days left for this year at my day job.

</comment><comment><USER8> Want to go,but I'll be out of town. Next time guys!

</comment><comment><USER9> Will be my first one, but see you there.

</comment><comment><USER10> Looking forward to it.

</comment><comment><USER3> See you all later :)

</comment><comment><USER11> Cutting it a little close with sending out the location aren't you? Or did I miss it somehow?</comment>


Ask HN: Which IMAP providers do you recommend? - <AUTHOR>

With the recent stories about Google deleting accounts, and with one of my parents losing access to their Yahoo account for no apparent reason, I'm planning to leave free email services behind.<p>What IMAP providers have you used in the past that you can recommend? I'm looking for a paid account that I can use with my own domain. Good spam filtering would also be a plus. <comment><USER2> <http://www.fastmail.fm/>

They've been around a long time.

</comment><comment><USER3> I've liked and used pair.com and fastmail.fm. Today I'd go with fastmail if all you want is email.</comment>


A Cloud Guru Has Acquired Linux Academy - <AUTHOR> https://twitter.com/acloudguru/status/1206692643637473281 <comment><USER2> So we don't to worry about lab environment anymore while getting acloudguru courses

</comment><comment><USER3> Backed by Bain Capital</comment>