Privileged Ports Are Expensive (2016) - <AUTHOR> http://adamierymenko.com/privileged-ports-are-causing-climate-change/ <comment><USER3> In terms of security/isolation, processes, users, containers and virtualization are all essentially the same thing. I wish the people working on these things would step back and notice the forest for the trees.

Whatever the ultimate "isolation unit" ends up being, it needs to be recursive. That means being able to run processes _within_ your process (essentially as libraries), create users _within_ your user account (for true first-class multi-tenancy), or VMs _within_ your VM (without compounding overhead).

It turns out that this author also wrote "Docker: Not Even a Linker"[1] which was also deeply insightful about unconscious/accidental architecture decisions. I'm impressed by his insight and disturbed that most people don't seem to understand it.

[1] [https://news.ycombinator.com/item?id=9809912](https://news.ycombinator.com/item?id=9809912)

<comment><USER4> Let me take that one step further:

>processes, users, containers and virtualization are all essentially the same thing.

...and so are modules/objects/whatever your language of choice calls them. Abstraction boundaries, to be precise. Abstraction, security, and type-safety, are all _very_ closely related.

These language-specific mechanisms for isolation _are_ recursive - <USER2> so. And language runtimes and compilers make security cheap - so cheap that it's ubiquitous.

Processes, users, containers and virtualization all rely on an operating system for security, which in turn relies on hardware features. Specifically, virtual memory and privileged instructions. And those hardware features are slow, and more importantly: they're not recursive!

But hardware-based isolation does have one key advantage over language-based isolation: It works for arbitrary languages, and indeed, arbitrary code.

I completely agree that recursive isolation is necessary. We need to figure out rich enough hardware primitives and get them implemented; or we need to migrate everything to a single language runtime, like the JVM.

<comment><USER3> Great point. The JVM tried for this position and failed IMHO (I think it abstracted too much). Now the browser is slowly honing in on it, and it might succeed (mostly due to sheer inertia). As opposed to the JVM, I like to call the ultimate goal the "C Virtual Machine" (just process isolation++).

I think moving isolation out of hardware is really important (both to make it recursive and portable). NaCl is an interesting step in that direction. If you could use something like it to protect kernelspace (instead of ring 0), syscalls could be much, much faster.

There's another problem with language-based isolation: it makes your language/compiler/runtime security-critical. Conversely, NaCl has a tiny, formally proven verifier that works regardless of how the code was actually generated, which seems like a much saner approach.

I'll also say that I don't think it's reasonable to expect every object/module/whatever within a complex program to be fully isolated (in mainstream languages at least). There's no need for it, and it will have too much overhead (in a world where objects in many languages already have too much overhead). Better to start relatively coarse-grained (today the state of the art is basically QubesOS), and gradually improve.

<comment><USER5> "NaCl is an interesting step in that direction. If you could use something like it to protect kernelspace (instead of ring 0), syscalls could be much, much faster."

It's actually partly inspired by how old security kernels work mixed with SFI. The first, secure kernels used a combination of rings, segments, tiny stuff in kernel space, limited manipulation of pointers, and a ton of verification. Here's original ones:

[http://www.cse.psu.edu/~trj1/cse443-s12/docs/ch6.pdf](http://www.cse.psu.edu/~trj1/cse443-s12/docs/ch6.pdf)

A Burroughs guy who worked with Schell et al on GEMSOS and other projects was the Intel guy who added the hardware isolation mechanisms. They were originally uninterested in that. Imagine the world if we were stuck on legacy code doing the tricks no isolation allows. Glad it didn't happen. :)

Eventually, that crowd went with separation kernels to run VM's and such that market was demanding. They run security-critical components directly on the tiny kernel.

[https://os.inf.tu-dresden.de/papers_ps/nizza.pdf](https://os.inf.tu- dresden.de/papers_ps/nizza.pdf)

The SFI people continued doing their thing. The brighter ones realized it wasn't working. They started trying to make compiler or hardware assisted safety checking cost less with clever designs. One, like NaCl and older kernels, used segments to augment SFI. Others started looking at data flow more. So, here's some good work from that crowd:

[http://dslab.epfl.ch/pubs/cpi.pdf](http://dslab.epfl.ch/pubs/cpi.pdf)

[https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/](https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/)

[https://www.microsoft.com/en-us/research/wp- content/uploads/...](https://www.microsoft.com/en-us/research/wp- content/uploads/2006/11/dfiOSDI.pdf)

So, have fun with those. :)

</comment></comment></comment></comment><comment><USER6> One of my biggest frustration while running a multi-tenant system running NixOS (Which is great btw, every user can install stuff in their home without sudo) is that HTTP is bound to port 80. And beyond the whole privileged port jazz, there's another trouble here : only one program can listen on myip:80.

Ideally, HTTP would use SRV DNS records. For the uninitiated, those are records that contain both an IP and a port, so instead of having a "default port" of 80 (which is completely arbitrary), you get to define on what port that service is running. Then I could just assign each user of my multi-tenant system a range of ports (With 1000 port each, we could get 65 users. Maybe less due to ephemeral ports, but it's already more than what I need anyway).

There are other solutions, such as machines connected to IPv6 could get millions of IP address essentially for free. But IPv6 coverage is still spotty. And in my case, the system was running on a kimsufi box, which gives exactly _one_ IPv6 address per machine. (And let me rant for a moment and say that this is __really __stupid, for multiple reasons such as ipv6 blacklist using blocks anyway).

I wonder if it's too late to get SRV records in, say, HTTP2. Or even in HTTP1 for that matter, as an amendment RFC. Because it would trully be awesome.

<comment><USER7> A reverse proxy can kind of act as a nice hack around this by multiplexing off of the the Host header (which is unencrypted even with HTTPS due to the way that the SSL handshake is done).

<comment><USER8> The HTTPS Host header is not unencrypted. You are talking about SNI which is an extension to TLS that adds the hostname as part of the handshake. The Host header is still encrypted when using SNI.

</comment></comment></comment><comment><USER9> In Linux the thing the author wants is /proc/sys/net/ipv4/ip_unprivileged_port_start which defaults to 1024 but can be set to anything you like. Such as 0.

Edit: I didn't realize how new that was. Kernels 4.11+ only. I think some people were using this on custom patched kernels though because I've been seeing it around. Was committed in January.

[https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4548b683b78137f8eadeb312b94e20bb0d4a7141)

<comment><USER10> No, the things that would have obviated the author's specific example were (a finer-grained version of) ip_unprivileged_port_start and SRV records _back in the 90s_ , when the customs actually developed.

<comment><USER11> That's the whole point about path dependence. We went down this _massively_ more complex and expensive path because we didn't do a few _teeny weeny little things_ to network interfaces and permissions back in the 90s.

</comment></comment></comment><comment><USER12> > Reproductive organs are what evolutionary biologists would call a highly > conserved system, meaning they don't tend to change often.

No. Reproductive genes are well-known for what evolutionary biologists (such as myself) call _positive selection_ , meaning they tend to change more often than you would expect.

See, for example, [https://www.nature.com/nature/journal/v403/n6767/full/403304...](https://www.nature.com/nature/journal/v403/n6767/full/403304a0.html)

<comment><USER13> Would anyone mind posting the PDF for those of us who can't afford it but would like to study it?

<comment><USER14> from scihub: [http://moscow.sci- hub.cc/7d2cc404e2efff26c37a6dd544258956/wu...](http://moscow.sci- hub.cc/7d2cc404e2efff26c37a6dd544258956/wu2000.pdf)

<comment><USER13> Thanks!

</comment></comment></comment></comment><comment><USER15> > Step two: extend user and group permissions and ownership into the realm of > network resources, allowing UID/GID read/write/bind (bind in place of > execute) permission masks to be set for devices and IPs. By default a bind() > call that does not specify an address (0.0.0.0 or ::0) will listen for > packets or connections on all interfaces that the current user has the > appropriate permission to bind and outgoing connections will default to the > first interface the user owns.

I think that this can be achieved with network namespaces and some iptables magic, entirely in userspace (and without the whole burden of containers).

Anyway, IMO the point of containers is not network virtualization, but rather isolation of dependencies. It's much easier to just pack everything into a container than to invent proper non-root package manager (like Nix).

<comment><USER16> The point of directories is to provide proper isolation of dependencies. But that followed the same path.

What leads to... It's better to add non-root mounting into the author's list. I don't think there's even anything that needs to change, permissions are already there.

</comment></comment><comment><USER17> It's not just ports, resource contention in general is largely the pain of multi-user systems.

If you've ever managed a host with thousands of users, you'd remember. Sure many of these have work-arounds now, but I've seen users (or have personally) run a machine out of:

\- /tmp space

\- processes

\- open file handles

\- inodes

\- shared memory segments

\- ephemeral ports

\- etc., etc.

You ever have a root shell on a box that's out of processes. Luckily modern shells have built-ins, so you can at least `ls` and stuff. But when you can't run, say, `lsof`, what do you do? That used to be a classic interview question.

And then there's the traditional resource contention folks think of like saturating links, i/o channels, disk space, cpu, memory, and so on.

But nowadays it's virtual machines all the way down. My favorite is to think of something like Puppet server (I like to pick on it) running in Docker. Count the abstractions in the chain:

\- JRuby interpreter

\- JVM

\- Process

\- Container

\- OS

\- Virtual Machine

\- Hypervisor process

\- Hypervisor OS

\- even UEFI to an extent

\- Hardware

You could say things like the process and OS are redundant, and you could probably add more if you consider some of the subcomponents of some of these. But each one is there to segment and/or simplify and then we add all kinds of holes for each to directly access one of its parental chain.

It is an evolved system where we are doomed to "reinvent Unix badly". But I'll be curious to see where we end up in 10-15 years...

</comment><comment><USER18> Somewhat related, but I always thought it would make sense for browsers to use DNS based service discovery to find the correct HTTP port, and then fall back to 80/443\. That alone would make it easier for users in a multi-tenant environment to self host, IMO, and eliminate some of the need for SNI.

<comment><USER19> This is what SRV records were (largely) created for. There have been a couple of attempts at making this a reality:

[https://tools.ietf.org/html/draft-andrews-http- srv-01](https://tools.ietf.org/html/draft-andrews-http-srv-01) [https://tools.ietf.org/html/draft-jennings-http- srv-05](https://tools.ietf.org/html/draft-jennings-http-srv-05)

Unfortunately, HTTP is just too widespread of a protocol - you would end up having to listen for legacy clients on 80/443 forever, making it a nonstarter.

<comment><USER20> The reality is that this is not that infeasible a change as everyone wants us to think that it is. It is simply a change to the DNS resolution logic for HTTP/HTTPS. The major browser vendors could make the change in relatively short order. The major hurdle is that it would increase the number of DNS queries to resolve a website.

XMPP is an example of a protocol that currently has this correct, in that it uses SRV records to control ports for a given host.

<comment><USER21> Why would it involve extra DNS queries? Why wouldn't the SRV record come back as part of the original response to the host-name resolution request?

<comment><USER22> There is no such thing as a "host-name resolution request". A DNS query specifies a domain name and a record type, and gets as the result a list of all records of that type under that name. Record types would be A (IPv4 address), AAAA (IPv6 address), MX (mail exchanger name), SRV (server name and port for a particular service), TXT (free-form text), and many others, most of them nowadays unused.

<comment><USER23> You can specify "ANY" as the type in the query and get all results. Try running "host -a ycombinator.com". It doesn't recursively resolve CNAMEs like an A query does, though. Also, at least Cloudflare refuses those requests, to reduce DNS reflection attacks.

<comment><USER24> "any" does not mean "all". People regularly make this mistake. One cannot do these sorts of tricks with ANY queries.

* [http://jdebp.eu/Softwares/djbwares/qmail-patches.html#any-to...](http://jdebp.eu/Softwares/djbwares/qmail-patches.html#any-to-cname)

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I'm the original author.

I wrote that a while back. My opinion hasn't changed too much, though I have to say that article could stand a rewrite. Don't really have time right now.

The real crux of the article is less about privileged ports and Unix permissions than about path dependence and how it leads to complexity explosions in systems. Instead of building a fix for X, maybe we should first question whether X really has to be that way and if there exists some simpler path to achieving our goals that involve some amount of change but far less complexity.

Sometimes you can't do that, but sometimes you can. I think privileged ports are a case where we could have easily eliminated a lot of complexity and headaches by just eliminating an obsolete feature.

<comment><USER25> Please search this HN comment thread for "jail" and my own comment - I am curious about your thoughts ...

<comment><USER11> Jails are closer. They are perhaps a way of achieving some of this. The problem (unless I'm wrong) is that you need root to create one, so you are back to needing root for everything.

<comment><USER25> Yes, of course the base systems root is necessary to create the jail, but then the jail has its own root user as well as its own /etc/passwd (and /etc/everythingelse).

For many, many purposes (almost all ?) a FreeBSD jail is indistinguishable to the root user from a bare metal server.

</comment></comment></comment></comment><comment><USER26> > How did we end up with the nested complexity explosion of > OS->VM->containers->... instead of the simplicity of multi-user operating > systems?

> [...], it pushes about 5-10 megabits of traffic most of the time but the > software it runs only needs about 10mb of RAM and less than a megabyte (!!!) > of disk space. It also utilizes less than 1% of available CPU.

> The virtual machine it runs on, however, occupies about 8gb of disk and at > least 768mb of RAM. All that space is to store an entire CentOS Linux base > installation, [...]

The exo-kernel and uni-kernel people have the opposite idea: go all the way to virtualization, and eliminate the traditional operating system.

You can still have all the conveniences of the programming models you like, but eg file-systems and network protocols will be implemented in user level libraries.

<comment><USER27> I can easily see a world where history repeats itself here. VMs get thinner, hypervisors get fatter to make up the difference, all the other layers of abstraction get squished out of the system, and what we're left with is just another process model. It's a model built on different assumptions than the one we're used to, with stronger guarantees about isolation than we might be comfortable with at first, but one that has an appeal even for desktop computing.

Later, once everyone has a hypervisor in their pocket, we tell our children about how cloud infrastructures used to be so expensive that only big corporations could maintain them, and they came in server racks the size of refrigerators.

<comment><USER26> No, the idea is not to make hypervisors fatter. Just the opposite: put the abstractions into userspace.

</comment></comment></comment><comment><USER28> This is one of the reasons why the kubernetes network model[0] is kinda neat. Every service running in each pod can bind to port 80 without any conflicts because every pod gets a dedicated ip address.

[0] [https://kubernetes.io/docs/concepts/cluster- administration/n...](https://kubernetes.io/docs/concepts/cluster- administration/networking/#kubernetes-model)

<comment><USER29> Right, I read this article as "IPv4 is causing climate change." If you have enough IP addresses, dedicated port conventions don't bother me at all.

<comment><USER30> Yeah, but even if each user has their own dedicated IP, they can't bind to a privileged port unless they're root. So it's still a problem, no?

<comment><USER31> No, not at all. This problem was solved decades ago via a myriad of different workarounds.

You can simply turn off privileged ports if you feel like it. Or do things like setuid such as how Apache starts as root but switches to userspace immediately.

It's not ideal, but to say the privileged port thing is an issue is bizarre to me. It's the least interesting item the author brings up in the article, and certainly was _not_ the primary driver behind multi-tenancy virtualization.

Better yet is using reverse proxies - service providers have been doing this for ages. A single shared HA cluster of haproxy, that maintains records for each individual application/tenant that lives wherever it likes. This is the model I prefer, since it allows me (as a sysadmin) to protect the developers from themselves by being able to easily filter at layer 7 in front of their application as needed. Also lets you direct traffic easily, and is generally the model used by any container orchestration service.

Very easy to expose all that via various APIs and tooling so users can self- service.

</comment></comment></comment></comment><comment><USER32> It was a fun read, and there is a clever thought there which I'll get to in a minute. But first ...

The author gets a lot wrong, it isn't uncommon because most software people see a "server" as a set of numbers "A" bytes of memory, "B" bytes of disk space, and "C" hertz of clock speed.

Computers aren't numbers of course, they are systems, and the systems are a collection of moving parts that are connected by linkages which constrain and amplify the various parts. In system design there is a concept of 'balance' which discusses how the linkages enable and constrain such that an understanding of the amount of "work" that can be done by the system is understood.

There is a great example in the article comparing a Raspberry Pi to a Sparcstation 10. Most of the software the author used "back in the day" is still around in one form or another, and a RasPi is cheap, so its easy to create modest recreation of the environment from that time and to build simulated users who can access the "server". Just two or three users and the Pi will "choke". Understanding how a system with so many "bigger" numbers is less capable of doing the same "work" as one with much smaller numbers is a useful exercise to run if you are into systems analysis. The system the Pi was _designed_ to run, and the one for which it is fairly well balanced is a smartphone. Something the SPARC 10 would truly suck at.

The clever idea however is making networking resources just another resource like disks. Interestingly enough, with IPV6 that is much easier than it was before (to the point about how things were done before in networking, vs now). It is straight forward to create a model of network interfaces that is equivalent to serial lines. Have the kernel get set an IPV6 subnet with a block of 32K addresses and let your user program open /dev/net/<1> through <32766> because its IP each "net" device comes with its own set of port numbers etc. IPC is just networking and a cleverly coded kernel running on a machine with a 64 bit virtual address space would do direct data placement. No need to do even any page table mashing.

<comment><USER33> > Just two or three users and the Pi will "choke".

Do you really know this? What's the limiting factor?

I've never played with a Pi and I don't really know, but I don't see offhand why it would be so bad. Interrupt-handling and context-switching times should be much better. There's plenty of main memory. The file system should be faster, running on flash. I don't see where the bottleneck would be.

<comment><USER34> Probably only limiting factor in the hardware is that R-Pi does not have any meaningful high-speed low-latency IO.

But on the software side there is one huge problem: today's Linux kernel and userspace is incredibly bloated in comparison to early 90's SunOS. On the other hand, said SunOS probably did not support many things that everybody expects today, like scalability to large-ish SMP systems (ie. >2 CPUs), shared libraries, loadable kernel modules, threads...

<comment><USER33> SunOS definitely had shared libraries -- I think those go back at least to 4.3BSD.

Anyway, how much I/O do you need to support people running terminal apps over Ethernet?

<comment><USER34> What I meant by the IO is that the aforementioned sparcstation had ethernet card connected to SBus which was capable of directly producing interrupts and doing DMA, and all the modems were connected through some multiport serial card/concentrator which tried very hard to offload stuff from the CPU. Which is completely different situation than R-Pi with everything hanging off USB with it's "novel" approach to interrupts.

</comment></comment></comment></comment></comment><comment><USER35> I just use iptables to forward all traffic on port 80 to port 8080. You can just replace your /etc/rc.local with this:

iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080

iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to- ports 8080

exit 0

Simple enough to add to a configuration management workflow. I used to maintain a configurable reverse proxy, until I decided that was way too much work and to only put one web facing app per server.

edit: rc.local may not be the best place to put these rules, check this link:

[http://bencane.com/2011/12/30/when-its-ok-and-not-ok-to- use-...](http://bencane.com/2011/12/30/when-its-ok-and-not-ok-to-use-rc- local/)

<comment><USER36> Unprivileged users can't use iptables, so that doesn't change anything about the blog post. I'd suggest that if it's so easy, but still doesn't get done, that still also leaves the blog post intact. In a way it doesn't help when there's a thousand ways to do something, so none of them get done.

This seems to be a common misunderstanding of the blog post. Yes, there's a ton of ways for root to create privileged ports, and a ton of ways to delegate it in various ways. But all of them require actions by root, and sysadmining, and aren't as secure as a system designed to work this way from the beginning would be, so nobody uses them, so in terms of addressing his discussion points, they might as well not exist.

<comment><USER37> But surely, creating the user requires sysadmin privs, and as part of setting up the user the appropriate work could be done to allow whatever hole-punching is required. The reality though is that if Mary, Joe and Tim all want to listen in port 80, there's going to be a mess - hence using a common server (on a privileged port!) to demux incoming requests.

</comment></comment></comment><comment><USER38> Even if port 80 was unprivileged, wouldn't you still run into issues if multiple users attempted to bind to the same port?

Really, what would the problem be with just deploying an nginx instance that listens on 80 and routes by domain?

<comment><USER39> Well, someone has to administrate the nginx routing config, or there needs to be a mechanism for each webapp sitting behind the nginx to register itself as a forwarding target.

What I want to know is why can't we get all the major web browsers to look for (and honor) SRV records, so that www.mydomain.com can transparently have the browser connect to port 30338 (or whatever port my app is listening on). Then we dispense with the need for proxies altogether.

<comment><USER40> All major protocols should, IMO.

</comment></comment></comment><comment><USER41> Isn't this exactly what shared hosting is? I bet companies like dreamhost regularly put 50 clients on a box. And Webfaction[0] will even give you pretty open shell access to a shared environment.

[0] [https://www.webfaction.com/](https://www.webfaction.com/)

<comment><USER42> Indeed this is exactly what webfaction does!

They get around the port issue by using a nginx frontend that uses the Host to route each request to the correct apache instance each user is running out of their home directory. All these apache instances are bound to higher unprivileged ports.

<comment><USER43> Imagine a world where you don't need a router! A world where the browser would look up a http/s service record for the host and connect directly to the unprivileged port.

</comment></comment></comment><comment><USER44> There are plenty of trade-offs but network resource ownership?

For a datacenter isn't it more efficient to pack workloads across dense machines than to over-provision idle ones?

While a factor, maybe, I don't think privileged ports are causing excess CO2 production.

Although I've been interested in low-power edge networking with uni-kernels, would be an interesting project to work on (thinking micro-dc's with homogenous ARM-based boards at the edge and having a dynamic system to boot up uni-kernels on demand closest to the requesting user... a system that mostly stays off unless it's needed).

</comment><comment><USER45> The biggest change which would have made this much easier in my mind is adding port numbers to DNS. Instead of having HTTP at port 80, have the port number for HTTP returned as part of the DNS query. You can allocate a block of ports to each tenant on a host and let them use them as they see fit, you don't need to use the low ports for anything.

<comment><USER46> [https://en.wikipedia.org/wiki/SRV_record](https://en.wikipedia.org/wiki/SRV_record)

SRV solves this problem nicely, unfortunately SRV support for HTTP 2.0 was rejected.

</comment></comment><comment><USER47> I'm pretty sure most or all of this port/sudo nonsense was fixed with Plan 9.

<comment><USER48> Well, ports are just files and have a owner, group and permissions. As simple as that.

</comment></comment><comment><USER49> > alter package managers to allow installation of packages into a subtree of > the user's home directory if the user is not root, etc.

Guix and Nix. I haven't used Nix, but Guix even allows ad-hoc containers running only specific programs and their dependencies:

[https://www.gnu.org/software/guix/manual/html_node/Invoking-...](https://www.gnu.org/software/guix/manual/html_node/Invoking- guix-environment.html)

That doesn't help with the privileged port problem, but per-user services can be dealt with.

</comment><comment><USER50> This work was already done in other "Unix" environments like Solaris and BSD (zones and jails)

My personal favorite is SmartOS (based on illumos) it runs zones on bare metal and even has support for running Linux containers (docker)! It does this by wrapping the Linux syscall APIs and translating them. Magic stuff!

Launch a Linux native container and run 'ps' and you will only see processes that you own!

reference: [https://wiki.smartos.org/display/DOC/Home](https://wiki.smartos.org/display/DOC/Home)

<comment><USER25> I came here to say just that ...

A FreeBSD jail does not emulate or create a virtual machine - it's just a fancy chroot mechanism that produces only the unix processes that actually get run inside the jail.

A jailed httpd does not take up any more resources than the exact same httpd run on the base system.

This makes it extremely efficient and, in fact, allows you to create an _even richer multi-user platform_ than the original one the op has nostalgia for: a multi user unix system where everyone gets to be root.

</comment></comment><comment><USER51> I've long thought that a combination of NixOS and illumos would be incredibly useful. Software would be installed into the nix store in the global zone, and mapped into dedicated zones for each user. IP and port virtualization via Crossbow.

User zones would be extremely light, with your typical database-driven web app taking _no_ space in the zone, and very little space in the nix store.

</comment><comment><USER52> The "privileged port" model was basically dead at the time of the Morris worm 30 years ago. It should have been sorted out in that time but there was never the momentum.

Everything in UNIX is a file, apart from the things that aren't. If ports _were_ files, you'd be able to chown them or put them in a group for delegation purposes.

</comment><comment><USER53>
        Container solutions like Docker get us part of the way there. In a
        sense you could argue that containerization is precisely the
        multi-tenancy solution I'm heading toward, except that it borrows
        heavily from the legacy path of virtualization by treating system
        images like giant statically linked binaries.

Nailed it. This is why Docker does not excite me at all, and why I think there's room for other container systems to improve up on the Docker model by solving this problem. I made my initial attempt awhile ago by adding basic container support to a package manager that allows users to use a virtualenv- like tool to create containers to hack in:

[https://www.gnu.org/software/guix/news/container- provisionin...](https://www.gnu.org/software/guix/news/container-provisioning- with-guix.html)

<comment><USER54> "Container solutions of the kind that are like docker" vs "Container solutions in general of which docker is an example". If he is using docker to write-off all container solutions, then that's a mistake, since it sounds like using the underlying container solution (on which docker then adds the ability to manage system images) is exactly what he wants.

<comment><USER53> Yeah, perhaps namespaces will satisfy them, but from what I've seen network namespaces leave something to be desired. I found that it wasn't too hard to roll my own container implementation and hook it up to a tool that was already providing software environments to unprivileged users, so I think there's plenty of room for other people to take the primitives that Linux provides and run with them.

</comment></comment></comment><comment><USER55> His proposed solution actually looks a lot like one early flavor of LXC implementations, where instead of unzipping a big tarball with your own environment, you'd just bind-mount system folders into the container. I suspect LXC devs are reading this post and thinking "we already have this!"

</comment><comment><USER7> The issue with apt install has more to do with a poor packaging mechanism on Debian's part rather than networking. Windows installers for once are actually superior in this regard since they generally provide an option to install for either the system or only the current user.

Also, to provide multi-tenant hosting, you could just create some sort of root level reverse proxy (possibly Nginx or Apache) with some sort of registration system for domains. You just use the Host header to multiplex the system's port 80/443\. For localhost stuff, I think you could set up something with DNS to be [http://{username}.localhost](http://{username}.localhost). It would be kind of an interesting idea to do as a happy medium between serverless and separate VMs.

<comment><USER11> Apple had the right idea with .app bundles. There should be no such thing as "installation." Android and iOS kind of get it right too.

<comment><USER56> Problem with Apple's .apps is that they still manage to mess up your ~/Library. You can often find gigabytes of data left in there on your average Mac.

</comment></comment></comment><comment><USER57> Easy solution: multiple IP addresses (one per user), and setuid wrapper to open listening socket on port 80 of the user's IP address and pass it to the web server after dropping privileges. No containers, VMs, or redesign of Unix needed.

Or put them all on different ports and front it with a proxy. Run only one MySQL instance.

Of course, containers and VMs aren't meant to solve this problem anyway. Containers are about deployment, and VMs are about virtualization/migration. Your customers would do well to use the former (for ease of deployment), and you the latter (for ease of maintenance).

Privileged ports really have nothing to do with this problem.

</comment><comment><USER58> Great articular... But, forgetting the (apparently GBs) of size, isn't this really what docker and such like are trying to achieve? (maybe I'm missing something). Especially when it comes to only storing a single instance of the base image if all 50+ users are sharing the same image? So it will only store the differences in each container. It sorts out your network issues, user can be root in the container and bind to whatever port they like and IP addresses are assigned to each container... What other overheads are there? (this is a rhetorical question - you seem much more knowledgeable than myself) :)

<comment><USER58> Just a quick apology - that was meant to read "this isn't a rhetorical question".. I was truly asking :D

</comment></comment><comment><USER59> Why not just run a single web server and set it up to serve public_html from people's home folders?

The difference to that multiuser system is that we want people to manage their own infrastructure. That's the real reason for the bloat - no matter how much that VM server costs, I bet you it costs less than paying people to administer a shared infrastructure. It has nothing to do with ports.

<comment><USER4> >Why not just run a single web server and set it up to serve public_html from people's home folders?

Doesn't work with dynamic content beyond CGI, which is too slow and usually not supported by modern web frameworks.

<comment><USER60> Could you elaborate on why it doesn't work with dynamic content beyond CGI? Is this some kind of insurmountable limit, or is it just the web frameworks don't care to support it?

<comment><USER4> It's not an insurmountable limit; it's not the web framework's fault either. It's that there's not a way for a individual user to indicate to a shared system-wide webserver, "Here is the FastCGI (or whatever) socket that you should connect to, to generate dynamic content for my home directory." But certainly a way to express that could be created.

</comment></comment></comment></comment><comment><USER61> Service location in the TCP stack is a bad idea. Why port 80 _every_ time for http? Put that into DNS SRV! Boom! IPv4 crisis solved (for now)

</comment><comment><USER62> I'm happy to know I'm not the only one who wants the death of privileged ports. When I proposed this for NixOS I wasn't exactly well received: [https://github.com/NixOS/nixpkgs/issues/11908#issuecomment-2...](https://github.com/NixOS/nixpkgs/issues/11908#issuecomment-250979432)

<comment><USER11> "systemd will handle that"

No no no handling with complex logic what can be achieved instead with simple design is how Windows got its suckage. Sigh.

</comment></comment><comment><USER63> I was a little bit annoyed (enough to comment) by the use of the wrong units. 'mb' is millibit, perhaps millibyte, but certainly not megabyte (MB or MiB).

Similarly:

> it pushes about 5-10 megabits of traffic most of the time

Bits are a unit of information, not flow. Probably the author meant Mib/s.

<comment><USER64> This annoyed me too. The author does not seem to be aware that case is highly important when dealing with SI/binary prefixes as well as unit abbreviations. If you don't even know (or care) that "b" is a bit and "B" is a byte, and you use them incorrectly, how am I supposed to trust your technical knowledge about the rest of the stuff you're talking about?

<comment><USER40> The author is the creator of ZeroTier. He is likely aware and didn't care because it's extremely pedantic and most people know what he means in informal settings, which this is. I expect he'd use the correct case in an RFC or something.

<comment><USER64> Getting abbreviations for bits/bytes correct is not "extremely pedantic", it's about communicating correctly. In a network context, which this _is_ , "b" means bits, but he was using it to refer to bytes. Also I've never heard of ZeroTier, but even if I had, I probably would not have made the connection that he was the author of it, so being correct about these things is important for establishing credibility with new audiences.

<comment><USER40> IIRC, most if not all uses of 'mb' or 'gb' were about disk space or RAM, so not really a network context - this is about bloat, ports are just the whipping boy.

And I'm not against pedantry in the right context, but this is just a casual, relatively nontechnical rant. Pedantry is really not needed.

</comment></comment></comment></comment></comment><comment><USER65> The original post seems a bit unix-centric. Much of what he is complaining about comes from the nature of TCP/IP networking, Unix can only be blamed to the extent that it doesn't abstract things like ports away.

The general thrust of his post is to fix Unix so that its (ohh so well respected!) permission model can extend across into the cloud. This will run into friction when more than one OS is involved.

That said: anything that unifies all the different kinds of virtualization as the OP wants would need to be an OS. And the only plausible candidate for an OS lingua-franca in this world is Unix.

So long live unixcentricsim?

</comment><comment><USER66> Since we're here anyway: if we would just change web browsers to connect to a default port number obtained via DNS instead of always using 80/443 by default, then we could offer web servers through NAT gateways (by hosting up to 65535 web servers on a single IPv4 address). We'd need some way to tell the NAT we want to expose a port (some variant of uPnP or some less-dumb protocol), but other than that, it would be easy. And then we'd have effectively 65535x as many IPv4 addresses, which would be enough for everyone, permanently, and IPv6 wouldn't be needed.

<comment><USER67> What? This isn't necessary at all. HTTP includes the requested hostname in the request so you can have as many DNS names as you want point to a single IP. No NAT or other magic necessary. The entire HTTP Internet could be hosted on a single IP.

<comment><USER68> True but if the sites were served by different processes then there would have to be a trusted reverse proxy process to dispatch.

</comment></comment></comment><comment><USER69> I'd point out that currently, access to port 80 for a domain is access to a certificate for that domain.

This means that, stupid or not, HTTPS essentially is as secure as privileged ports are.

<comment><USER70> Slightly more secure, since you can have multiple A records on a domain and your domain verification should really check them all.

It's more fair to say HTTPS is basically as secure as DNS. If someone hijacks your DNS for 5 minutes they can have a TLS cert from LetsEncrypt for your domain for 90 days

</comment></comment><comment><USER10> > _(USB is slowly replacing the lighter plug, but most cars still have them.)_

And yet, that's just trading for different path dependence annoyances. 5V, 500mA limit (or proprietary higher-voltage hackland), and that feeling of connecting your valuable phone to a low-bid switcher? Airports, cars, home outlets - I'll choose the appropriate third party device every time! I just wish that car power socket wasn't so dildonic so that more outlets would come already built in.

</comment><comment><USER71> Ultimately the issue is that IPv4 addresses are expensive. Allowing multiple users bind to port 80 on different IP addresses, assume you can have multiple IP addresses on that IP. Unless they are private IPs, that's going to be the main issue.

Any practical solution would need a system wide nginx to proxy to all of the tenants, who would run their apache/nginx/node etc. on a high numbered port.

</comment><comment><USER72> > How did we end up with the nested complexity explosion of OS->VM->containers

If someone is putting containers into VMs they've eliminated the performance benefits of containers and added an additional layer of complexity for no reason: ie they don't know what they're doing and wanted to run Containers on their existing VM only cloud platform.

</comment><comment><USER73> does [https://github.com/redox-os/redox](https://github.com/redox-os/redox) address any of these pain points?

i asked: [https://github.com/redox- os/redox/issues/987](https://github.com/redox-os/redox/issues/987)

</comment><comment><USER74> Well, FreeBSD's mac_portacl(4) has been around since 5.1-R (June 2003) and allows per-user ACLs on privileged ports. Although the permission is for all IP addresses, not a specific one. But one could create a virtual network device per user and assign it a mac_mls policy to restrict that interface to that user... hmmmmm...

</comment><comment><USER75> We also currently have technology, and could have allowed HTTP/2.0 to allow for this, if we just used SRV records.

If we had SRV records, you could whatever damn port you want, and it would be invisible to users.

It would also allow us to not have to get load balancers for most (definitely not all) setups - but that is different rant.

</comment><comment><USER14> This wouldn't change anything because you cannot run multiple service instances on same port so users still would have to specify port in URL. SRV record in DNS could help if anything supported it. But it is not needed now because with IPv6 we have enough addresses to assign each process unique IP.

</comment><comment><USER76> Good enough + soon enough beats "best possible", or even "better later".

In this case Ethernet + TCP/IP beat OSI.

Also, JavaScript beat any number of sensible scripting languages. Houses are frequently constructed such that they are not very serviceable.

<comment><USER77> I'm not sure your example matches your mantra. OSI was usable before ISPs became a thing, you ran OSI over X.25.

<comment><USER76> X.25 isn't the whole networking stack. Elements of OSI were available, and are still available, but most people use something else because it's easier: good enough, soon enough.

<comment><USER77> I was developing for OSI at a time when it was not possible to do equivalent things on a WAN in Europe using TCP/IP.

TCP/IP didn't win by being first.

<comment><USER76> How did TCP/IP win?

<comment><USER77> I'm not sure that the OSI people even realized that they were in competition with anything else. I think they basically neglected the low end of the market.

At the time there were multiple LAN protocols that were mostly used for file sharing, Netware, Appletalk, NetBEUI, etc. You had NFS for TCP/IP but it wasn't really used for anything other than UNIX workstations.

Probably it would have required somebody to write a cut-down OSI stack for MS- DOS that could be linked to a particular killer application (whatever that was, maybe FTAM).

This would still have had to compete with the way that TCP/IP was able to swallow up the other LAN protocols, we wouldn't have had to go through the IPv4 to IPv6 migration though.

</comment></comment></comment></comment></comment></comment><comment><USER78> Umm, the real problem is that I can't discover what _port_ you are on.

So, in the article's instance, the machine owner needs to set up a web server on default port 80 that redirects to the correct web server on the correct non-privileged port.

<comment><USER43> Sure you can, this is the entire point of service discovery. We've built so much crap to deal with the fact that web browsers don't resolve SRV records (and at least for HTTP/1.1 prohibited by RFC from doing so).

</comment></comment><comment><USER64> The article does not mention backups or standardized images at all. These are huge reasons why virtualized machines are attractive, and which multi-tenant OSes don't provide.

</comment><comment><USER79> In the wordpress example you could use named pipes/sockets, but it's rather complex to manage. A better idea is to hand out ipv6 addresses and let users bind to them.

</comment><comment><USER80> Why was the original title ("Privileged Ports Cause Climate Change") changed ?

<comment><USER48> I guess because it's quite silly, grossly imprecise and click-bait-y.

</comment></comment><comment><USER81> If DNS returned IP:port for hostname, that would sort it too.

www.foo.com --> sharedserver:8000, www.bar.com --> sharedserver:8001 and so on.

<comment><USER82> It can, with SRV records, which is what things like ActiveDirectory and XMPP use to discover the correct server and port for a given name.

Its use for HTTP (and most other L7 protocols in general) just never really caught on.

<comment><USER81> I believe Consul can do this and it's a pretty nice capability but the effort of retrofitting it onto things that just assume well known ports would be phenomenal.

I mean even if you know you want http the "right" thing to do is look up the port number in /etc/services but who ever does that? We all just hardcode 80. So that's another potential technique that we can't use...

</comment></comment></comment><comment><USER83> This article makes no sense. Virtualization's first use was to run a guest OS on a different host OS. Containerization's first use was to achieve reproducible configuration at a lower cost than virtualization. Neither were trying to solve a multi-tenant hosting problem where everybody is running the same system -- were it that simple!

<comment><USER5> The first use of virtualization that I can remember was VM/370\. It did the things you said plus hosted systems for many users connecting via terminals. Supporting legacy systems and getting higher utilization with multiple workloads were also advertised benefits of VMware on x86. IBM and NUMA vendors touted isolation of multiple workloads or users with things like LPARS. The cloud providers now claim a lot of this stuff.

So, it seems like it was intended to do these things going back to the 70's. They also designed insecure, bloated solutions that INFOSEC founders like Paul Karger called them out for. Resulted in better designs like KVM/370, KeyKOS, VAX VMM, separation kernels, and recently mCertiKOS. Most peddling virtualization for multiple users still use bloated, untrustworthy components though.</comment></comment>


Proposed encryption policy for government of india [pdf] - <AUTHOR> http://deity.gov.in/sites/upload_files/dit/files/draft%20Encryption%20Policyv1.pdf <comment><USER2> >"Users within C group (i.e. C2C Sector) may use Encryption for storage and communication. Encryption algorithms and key sizes will be prescribed by the Government through Notification from time to time. All citizens (C), including personnel of Government / Business (G/B) performing non-official / personal functions, are required to store the plaintexts of the corresponding encrypted information for 90 days from the date of transaction and provide the verifiable Plain Text to Law and Enforcement Agencies as and when required as per the provision of the laws of the country."

So you can encrypt your data but any government/business communications must be retained in plan text for 90 days. Nice.</comment>


Meet the 7 Startups at StudentRND this Summer - <AUTHOR> http://www.geekwire.com/2012/meet-teams-student-rnd-summer-incubator/

<comment><USER2> I guess #6 (locatr) hasn't heard about <http://www.loc8tor.com/> :-)

I actually have a loc8tor unit and a bunch of fobs and they really do let you find things easily. Perhaps the coolest thing just just being able to answer the question 'is it in this room?' quickly and accurately.</comment>


Google GLASS Infographic – forbidden APPs - <AUTHOR> http://glass-apps.org/google-glass-apps-terms-of-service-meme-infographic

<comment><USER2> No adds, no charges... How is one supposed to make money?</comment>


Chaos Theory vs Clojure - <AUTHOR> http://bestinclass.wordpress.com/2009/09/24/chaos-theory-vs-clojure/ <comment><USER2> Awesome quote:

"If you're on Windows it's basically the same approach, _you just have to start by installing Linux_ , but afterwards you'll also get this result in all its animated splendor:"

(Emphasis mine)

<comment><USER3> It would be funnier if it were true. Instead this would run in the command line.

</comment></comment><comment><USER4>
       (-> #(animate panel) Thread. .start))

>I doubt you'll ever see such impressive Java Interop mixed with Closures in other languages!

Rhino is pretty impressive too:

      new Thread(function () { print("\nrun"); }).start()

<comment><USER5> Your Rhino example isn't creating a closure, is it?

</comment></comment><comment><USER6> Most interesting thing was two links away and unrelated:

<http://www.assembla.com/wiki/show/clojure/New_new>

</comment><comment><USER7> I'm not sure if this demonstrates great ability for Clojure.

One of the first programs I ever wrote, when I was still in High School, was a simple simulation of planetary motion using BASIC. For this, I used a naive approximation of Newton's laws. I was disappointed to discover that because of approximation error, my planets all fell into the sun! But the program did put the death-spirals on the screen in black and whie!

My impression is that while mathematical formulas generally aren't that hard to interpolate into any programming language, the art of such interpolation is making sure that arithmetic error and such don't pile up to spoil things.

Given this, Chaos Theory graphics would be great fodder for _seeming_ like you've done something amazing since you don't have to simulate anything correctly, you only have to create a swirl which looks intriguing. Oddly enough, planets orbiting a sun are better since you've got to create a useful simulation with the properties of the original system.

<comment><USER8> I've always wanted to write a planetary simulator. I think clojure might be a decent language for it, as long as you avoid floating point as much as possible and use clojure's Ratios class instead.

      user=> (type (/ 1 3))
      clojure.lang.Ratio

      user=> (* (/ 1 3) 3)
      1

I'd really like to see an arithmetic macro for clojure to allow infix notation, constants, and lazy evaluation. Something like (arith 2m/s^2) would be crazy awesome.

<comment><USER9> check out "the structure and interpretation of classical mechanics" by sussmann et al for a whole raft of things like this done in scheme.

I don't see why would you want to stay away from floats, using rationals isn't going to magically fix the numerical errors from approximating derivatives and such.

I believe they develop an automatic differentiation routine in SICM that will actually allow you to make a pretty good planetary motion simulator, i.e one that won't just go horribly wrong if you run it for a few thousand years.

Of course if you're really interested/mathematically inclined one of the best ways to simulate Hamiltonian systems is via iterated maps, not unlike what was in the article. A good (but quite hard) ref to this is "simulating hamiltonian dynamics" by Leimkuhler and Reich.</comment></comment></comment>


Life Could Continue to Evolve – Issue 88: Love and Sex - <AUTHOR> http://nautil.us/issue/88/love--sex/how-life-could-continue-to-evolve <comment><USER2> Please stop spamming HN. You have 33 submissions including this one twice in the same hour.</comment>


An Insider's Look Into Incident Management - <AUTHOR> http://blog.statuspage.io/an-insiders-look-into-incident-management-with-codeships-florian-motlik

<comment><USER2> Interviewee here (Flo), happy to answer any questions that might be coming up as well as a follow up. You can always send me an email to firstname.lastname@example.com or answer here, would be great to get the thoughts of the community on these topics</comment>


Concurrent Programming for Scalable Web Architectures - <AUTHOR> http://erb.io/thesis

<comment><USER2> downloaded! thx</comment>


Ask HN: How to retain core competency in math when your job doesn't require it? - <AUTHOR> <comment><USER2> Simple answer is past-times. For example, I have a side interest in ODEs and PDEs, which I basically never get to apply for work.

So I'm working on 3 problems from orbital mechanics, control theory and investment science respectively, each which is interesting in itself but each which flexes my ODE and PDE muscles.

In short, just invent problems you find engaging and then scratch away at them on planes and trains, and in all those in-between times.

</comment><comment><USER3> It depends what the OP call core math competency.

If you mean the ability to solve and think critically about problems, I like a combination of "How to solve it" by Pólya ([https://en.wikipedia.org/wiki/How_to_Solve_It](https://en.wikipedia.org/wiki/How_to_Solve_It)) for more general ideas and regularly finding puzzles onm different topics such as the ones proposed here ([https://fivethirtyeight.com/tag/the- riddler/](https://fivethirtyeight.com/tag/the-riddler/)).

For deeper mathematics, I feel like that's hard. I come back to my notes from Grad School once in a while, or try to follow proofs for topics of interests. But to be honest, most of them are a little beyond me at times. Maybe Fermat's library ([https://fermatslibrary.com/](https://fermatslibrary.com/)) can offer some annotated reading which would be helpful?

Regardless, I wish you good luck. Please do share if you have any more good ideas.

</comment><comment><USER4> 1\. Khan Academy

2\. Youtube (see: Professor Leonard, 3blue1brown, Gilbert Strang, etc.)

3\. Books like:

[https://www.amazon.com/Humongous-Book-Calculus-Problems- Book...](https://www.amazon.com/Humongous-Book-Calculus-Problems- Books/dp/1592575129/ref=sr_1_3?s=books&ie=UTF8&qid=1537278451&sr=1-3&keywords=calculus+problems&dpID=51K61QI- MPL&preST=_SX218_BO1,204,203,200_QL40_&dpSrc=srch)

[https://www.amazon.com/Calculus-Practice-Problems-Dummies- On...](https://www.amazon.com/Calculus-Practice-Problems-Dummies- Online/dp/111849671X/ref=sr_1_4?s=books&ie=UTF8&qid=1537278451&sr=1-4&keywords=calculus+problems&dpID=51nPdaLibEL&preST=_SX218_BO1,204,203,200_QL40_&dpSrc=srch)

[https://www.amazon.com/Schaums-Solved-Problems-Calculus- Outl...](https://www.amazon.com/Schaums-Solved-Problems-Calculus- Outlines/dp/0071635343/ref=sr_1_5?s=books&ie=UTF8&qid=1537278451&sr=1-5&keywords=calculus+problems&dpID=51tAGvcE80L&preST=_SY291_BO1,204,203,200_QL40_&dpSrc=srch)

[https://www.amazon.com/gp/slredirect/picassoRedirect.html/re...](https://www.amazon.com/gp/slredirect/picassoRedirect.html/ref=pa_sp_atf_stripbooks_sr_pg1_1?ie=UTF8&adId=A0504778C2N61BSH4IS6&url=https%3A%2F%2Fwww.amazon.com%2FChallenging- Calculus-Problems-Fully-Solved- ebook%2Fdp%2FB07H39K6BJ%2Fref%3Dsr_1_1_sspa%3Fs%3Dbooks%26ie%3DUTF8%26qid%3D1537278451%26sr%3D1-1-spons%26keywords%3Dcalculus%2Bproblems%26psc%3D1&qualifier=1537278450&id=7327685849459176&widgetName=sp_atf)

4\. And maybe not as calculus specific, but doing Project Euler problems might also be useful.

</comment><comment><USER5> I am not very good at classical mathematics. I am quite confident that there are better more intuitive ways of describing the same ideas, so until one such system emerges I tend to learn the concepts that are being expressed in mathematics and convert them away from mathematics to just a logical description of the operation. I haven't run across anything in calculus for instance that I can't describe in python with loops and arrays. One might argue that it is all math ultimately but that's fine so long as I can deal with the raw idea and not the idea expressed in terms of math operations.

<comment><USER6> You might be more comfortable with Python syntax than with standard mathematical syntax, but I wouldn't think of it as being closer to "just a logical description" or closer to being the "raw idea." You're just choosing to use different notation.

I would guess that most people comfortable with both representations would feel that standard mathematical notation is lighter and conveys the "raw idea" more directly.

</comment></comment><comment><USER7> If you're good enough then volunteer or hire yourself as a tutor or night college teacher.

Also, there are a number of people that produce calculus tutoring youtube videos. Maybe you can try that.

Unfortunately, a large percent of what we learn will eventually be forgotten due to lack of use. Most of what we learn will not be used at work. I remember spending tons of time studying calculus yet I've yet to use any of it. If ever I need to use it I will need to review it to refresh my mind but most likely I won't remember most of it.

</comment><comment><USER8> I believe the place of calc (just like trig) is overinflated in pre-real- analysis education. As far as I am concerned, sin is just a function defined a certain way and I ain't worried about SOH-CAH-TOA or anything to that effect. I don't think one requires 1000 page book to learn a few core calc concepts either. Derivative is a slope. That's it. It's the same value along the length of a line. The only problem is non-linear functions where slope changes along every point. In which case we bring in the machinery of tangent/secant lines and take a limit for a very intuitive reason, but the underlying concept remains the same. This informs the definition of derivative. From this point on derivative is just a function defined a certain way and in this regard is no different from any other function like indicator function or whatever. The rest of the talk about derivatives is often just a padding and general interest. Have you ever seen a 500 page book written about the step function?

When you take a bit more advanced class, the symbols for partial derivatives and integrals will be interspersed with other symbology and flash before your eyes like elements in a humongous matrix. You won't have time to think about stuff you learned from 1000+ page doorstop. You have to learn to think more nimbly and abstractly like a mathematician. To that end, check out "intro to math proofs" textbooks.

If by calculus you meant modern math analysis, then simply disregard the stuff above.

</comment><comment><USER9> It might depend on the person, but I've retaught myself some math stuff, and I think you relearn it a lot faster the second time. The real problem is not getting bored. It was a bit slow-going at first because I was out of practice, but after a little bit of settling in, it was alright. YMMV, it depends how well your long term memory works.

</comment><comment><USER10> I struggle with this so much and am glad you asked it. I reopen the books and things come back to me slowly, but it is very hard to retain without daily use.

</comment><comment><AUTHOR> You spend a non-trivial amount of time learning calculus, for example, but how do you maintain that competency?

<comment><USER11> maybe you could compile a list of problems and use anki to regularly train yourself?

There's also Schaums outline for Calculus

You can also try going through Art of Problem Solving Calculus (though it's much more difficult than the typical calculus text)

<comment><USER12> > maybe you could compile a list of problems and use anki to regularly train > yourself?

That seems to favour rote learning instead of actually developing or keeping core competences.

<comment><USER4> Given that the OP has already learned the material and is mainly interested in retention, that shouldn't be a problem.

In my experience, it's the mechanical stuff that you forget most quickly without use, as opposed to the conceptual stuff.

</comment></comment></comment></comment><comment><USER13> What about teaching?</comment>


Beware of the increasing dehumanisation of warfare - <AUTHOR> http://www.news.va/en/news/archb-tomasi-beware-of-the-increasing-dehumanisati

<comment><USER2> This reminds me of this photo [http://www.sfwpost.com/wp- content/uploads/2014/04/notabugspl...](http://www.sfwpost.com/wp- content/uploads/2014/04/notabugsplat-view-from-drone.jpg)

Where a village put up a big photo to show who the drone operators are actually killing.

Makes me sad every time I think about it.

</comment><comment><USER3> I'm not religious but agree 100%.

Automated killing machines are no good and humanity should not tolerate their use.</comment>


HN Meetup Munich: Date Poll - <AUTHOR> http://www.doodle.com/5durtx5tctbrrmxv

<comment><USER2> Voted! And sorry that i couldn't make it the last time, but the business trip and the cold I got from the air conditioning just plain killed me. ;)

</comment><comment><USER3> Any Dutch HN Meetups coming soon?</comment>


This Is Why The Internet (And Twitter) Wins  - <AUTHOR> http://www.techcrunch.com/2009/11/27/internet-twitter-tiger-woods/ <comment><USER2> Yesterday, I was thankful for _not_ paying any attention to the news while I cooked, served friends, and then relaxed. I turned the radio off and didn't check any websites. It was nice.

I don't mean to be callous about Mr. Woods' injury, but is this example really important? To put it a better way: it is of zero (maybe negative) value to me to hear "X happened to celebrity Y", or "celebrity Z died", for that matter, 45 minutes earlier than I would have otherwise. What _would_ be of value to me (and seriously, I would pay a lot for this) would be some kind of magic device that filtered out such items out of my media consumption and coughed up a precis on Saturday afternoon after I do the crossword.

<comment><USER3> That's a brilliant idea!

I agree I would pay for that!

Plus one to you!

</comment></comment><comment><USER4> How significant is a 45 minute delay to news that Tiger Woods was in a car crash?

Unless it's 30 minutes to the start of a tournament and betting hasn't closed yet the delay is insignificant.

<comment><USER5> I think you are missing the point of the post. Tiger Woods getting into a car accident in a non-tournament day is for the most part "useless" breaking news. Some might argue that it's still useless even in a tournament day.

However, what if you received flooding information 45 minutes before it happens in _your_ area. (I'm from the Philippines and we experienced heavy flooding recently and my house was nearly flooded b/c the "normal" news channels didn't know how bad it got before it got really bad). Of course, that's just one example.

Point is the power of Internet/twitter allows for breaking news that might be relevant to you and might give you enough time to avoid disaster, etc.

<comment><USER6> You're missing the point of the parent. If CNN breaks it on Twitter, they COULD break it on cable immediately. They've done it many times when real news occurred. They COULD break the flooding in your area (and probably would) as soon as they learned of it. I'm guessing the actual lead time, if it's something as important as a major flood, is a matter of at most a minute or two.

Which means that all Twitter in this case did is bombard people with stupidity 30 minutes before cable TV did the same. That's not a value add. If anything, it's an extra 30 minutes before this stupid fact pushes something useful out of my brain. Because I avoid Twitter, I knew who won the Battle of Trafalgar for an extra half hour before cable ruined that.

If you wanted to give an example of Twitter being useful, you'd give one where some normal person (not a cable news network) broke a story that got big before the MSM even found out.

<comment><USER5> That is my point. Someone else (a normal person) breaks a major flooding news in her area via Internet/twitter before CNN or other MSM. In our area at least, it would be at least 10 minutes or more before our MSM gets any news on this.

Another recent example, tied to the flooding, we had brownouts but the MSM was reporting incorrect brownout schedules. We went to twitter and got the tweets directly from the utility company. This example is not breaking news per se but it affects our daily lives.

Sorry if that wasn't clear before.

<comment><USER6> Right, but that sadly wasn't the point of the original post. Like most MG Seigler entries, it was logically incoherent and claimed that Twitter "wins" because CNN broke a story there before they broke it on their news. That's just dumb.

</comment></comment></comment></comment></comment><comment><USER7> You know why the Internet _really_ wins? Because I DON'T have to deal with stupid worthless news like this. Sorry, but unless I suddenly wake up as the CMO of a sponsor company, a trader with options in such a company, a competing golfer, or David Murdock (who owns the Sherwood golf club), this news is NOT essential to my life and I am glad my only connection to it is this stupid TechCruch article.

By allowing me to customize who and what I consume, the Internet allows me to get rid of all of the crap that CNN et al try to shovel at me. That's mainstream media's real problem -- they have completely alienated the most affluent and intelligent among their readers.

And in case you're wondering what I'd consider "vital news" -- how about the potential default of a sovereign nation (Dubai) which could trigger a CDS event which could re-trigger the global financial crisis?

<comment><AUTHOR> _And in case you're wondering what I'd consider "vital news" -- how about the potential default of a sovereign nation (Dubai) which could trigger a CDS event which could re-trigger the global financial crisis?_

I think the point is-- no matter what you consider vital news, the Internet is a better way to get that news faster and more accurately than traditional mainstream media.

</comment></comment><comment><USER8> It only wins if it supplies and continues to supply correct and timely information.

The internet is like the telephone game with millions of people and until something changes I trust my local news station many times more than the internet on breaking news stories.

<comment><USER9> And this is the part that society has to cope with - at least for the time being. As a population we are still used to trusting authoritative news sources, ones that are presumably bound by law, ethics, or policy to verify their information.

The result is that we _react_ rather quickly to news. This can (and has been) disastrous in internet communities where the news has turned out to be incorrect or fraudulent.

The consequence of the Twitter phenomenon is that collectively we all need a much better bullshit filter than the one we have right now.

</comment></comment><comment><USER10> > Early reports had him in serious condition (which remember, is better than > critical condition) after he apparently hit a fire hydrant and a tree while > leaving his home in his SUV.

I see this as a problem for twitter and the internet. They got a very important part of the new wrong.

Tiger Woods has already been released from the hospital and wasn't injured.

</comment><comment><USER11> Actually, here's what MG gets wrong.

The Twitter story was this: "BULLETIN — REPORT: FAMED GOLFER TIGER WOODS SERIOUSLY INJURED AFTER CRASH NEAR FLORIDA HOME."

And the CNN story (30-45 min later), was this: "(CNN) — Golfer Tiger Woods was injured in a car accident near his home, Florida officials say."

When all was said and done, which was more correct? The answer is CNN's. Woods was not "seriously injured," he was treated for facial lacerations and was, quote "fine," according to his agent (<http://twitter.com/darrenrovell1/status/6121135958>).

Twitter might win on speed, but CNN ultimately won (this time) on accuracy. They knew about the Woods incident as soon as everyone else did -- to think they don't watch Twitter like everyone else would be silly -- but they likely spent that extra half hour doing what no one on Twitter does: actual journalism (as in, checking sources).

(Note: I'm personally not a fan of process journalism, something that MG's boss avidly promotes... report first get the facts later is a sad future if it is indeed the future of how the news is disseminated, imho.)

</comment><comment><USER12> The question arises - do I really need to know about the details of celebrity golfer's lifestyles the very second something happens? If I read the news the following day, would that really make a significant difference to my life?

Maybe if I was an professional golfer it would.

But I generally agree with the idea that microblogging, blogs and other sorts of online media will eventually replace newspapers and even TV news reporting. Most newspapers already have a significant online presence and an entourage of bloggers.

</comment><comment><USER13> Win what? I don't even know who Tiger Wood is. And I certainly don't care if he had some accident or whatever so much, that I would want more than a short notice later that day (in fact, I simply don't need to know it at all).

If you think that "win" is delivering uninteresting tabloid allcaps "news" 60 minutes ahead of CNN, then, well... you have a problem.

<comment><USER4> While I generally agree with your point the way you present it smacks a little bit of <http://www.theonion.com/content/node/28694>

(At the very least if you have been to an airport in the last 5 years you will have seen an Accenture ad on a wall somewhere.)

<comment><USER13> While it is possible that I have heard the name before, I didn't remember it. I don't live in the USA and I'm not interested in golf, I might have heard the name few times in similar short notices, like I have heard about thousands of other people that I don't remember either. It's quite simple, really, I didn't know who TW is, and said that only to stress my point. It's surprising to me that it could be considered such a basic knowledge.

</comment></comment></comment><comment><USER14> I didn't hear it from the news on TV or Radio or the web or BBC News alerts, via email, IM or twitter.

Which is why the TV news wins over the internet - because I can turn it off. Misery from all over the world doesn't route around the TV's off button and get to me anyway - well, I guess it _does_ these days. :(

"The internet treats off buttons as damage and routes around them", to paraphrase a famous comment.</comment>


For Many Older Americans, an Entrepreneurial Path - <AUTHOR> http://www.nytimes.com/2014/02/08/your-money/for-many-older-americans-an-entrepreneurial-path.html?ref=business&_r=1

<comment><USER2> Here's what is worrying: _But it is not an easy road. Money is the biggest stumbling block. Most start-ups like Ms. Arnold's are underwritten with personal savings._

Also: _Many entrepreneurs do not pay themselves for a year or so to allow their businesses to gain traction._

Considering that most businesses fail within 5 years, I'm not enthusiastic about the older-than-55 crowd starting their own business. If you are fresh out of college you may be able to set up a company - you have time you put your finances back together, should the effort fail, but as a retiree you are looking at old-age poverty.

<comment><USER3> > should the effort fail, but as a retiree you are looking at old-age poverty

As mooreds mentioned in his comment[1], they might already be looking at old- age poverty. People are living longer, healthcare costs are rising, retirement funds are not keeping up, pensions and retirement benefits are being gutted by corporations and governments alike. So, speaking as someone who has a retired relative trying this idea, the downside might be minimal.

[1] [https://news.ycombinator.com/item?id=7208690](https://news.ycombinator.com/item?id=7208690)

</comment></comment><comment><USER4> I think this points to how much easier it has become to start a business nowadays (a normal business, not a tech business). I'm sure some of this is due to the marketing revolution that is the web, but I think it's also due to the move to services that has happened in the USA over the past 4 decades. (It's a lot easier to run a one person services business than a one person manufacturing business.)

Pair that with a) boomers needing more money because they have fewer pensions and are living longer, b) older folks have some accumulated capital (either financial, human or relationship) and c) the flexibility of a lifestyle business (even if you work longer hours, you can timeshift), and it doesn't surprise me that older folks are starting more businesses.

</comment><comment><USER5> Refreshing to read a NYT article that doesn't go out of its way to find the cloud in the silver lining.

</comment><comment> UK-AL I highly respect this is because of ageism in the jobs market, not because they want to.

</comment><comment><USER6> Medicare probably helps a bit too

</comment><comment><USER7> Read New York Times, I will not.</comment>


There are more open SEO roles in the Bay Area than qualified candidates - <AUTHOR> https://www.elischwartz.co/the-seo-job-outlook-everything-you-wanted-to-know/ <comment><USER2> I wonder how many of these SEO roles will be remote and/or outsourced</comment>


Is anyone else interested in fighting climate change as a software developer? - <AUTHOR> I created an app for sharing checklists and I then used this app to create a step-by-step guide for a process for finding work as a dev fighting climate change: https://voice.ch.ckl.st/how-i-am-fighting-climate-change-as-a-software-developer/ <comment><USER2> I've been writing code to feedback to electricity consumer's in Texas the degree to which renewables are present on the grid. Little things like setting the dishwasher to delay operation until peak wind % time, helps. In Texas, this time falls between 1-4AM, depending on the weather.

</comment><comment><USER3> IMHO as a software developer, one should always keep power consumption in mind. Optimizing whatever can be optimized.

</comment><comment> 70122-_6 Yep, and I have the perfect way to-do it. I just don`t have your tutanota.email ?</comment>


Ask HN: How do non-traditional CS people close their knowledge gaps? - <AUTHOR> I'm an EE. I work on embedded systems and FPGAs autodidactically and have also mostly worked alone. I've since worked in software companies and now am back to an embedded role. I've noticed that there are significant gaps in my understanding of CS concepts. I've no intuitive understanding, I never failed to finish my work but don't have mastery over the basics as I do over EE. I find that my code is not Idiomatic. I tried Leetcode but don't see any learning.
How did you get over this issue?  What are the topics that helped you most in your work?
Any Books? MOOCs? <comment><USER2> Don't worry, most if not all CS students and graduates have huge gaps in knowledge. Be it specific domains where CS is applied or concepts of CS itself, which they never heard about at university. There is simply too much to learn, to learn it in 5 years of university lectures. So you might not be as far behind as you think.

I've studied CS myself, but there is so much out there, if you are willing to get in touch with it, that I read something I don't understand or don't know what it is almost every day. Sometimes it is only new names for old things, but often it is something I simply never had to deal with. This might not happen at the job, but when I learn things in my free time.

I always recommend to look at and at least begin working on (doing the exercises of) SICP. That book has given me so much and led me to write so much more clean code and I am "not even" through the whole book. Well, it has 800 pages or so. So at least get to something like 40% and do the exercises, it will be enlightening. I have yet to read another book, that teaches its knowledge more to the point and gave me that many "Aha!" effects. Although my guess is, that the book will make you love Scheme, the knowledge within is applicable to almost whatever other language you use. I wish I had started to learn with SICP, instead of whatever was used in school and university instead. Reading and doing the exercises really helped me to get some concepts straight and to actually understand some things, that I did not have a good grasp of coming out university. Side fact: At least one of the authors taught EE.

Regarding to idiomatic code, you can specifically ask for idiomatic code or search for that, when you have a specific question. Over time you will absorb, how to do things best in this or that language. On top of that there is a certain amount of things that are objectively idiomatic, because they have the best performance available in that language for the task you want to get done _and_ have the most readable code as well. However, there are also parts, where idiomatic becomes a bit of an opinion. So worry about it, but don't worry too much about it.

<comment><AUTHOR> Thanks for the book suggestion, I'll order one and start on it. One of the lessons I learned this year is not to bite off more than I can chew, so next year I want to pick one book and work through it, I'll try SCIP and see how it goes.</comment></comment>


Scrum or Kanban? - <AUTHOR> Which does your team prefer? Personally, Scrum is far too regimented, compartmentalized and proceduralized. Kanban feels like it lets teams self-organize better and is thus truer to the original Agile Manifesto goals. <comment><USER2> Scrum is better for a more junior team that's less capable of self-organizing and needs more structure. It's also better for a green-field project where you're shipping new features on a regular basis and you want to set targets for completing them. It works well with larger chunks of work and is less flexible around things like prioritizing newly discovered production bugs.

Kanban requires a team of more senior and self-sufficient engineers who can manage their own work. It's also better for projects that are largely in a maintenance phase where you're constantly picking which fix to work on next out of your backlog.

Finally, scrum makes more sense for packaged software where you want to get into a regular rhythm of releases or where you have to bless defined releases (eg for App Store review). Kanban matches well with continuous delivery web environments where you never develop significant inventory of undelivered changes.

Edited to add: one more thing - scrum enforces certain types of communication about your project that with kanban you have to rely on company culture or external processes to manage. This tends to make scrum more effective the larger your engineering organization is.

</comment><comment><USER3> duplicate of [https://news.ycombinator.com/item?id=12808486](https://news.ycombinator.com/item?id=12808486)</comment>


Flickr Co-founders Join Mass Exodus From Yahoo - <AUTHOR> http://www.techcrunch.com/2008/06/17/flickr-co-founders-join-mass-exodus-from-yahoo/ <comment><USER2> I know it's been said to death but seriously tech crunch is like one of those really cheap and inaccurate "magazines" you see (only once staring at the packs of gum wears thin) as you wait in line at the grocery store. Grabbing on to any piece of "juicy" gossip and yielding approximately NO constructive criticism on a technical level let alone a cohesive/interesting economic perspective. Seriously the dude needs to move onto writing trash about politicians and celebrities. No kidding when I read his articles I imagine they are being narrated by that "perez hilton" guy. Same tone and substance.

<comment><USER3> So true :) When will TC come up with a cute "Brangelina"-like name for Fake & Butterfield - maybe Stewarina? As in "Stewarina are having twins!" "Stewarina are leaving Yahoo!-!"

<comment><USER4> I have no clue what you are talking about. I suppose that is a good thing :)

<comment><USER3> It sure is - you must be spending much less time in grocery store queues ;)

</comment></comment></comment></comment><comment><USER5> I have a friend that works as an engineer at Flickr, and he sent out reassurance:

"These two have been away from Flickr for the better part of a year, being on parental leave - now it's just more official. In short, we've been building cool stuff for the last year without these two officially around, the sky is still up where it should be. ;)"

<comment><USER6> Yup. (Disclosure: I work for Yahoo).

Consider this more like belated recognition for Kakul rather than Caterina + Stewart's departure. It's really, really sad that they're gone but the talent pool at Flickr is still pretty deep.

And in other Arrington scoops: Aaron Swartz is leaving Reddit.

</comment></comment><comment><USER7> Yahoo employs over 13,000 employees. 5 leaving in the span of a couple weeks is not a "mass exodus". And if these 2 are to "join", that means that he's calling 3 a mass exodus.

<comment><USER8> To be fair, the story does refer to "executives". I don't know how many of those Yahoo has, but it will be a small percentage.

</comment></comment><comment><USER9> I repeat, why is it news that two people who's company was acquired (for cash!) are not staying with the acquirer (pretty sure that's not even a word) for ever and ever? Because some site writes a sensationalist headline about it? Seriously folks, this is really not a big deal.

</comment><comment><USER10> What does this mean? 1) some of Yahoo's executives wished Microsoft had bought Yahoo or 2) usual brain drain or 3) unusual brain drain

<comment><USER9> This pair leaving likely means little more than a couple who don't need to work (for anyone) wanting to do something different. They are in a fortunate position where they can. Good for them. They should take a nice vacation.</comment></comment>