What we learned after a year on Kubernetes - <AUTHOR> https://about.gitlab.com/blog/2020/09/16/year-of-kubernetes/ <comment><USER2> > Increased billing from cross-AZ traffic

Yep, we (Basecamp) have been bit by this with EKS too. By default the ALB Ingress Controller will put all cluster nodes into the target group and a request can hit kube-proxy in any AZ before being directed to the right place, causing a lot of inter-AZ traffic churn. It's mildly annoying.

(with alb-ingress-controller you can change target-type to IP to have traffic go direct to pods that will actually back a given service instead of via kube- proxy on a random node, but the big fault there is that if alb-ingress- controller gets hung up on something and you deploy, you can send traffic to nowhere because the target group still has old IPs for pods that don't exist).

<comment><USER3> It's so worth it, though. You may not realize it, but your Healthchecks don't work unless you are using target-type IP. In the standard mode, AWS healthchecks each kubernetes node, but not your pods themselves. It will keep sending traffic to pods that are unhealthy (such as in state: terminating) if it already has connections open through kube-proxy to them, and stop sending traffic to healthy pods if the connection to them was through a node it considers "unhealthy". I wrote at tool[1] to handle this at Houseparty before alb-ingress-controller supported this mode, and it gained us a whole nine as measured by the ALB.

The same failure mode actually exists with instances, though they typically rotate much more slowly. Monitor your ALB-ingress-controller and treat it as critical infrastructure, because it is.

[1][https://github.com/GauntletWizard/targetgroupcontroller](https://github.com/GauntletWizard/targetgroupcontroller)

<comment><USER4> Can you expand on this: "AWS healthchecks each kubernetes node, but not your pods themselves".

Are you talking about a keepalive connection to an unhealthy pod which is reused for multiple requests? So the failure modes are, if I understand you correctly, a) the ALB keeps sending requests through an established keep-alive HTTP connection which terminates in an unhealthy pod, but which it sees as healthy because the node is healthy and can route traffic to another, healthy pod, and b) the health of an established HTTP keepalive connection is perceived to be that of the node rather than the destination pod, so nodes which become unhealthy can cause the ALB to unnecessarily terminate a keepalive connection.

We had to switch to using target-type=instance because of issues with pods not being deregistered. I'd prefer to use target-type IP but it seemed like preventing 500s on rollouts required a bit of testing and tuning with a very specific approach. e.g. introducing a longish delay on pod termination with a lifecycle hook and using the pod readiness gate support recently added to alb- ingress-controller.

<comment><USER3> You've got it exactly right. Your problem of pods not being deregistered is a real problem, but also with a quick fix: The default "Deregistration delay" for ALBs is 300 seconds but for kubernetes pods the TerminationGracePeriod defaults to 60 seconds. This means that your load balancer keeps trying that pod for 4 whole minutes after it's been hard-shutdown.

Here's the annotation that I used to fix that:

        alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=30,slow_start.duration_seconds=30

</comment></comment></comment></comment><comment><USER5> I learned not to use it and wait until the k8s fad goes away.

<comment><USER6> Why would you think it would go away?

Its literaly the best option currently for what it is doing.

It has an unprecedented support behind it as well.

Multiply Vendors support it through a certificate k8s managed service.

It solves really problems out ouf the box like load balancing, ingress, cert management, autoscaling, health checks, autorepair.

It allows for simple IaaC.

Is it young? Yes. Do we need more people with more expierence? yes.

Is this a problem? No.

<comment><USER7> I dunno, because of writeups like GitLabs? We're lucky GitLab is so transparent. It lets us see that their claims don't appear to always match the actual reality.

Here's an example.

Blog post says: _" After transitioning each service, we enjoyed many benefits of using Kubernetes in production, including much faster and safer deploys of the application, scaling, and more efficient resource allocation"_

Wrong. Actual analysis by the engineers of one of their migrated jobs (urgent- other sidekiq shard) says their cost went from $290/month when running in VMs to $700/month when running in Kubernetes. They tried to use auto-scaling but failed completely and ended up disabling it:

[https://gitlab.com/gitlab-com/gl- infra/delivery/-/issues/920...](https://gitlab.com/gitlab-com/gl- infra/delivery/-/issues/920#shard-performance)

Kubernetes looks like a massive LOSS in the case of this service. The perceived scaling benefits didn't materialise at all, and their costs more than doubled. They also spent a lot of engineering time optimising startup time of this job: 7 PRs and complex writeups/testing were required. Just so they could try to auto-scale a job to reduce the hit of the hugely multiplied base costs. If not for Kubernetes that eng time might have been spent adding features to the product instead.

 _It solves really problems out out the box like load balancing, ingress, cert management, autoscaling, health checks, autorepair_

Most of these problems are created by Kubernetes, like "ingress" which is a term only Kubernetes and low level networking uses, "cert management" which often isn't required in more traditional setups beyond provisioning web servers with the SSL files, "health checks" are a feature of any load balancer and the whole point of VMs is to avoid the need to repair hardware. Finally the demand auto-scaling as seen in this case is (a) possible without Kubernetes and (b) not actually working well anyway.

Frankly this set of bugs, writeups, PRs etc is quite scary. I worked with Borg at Google and saw how it was both a force multiplier but also a huge timesink in many ways. It made some complex things simple but also made some simple things absurdly complex. At one point I had a task to just serve a static website on Borg, it turned into a three month nightmare because the infrastructure was so fragile and Google-specific. A bunch of Linux VMs running Apache would have been far faster to set up and run.

<comment><USER6> Its really weird that your argument is based on just the cost.

I mean you mentioned yourself you worked at google right?

Perhaps you just haven't actually experienced the issues kubernetes is solving?

How often have you seen that certificates expired? I have seen that. Plenty of times. Its not an issue creating that lets encrypt cronjob, its still something you need to do right.

Security Updates? Have you seen how many companies run with old non updated VMs?

Disk full due to logs? Yes seeing this regularly.

Memory leak on a service and someone needs to restart it manually until someone else fixes the issue? Yes!

Requesting a VM, hardware, infrastructure, getting it and the whole lifecycle management of it in the backend? Its real.

Ansible Scripts, puppet or just bash scripts and a word document to tell you how this magic machine was set up? Yepp.

Kubernetes solves all those problems.

Your static website on borg, if it sill runs, has probably still a valid certificate, is running on a secure infrastructure, is equally configured on every instance and not weird on 1 of 6 servers and just runs.

A smart person taking responsibility for all of this, costs you much more then just a few hundred bucks a month. And you need that person. With Kubernetes, this person now can manage and operate much more servers under his/her fingertips better easier and more secure then if it would have been vms.

And in my personal experience: That shit runs more stable because that shit can restart and being recreated and it just solves a handfull of shitty memory or disk full issues.

<comment><USER7> I've been running my own Linux servers for about twenty years, so I'm not entirely unexperienced with these things.

Borg is/was great for running huge numbers of services at truly massive scale, when those services were developed _entirely_ in house and done in the exact way it wanted services to be done. It was a terrible cost the moment you wanted to run anything third party or which wasn't written in that exact Google way, and the costs were especially high if you didn't need to handle huge traffic or data sizes.

Borg and Kubernetes don't auto-magically do sysadmin work. A ton of people work in infrastructure at Google. Software updates still need to be applied etc. In Kubernetes that means rebuilding Docker images (which in reality doesn't happen as the tools for this are poor, so you just have a lot of downlevel Ubuntu images floating around).

And worse, the whole K8s/Docker paradigm is totally backwards incompatible. At Google this didn't matter because the software stack evolved in parallel with the cluster management. Programs there expected local disk to be volatile, expected to be killed at a moment's notice, expected datacenters to appear and disappear like moles. They were written that way from scratch. But that came with a terrible price: it was basically impossible to use ordinary open source apps. You could import libraries and (carefully!) incorporate them into Borg- ized projects, but that was about it.

When this tech was reimplemented and thrown over the wall to the community, the cultural expectations that came with it didn't come with it. So I've seen situations like, "whoops, we deleted our companies private key because it was stored to local disk and then the Docker container was shut down, help!". This was even reported as a bug in the software! No, the bug is that your computer is randomly deleting critical files for no good reason, and normal software does not expect that to happen. How about the way in a Dockerfile you have to write 'apt-get update && apt-get upgrade' if you want a secure OS image when the container is rebuilt? If you put the two commands on separate lines it will _appear_ to be working, right up until you start getting weird errors about missing files from Debian mirrors, because Docker assumes every single command you run is a pure function! And then we get to security.

Screwups seem to follow Kubernetes/Docker around like flies. The tech is complex and violates basic expectations programs have about how POSIX works. When it goes wrong, it leads to mistakes in production.

Now there have been a few trends over time:

1\. Hardware has got a lot more powerful. It has been outpacing growth in the internet and economy. Many more businesses fit in a smaller number of machines than when Borg was designed (an era where 4-core systems were considered high end).

2\. Cheap colo providers have been driving the cost of powerful VMs down to the ground.

3\. Linux has got easier to administer.

These days setting up a bunch of Linux VMs that self-upgrade, run some services via systemd etc isn't difficult, and properly tuned such a setup should be able to serve a monster amount of traffic (watch out though for Azure, which seems to overcommit capacity pretty drastically and their VMs have very unstable performance).

Most businesses that are deploying Kubernetes today quite simply do not need a million machines. Even companies that give away complex services like GitLab, as we can see from this thread, they don't _really_ need huge scalability. It's just nice to imagine that the business will experience explosive growth and that growth is now automated, but ironically, the effort to automate business infrastructure scaling takes away from the sort of efforts that actually grow the business.

As for the other benefits, you can write a systemd unit that is much simpler than Kubernetes configuration that will give you auto-restart, including if memory limits are hit, you can view the activity of a cluster easily using plain old SSH or something like Cockpit, it will handle log rotation for you out of the box, sandboxing likewise, and so on. And of course the unattended- upgrades package has existed for a while.

I agree you need someone to do admin work, whatever path you choose. Having used Kubernetes, and the system it is based on, and plain old Linux, my intuition is that the base cost of Kubernetes is too high for almost all its users. Too many ways to screw it up, too many ways for it to go wrong, too much time spent screwing around, and too expensive. If you become another Google or Facebook then sure, go for it. Otherwise, better avoided.

<comment><USER6> I'm not saying its not possible for small companies to run a handfull of VMs for themself.

But you reach quickly enough an size where a central managed kubernetes cluster is very versatile for your whole company.

1-2 People take care of the cluster, the other teams then use that kubernetes cluster for your build, hosting of staging envs etc.

And then you have a very small infra team which is much better able to provide those services to others internally much easier and safer.

</comment></comment></comment></comment></comment></comment><comment><USER8> > We found that with an application that increases its memory utilization over > time, low requests (which reserves memory for each pod) and a generous hard > limit on utilization was a recipe for node saturation and a high rate of > evictions. To adjust for this we eventually decided to use higher requests > and lower limit which took pressure off of the nodes and allowed pods to be > recycled without putting too much pressure on the node.

Kubernetes works at its most efficient when you scale by adding more Pods, rather than scaling within a Pod. Requests and limits should be equal to each other, and a HorizontalPodAutoscaler (or KEDA Scales object) should be used to add more Pods as utilization increases. Using Cluster Autoscaler then makes sure that your cluster has sufficient Nodes to schedule the Pods. Then you just need to set up alerting to tell you if the autoscaling reaches minimums (indicating that you should consider lowering the minimum, to improve efficiency) or maximums (indicating that you should consider increasing the maximums), and you can leave things well enough alone, unless the speed at which scaling occurs is insufficiently slow for your use case.

</comment><comment><USER9> It reads like a press release, no gotchas, no complaints...

[https://k8s.af/](https://k8s.af/)

<comment><USER10> There are some complains that we can share, but honestly they are not really that fun to read. We are still early in our transition and one major complaint I have is about helm and lack of flexibility of it and operator being very flexible but a huge time sink.

I think a lot of complaints we would usually have are reduced due to the fact that we use managed K8s with GKE.

All other complaints I can think of are related to our application and some architecture decisions we made early on, but that won't be interesting to anyone but people at GitLab.

<comment><USER9> > complains that we can share, but honestly they are not really that fun to > read.

"You are mistaken counselor" (c) The Descendants

</comment></comment></comment><comment><USER11> I do have only very minimal xp in k8s, as I have tried to use it only once (and never again). To me it seem like mOst of the comments are about the unusable and unmaintainable yaml configuration files. This seems like exactly the issue I had a few years ago, when I was cursing, because I could not find a spec or docs for the yaml stuff in all the pages and pages of seemingly important documentation and thus could not figure out how to transform a docker only deployment into k8s deployment. I also remember having to set up helm, according to whatever tutorial or official guide I was following. It was all a distant nightmare, that I do not wish to repeat.

<comment><USER12> There's definitely a need for an opinionated layer over the top, with much simpler configuration. Probably more than one, with different use cases. Nobody should have to wrangle that much yaml.

<comment><USER13> [https://convox.com](https://convox.com)

It's basically heroku on top of k8s and you bring your own cloud.

</comment></comment></comment><comment><USER14> We are using k8s as well for a project that we also provide on-prem install too. We decided to use kustomize for now vs doing helm for now. Curious, have someone had experience with both to compare. What are the benefits of using helm?

<comment><USER15> `helm` is an absolute garbage. I'll reserve judgment about `kustomize` until I have more practical experience, but so far it looks to me that it's going to be another YAML disaster.

The problem with k8s ecosystem is that developers conflated k8s using yaml for a semi-human-readable serialization of k8s resources, as reason to employ YAML to absolutely everything.

What I'd like is to generate serialized yaml files ... you know ... with code. Using some typed programming language, to be able to build any abstractions required for the job, and get some checks, compiler errors, and even ability to have asserts, unit-tests and so on. Instead, I have to dig YAML with a pickaxe in the YAML-mine without any technology to assist me.

<comment><USER16> Could you explain why helm is garbage? I think it suits its purpose rather well without being too complex. You can essentially "plug-in" different types of resources rather easily. Especially in v3 now that you don't need to install Tiller and can avoid setting those cluster permission requirements.

Have you tried some Kubernetes api libraries? You can generate and configure resources with [python kubernetes-client]([https://github.com/kubernetes- client/python](https://github.com/kubernetes-client/python)) without much trouble. Personally I prefer editing them as JSON instead of python objects, but it isn't too bad.

<comment><USER17> > Could you explain why helm is garbage?

Not the OP, but..

1\. YAML string templating makes it very easy to get indentation and/or quotation wrong, and the error messages can easily end up pretty far from the actual errors. Structured data should be generated with structured templating.

2\. "Values" aren't typechecked or cleaned.

3\. Easy to end up in a state where a failed deploy leaves you with a mess to clean up by hand.

4\. No good way to preview what a deploy will change.

5\. Weird interactions when resources are edited manually (especially back in Helm 2, but still a thing).

6\. No good way to migrate objects into a Helm chart without deleting and recreating them.

7\. Tons of repetitive boilerplate in each chart to customize basic settings (like replica counts).

It's a typical Go solution, in all the wrong ways.

<comment><USER18> It's not going to solve all your problems but dhall can fix your first few gripes. I've been using it for several months and it's an excellent way to write configuration imo.

<comment><USER17> Yeah, I have used Nix to generate them in the past, which worked pretty great too. But Helm does, admittedly, solve a real problem: garbage collecting old resources when they're deleted from the repo. I just wish we could have something much simpler that only did that...

<comment><USER19> `kubectl apply --prune` should nominally do this. Irritatingly (I acknowledge I'm almost as responsible as anyone else for doing something about this), it's had this disclaimer on it for quite some time now:

> Alpha Disclaimer: the --prune functionality is not yet complete. Do not use > unless you are aware of what the current state is. See > ⟨[https://issues.k8s.io/34274⟩](https://issues.k8s.io/34274⟩).

I haven't used it in anger, so I can't add any disclaimer or otherwise of my own.

 _kpt_ is the recent Google-ordained (AFAICT) solution to this problem, but is ot yet at v1.

You could also resolve this yourself by either:

* versioning with labels and deleting all resources with labels indicating older versions

* using helm _only_ in the last mile, for pruning

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> My question. Why not convert the rails code/APIs to blob/FaaS and skip all the DevOps complexity?

Are containers providing enough long-term cost-effectiveness to a complete cloud-native application architecture?

Or is it just mapping VMs to something "like" VMs so your topology remains mostly the same?

<comment><USER21> Serverless has its own complexity. Deploying a smaller, lightweight purpose built image, with no orchestration lock-in is the main benefit.

<comment><USER20> Those are fair infrastructure reasons, but not necessarily cost-effective. There are always competing perspectives. What does the business want. What do the developers want. What does the infrastructure team want. When power is centralized in one area, decisions can be made that overlook those competing purposes.

I see containers being adopted without openly discussing the options and long- term consequences.

</comment></comment></comment><comment><USER22> Was anyone else surprised by the timeline of this post? I was under the impression that Gitlab had been fully on K8S for years now, given the hard push of their K8S offering 18 months ago.

<comment><USER23> It has been possible to run GitLab the application on K8s for some time now for the majority of self-managed deployments. There were some limitations however, for running GitLab on K8s at the very large scale of GitLab.com that we have been working through in the last year.

(disclaimer: blog post author)

<comment><USER22> > August 2019 when we migrated the GitLab Container Registry to Kubernetes, > the first service to move. Though this was a critical and high traffic > service, it was a good choice for the first migration because it is a > stateless application with only a few external dependencies

I was under the impression that porting a large stateless service was something that wasn't too bad to do with K8S? What limitations prevented this from going ahead earlier?

<comment><USER23> Yeah, this is why we decided to migrate it first about a year ago, and we probably could have done it sooner than that. Other services were not yet ready to migrate until recently due to some blocking issues which was a factor in the decision.

disclaimer: blog post author

</comment></comment></comment></comment><comment><USER6> I'm not sure if i get it but normally internal traffic across AZs should be included?

It was also not clear to me after reading their issue page.</comment>


How Fastmail Connect Their Datacentres Together - <AUTHOR> https://blog.fastmail.com/2016/12/19/secure-datacentre-interlinks/ <comment><USER2> Interesting use of openvpn. Eventually your sysadmin might have his day, as managing the overhead (both in engineers and CPU resources) for site-to-site networking becomes a pain without hardware designed to do it.

My problem with setups like this is they are usually fairly brittle in the long run, and the person who created and maintains the links gets bored but nobody has the desires or skills to take over the project. It breaks once and suddenly the hardware cost just isn't that big of a deal.

</comment><comment><USER3> This solution provides no forward secrecy and doesn't use TLS, as stated.

[https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage](https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage)

Search the page for '\--secret'.

Frankly I would have expected better from FastMail.

(And I'll resist the urge to hate on OpenVPN, although I could do so all night)

</comment><comment><USER4> I seriously can't wait for Wireguard to reach a certain level of maturity. It looked awesome and very simple to use.

The best thing about OpenVPN is ease of use but for these kinds of cases, isn't performance an issue?</comment>


All 500 of the world's top 500 supercomputers are running Linux - <AUTHOR> http://www.zdnet.com/article/linux-totally-dominates-supercomputers/ <comment><USER2> A win for open source. I was just thinking the other day that it's a shame that Linus is not a billionaire. Seems he's doing just fine however:

"Finnish-American software engineer and hacker Linus Torvalds has as estimated net worth of $150 million and an estimated annual salary of $10 million. He earned his net worth as the principal force behind the development of the Linux kernel."

Source: [https://www.therichest.com/celebnetworth/celebrity- business/...](https://www.therichest.com/celebnetworth/celebrity- business/tech-millionaire/linus-torvalds-net-worth/)

<comment><USER3> I don't think its so much a shame that Linus isn't a billionaire than it is we have billionaires at all.

There is limited rational or ethical reason for one human being to accumulate so much centralized wealth. There is being rewarded for hard work / skill / innovation and there is exploiting people to become rich, and the line between those is very hard to distinguish when talking about individuals making more than 9 figures off of the endeavor.

It is also worth considering you don't make billions operating a nonprofit or making a valuable software product (or any product, in general). You make billions founding or investing early into a multinational corporation that amasses more wealth than many nations.

And that is where I think the conversation breaks down - people take personal offense to conversations about wealth and equality but we aren't talking about people wielding the influence of themselves _as_ individuals, we are talking about people who control more assets than some nations, that can potentially have financial influence on the world stage greater than the combined influence of millions of fellow humans.

<comment><USER4> > There is limited rational or ethical reason for one human being to > accumulate so much centralized wealth

First: yes, there _are_ ethical reasons to accumulate so many _resources_ : because some problems are incredibly hard to solve and need a lot of resources to even start to solve.

Second, it is not on the people who have earned something to prove why they should be "permitted" to keep it; it's on the people who believe it should be taken away from them to justify _that_. Saying "it's a shame that there are billionaires" is a complaint that some people are wildly successful and shouldn't be, or that they shouldn't be permitted to keep what they earn. There is justifiable resentment and offense to a conversation that starts with a thinly veiled demand for justification, rather than a presentation of justification. The burden of proof is approached the wrong way, and nobody seems to discuss _that_.

(I'm not interested in quibbles over the word "earn" here.)

It's not a shame that there exist billionaires. It's a shame that there exist people with nothing. The former doesn't need fixing; the latter absolutely does.

<comment><USER5> > _Second, it is not on the people who have earned something to prove why they > should be "permitted" to keep it; it's on the people who believe it should > be taken away from them to justify that._

This is a great example of status quo bias.

You believe that the onus of arguing ethics is on the people who want to change the status quo when really, we should start by thinking about what society ought to look like from first principles. And then you absolutely do have to give arguments for why you want _any_ inequality to exist in the first place, let alone billionaires.[0]

> _It 's not a shame that there exist billionaires. It's a shame that there > exist people with nothing. The former doesn't need fixing; the latter > absolutely does._

While I'd agree that the latter problem is more urgent, the former can't be dismissed so easily, either. For one, the existence of billionaires (and upper-range millionaires, really) is pretty antithetical to one of the foundational principles of democracy, which is "one person one vote". It gets even worse when you consider the possibility and practice of inheritance.

And besides, some people might argue that if we didn't have the former problem, it might be easier to tackle the latter -- because when there is less inequality, there tends to be less focus on wasteful jockeying for social status overall, and then people might be more inclined to help those that are below themselves on the social ladder.

[0] There are good arguments in favor of some inequality, but the _degree_ of it is pretty important.

<comment><USER6> Equality of resources and status, or equality of opportunity?

Some people are worse than others. Some people are less useful to society than others, though that doesn't make them worse. Some people, given large amounts of money, can do amazing things for the world. On a whim, I can think of Elon Musk. If he weren't allowed to have his fortune from Paypal because inequality is unethical... who would have gotten that money? A government? What would they have done with that money? And why is government/the public always the best decider of how wealth is spent?

Your argument may be reasonable in degrees: Who needs $500 billion dollars? Should there be a legal requirement to invest that money in the public at a certain point, if it isn't taxed directly? I don't know. But I believe inequality is ethical and necessary.

<comment><USER5> _Equality of resources and status, or equality of opportunity?_

Both, for two main groups of reason.

The first one is that when the available positions in society are highly unequal, then equality of opportunity may simply not be enough for a fair outcome, as who ends up in the comfortable and high-status position ends up largely being down to mere luck.

The second one is that (at least as long as families and inheritance are a thing) the two are linked anyway. Inequality of resources and status is the leading cause of inequality of opportunity, after all. You basically have to either go Brave New World or increase equality of resources in generation N in order to make serious progress towards equality of opportunity in generation N+1.

And we agree that a lot of it is about a matter of degree -- I've said as much in my original post, after all :)

</comment></comment></comment></comment></comment></comment><comment><USER7> Note that BlueGene/Q is commonly reported to run "Linux" but 99% of the nodes are running CNK instead. [https://computing.llnl.gov/tutorials/bgq/#Software](https://computing.llnl.gov/tutorials/bgq/#Software)

<comment><USER8> > This is an open source, Linux-like, light-weight, 64-bit operating system > that runs on the compute nodes

Seems to be a Linux kernel fork with performance optimizations.

EDIT: I stand corrected [https://en.wikipedia.org/wiki/CNK_operating_system](https://en.wikipedia.org/wiki/CNK_operating_system)

</comment></comment><comment><USER9>
      With an anti-science regime in charge of the government, America will only continue to see its technological lead decline.

Brutal to be confronted with such a cutting opinion in an otherwise congenial article and to have no thoughts of disagreement.

<comment><USER10> America is a bet that chaos has some advantages over good governance.

<comment><USER11> Is good governance obtainable in a country of such scale and idological diversity?

</comment></comment></comment><comment><USER12> I remember Linus mentioning [0] about the major benefits of tuning Linux for super computers that came to be really useful for when trying to get performance out of a mobile phones decades later.

[0]: [https://m.youtube.com/watch?v=MShbP3OpASA&t=39m](https://m.youtube.com/watch?v=MShbP3OpASA&t=39m)

</comment><comment><USER13> Serious question (not in IT), what else can they run?

<comment><USER14> They could likely run any of the BSD variants. Some form of commercial RTOS might also be viable. Even Minix might be an option (hire away some of those Intel folks!) though I'm not sure of its hardware support.

That said, why bother? Linux is customizable enough to strip out almost everything extraneous for nodes and has the largest pool of experienced developers, and it's unlikely that any of the other options would have any performance advantage - and less likely that they could _keep_ any performance advantage once it was identified.

<comment><USER15> Why on earth would you want a RTOS for HPC?

RT is all about privileging latency over throughput, whereas in HPC throughput is everything.

<comment><USER16> Perhaps because a lot of tasks are not embarassingly parallel, which means moving data around, which means network interconnects, which means latency matters. Every millisecond delay in forwarding a packet is thousands of instructions stalled.

I'm not saying this is actually a sensible tradeoff for any extant HPC architecture, but it _could_ be. You certainly want the 'OS' that runs the pipeline on your CPU to be real time!

<comment><USER15> It is a misconception that RT means "lowest possible latency".

It actually means "latency with a known upper bound". The point of it is to be able to give engineering guarantees - the actuators on the rocket engine vanes _will_ move within N milliseconds of the input voltage from the gyroscope module changing; the robot arm motor _will_ be de-energised within N milliseconds of the laser perimeter sensor activating.

Bounding the worst case can often mean making trade-offs in the best case, and that doesn't make sense in HPC because the hard upper bound just isn't necessary in this environment.

</comment></comment></comment></comment></comment><comment><USER17> Where does that put Minix if the supercomputers have the ME chip? Based on the findings this past month, Minix is on the most computers.

</comment><comment><USER18> The real question is are they all runix minix(Intel ME)

<comment><USER19> Most of them run Xeon, so yes, but about 10% use either Opteron, SPARC64, Power, and Sunway.

</comment></comment><comment><USER20> How does one go about learning more on this topic -- being a Linux admin for a supercomputer?

<comment><USER21> I don't think there's a degree for supercomputer admin, it's mostly on the job training. And to be fair, a lot of it is similar to generic Linux sysadmin stuff. Especially so nowadays with the focus on devops/infrastructure-as- code/etc., or generally the idea of how to administer many nodes in a scalable fashion.

If you want to play around, go to [http://openhpc.community/](http://openhpc.community/) , fire up a few virtual machines, follow the installation instructions, and start playing around.

</comment></comment><comment><USER22> These linux OS are customized, compiled and not out of the box, correct?

<comment><USER23> At [https://www.top500.org/statistics/overtime/](https://www.top500.org/statistics/overtime/) click "Operating System" and "Performance Share" to see:

      - 300 PFlops: Linux
      - 260 PFlops: other
      - 150 PFlops: Cray Linux
      - 130 PFlops: CentOS
      - 1 PFlop: bullx supercomputer suite (supports RHEL/SUSE/etc)

</comment></comment><comment><USER24> I vaguely remember that Microsoft offered version of Windows Server for HPC environments at one point.

Are the reasons for not using Windows purely financial (in a cluster with thousands of nodes the licensing costs can become substantial) or are there technical reasons as well?

<comment><USER25> I doubt licensing costs have much to do with it, it seems like flexibility would be the driving factor. Supercomputing centers, vendors, institutions which use supercomputers, etc. all frequently modify and contribute modifications to the Linux kernel and associated userland. You have to imagine they are making changes which Microsoft could not afford to care much about, nor turn around in competitive time.

<comment><USER24> That makes sense. Thank you!

</comment></comment></comment><comment><USER26> Ronald Minnich (if you don't know him see [1]) comments :

[1] [https://share- ng.sandia.gov/news/resources/news_releases/san...](https://share- ng.sandia.gov/news/resources/news_releases/sandia-computer-scientists- successfully-boot-one-million-linux-kernels-as-virtual-machines/)

\</comment><comment></comment><comment>-----

"The semiannual TOP500 Supercomputer List was released yesterday. It also shows that China now claims 202 systems within the TOP500, while the United States claims 143 systems."

My fellow Americans:

Did you ever want to directly experience the transition to being a Tier 2 country? This is what it's like. It starts with the cancellation of the collider in Texas in 1994 (see Tyson's comments on that one). It continues as steady decline through the next two decades, under both Republicans and Democrats, and now it is reaching its culmination in the Trump administration with the replacement of scientists by political hacks. It's not fun. It's quite sad to watch, that's for sure.

<comment><USER27> Trump is not the problem. It is all the non-elected dark figures behind it all.

<comment><USER28> From what I can see anti-science, anti-technology attitude is endemic to modern English speaking western culture.

Elected and un-elected officials are just a symptom of the problem. They'd all flip-flop if they thought the political and media climate was shifting.

</comment></comment></comment><comment><USER29> _With an anti-science regime in charge of the government, America will only continue to see its technological lead decline._

This is idiotic. The new admin could well commission new supercomputers for espionage or other military purposes.

The constant politicization of every single aspect of life is going too far.

</comment><comment><USER30> "For example, the new Linux 4.14 enables supercomputers to use Heterogeneous Memory Management (HMM). This enables GPUs and CPUs to access a process's shared address space. Exactly 102 of the Top500 are using now using GPU accelerator/co-processor technology. All of these will perform better, thanks to HMM": it sounds like some of the 500 supercomputers use Linux 4.14. I guess none of them uses Linux 4.14 yet.

</comment><comment><USER31> I think it's hilarious that so many run CentOS instead of RHEL. Redhat has managed to make their product worse than free with their licensing bullshit.

<comment><USER15> It makes sense though, because a supercomputer installation is exactly the kind of place you'd expect to have its own experts that support and customise the OS.

And there really aren't that many supercomputer installations, so any that RHEL does have are probably more for the prestige than the revenue.</comment></comment>


It's time to split Linux in two - <AUTHOR> http://www.infoworld.com/d/data-center/its-time-split-linux-in-two-249704

<comment><USER2> We all know how well it went for Microsoft when they split Windows desktop from Windows RT while pretending they were one and the same.

Also removing systemd would only create more headaches for developers. For the packages that are common to desktop and server they would have to support two init systems.

My conclusion is this guy has no clue what he's talking about.

</comment><comment><USER3> Managing "hundreds and hundreds" of dependencies is a simple task with simple build scripts. You have a much bigger problem if you're doing things like this manually. If you want a fine-tuned server version of your distro, chances are it's out there already anyway.

</comment><comment><USER4> The answer is simple -- Linux can achieve this through configuration, not by splitting it in two. There's no reason for major surgery when simple configuration choices can achieve the same thing.

</comment><comment><USER5> isn't android the 'desktop' linux and ubuntu server/centos the 'server' linux? almost?</comment>


2048 was too easy : welcome 4096 - <AUTHOR> http://www.nemoidstudio.com/4096

<comment><USER2> How do you like dem apples? [http://imgur.com/fq1lQXZ](http://imgur.com/fq1lQXZ)

I beat it on my first try, using my strategy (and far too much practice) from 2048. However, when I play 2048, I have the board completely full at some point in maybe ~50% of games won, so I'd guess this version is winnable roughly half as often, since an extra spot is taken up for the whole second half of the game by the 2048 tile.

<comment><AUTHOR> Well done! Should we make a 8192 version ? :)

<comment><USER3> Here you go, one without a limit: [http://alxndr.github.io/2-n/](http://alxndr.github.io/2-n/)

</comment></comment></comment><comment><USER4> favicon still says "2048" ;-)

</comment><comment><USER5> Down already?

<comment><AUTHOR> Working here. What error do you have ?

<comment><USER5>
        Unable to connect

        Firefox can't establish a connection to the server at www.nemoidstudio.com:8080.

<comment><AUTHOR> Hmmm, remove the :8080 it should be on the standard port 80

<comment><USER5> Ok, looks good :)</comment></comment></comment></comment></comment>


GDPR and automated email marketing - <AUTHOR> https://www.gdprhq.io/post/the-new-european-general-data-protection-regulations-and-automated-mail-marketing <comment><USER2> As somebody who has hated spam for years, I can only wish that I were in the EU.

There is a whole swathe of companies that is somewhere between casual and negligent with email addresses, and it would be my distinct pleasure to have a stick like GDPR to beat them with.

<comment><USER3> Spammers don't tend to operate from first-world jurisdictions. (When they do, CAN-SPAM is decent about requiring working unsubscribe buttons). Spam is not a problem you can solve with regulation.

<comment><USER4> Sure, I don't expect to stop receiving invitations to enlarge my genitals in my spam folder because of GDPR, but I'll be happy enough if it discourages dodgy online shops and growth-hacky startups from automatically signing me up to their mailing list because I made a one-off transaction and "consented" to receive their special offers for all eternity on page 25 of their terms and conditions.

</comment></comment></comment><comment><USER5> I'm really curious whether or not this will have an effect and to what extent.

I have been using the last 6 months documenting all of our company's processes that handle customer interaction and data (which is basically all our processes), created flowcharts of how data moves between us, third part providers and customers as well as creating a document for each of these flowcharts that pinpoint exactly how we are complying with GDPR for every sub- process.

If for nothing else, we now have a total overview of what we do and how we do it - in an easily shareable collection of visualisations and documentational material.

<comment><USER6> Same here. My company is a lot smaller (at least from the sound of it) but GDPR made me review my processes as well.

There were still a few processes that I could simplify and automate.

<comment><USER7> Would you be interested in a compliant user data management service? What specifically would you look for in one?

</comment></comment></comment><comment><USER8> > Part of this opt-in verification process must include clear documented proof > that the person opted in with a full understanding of what they were signing > up to.

Does anyone have any idea how to actually do that? How do I _prove_ that a given user actively checked a box?

<comment><USER9> Would something like a verification email asking them to double verify answer that? They click the box, then they have to open an email and click a link also verifying it?

<comment><USER10> But is that sufficient under GDPR? Although a double opt-in has generally been considered good practice for a long time, it only demonstrates that a recipient has agreed to receive mail for _some_ purpose, not for any _specific_ purpose.

Even if you've been building up your mailing list for years, following generally accepted good practices, and only signing up genuinely interested recipients, it seems you could now to be in a position where either:

(a) when you signed people up, you provided sufficient information about what you would be sending to them _and_ you can still produce evidence of that today;

(b) you need to contact everyone on your list to obtain explicit, specific consent for whatever you actually send to your list; or

(c) you have to remove anyone who isn't covered by (a) or (b) above (or delete your whole list).

As with so much about the GDPR, what will be accepted as reasonable evidence of informed consent for earlier subscribers to a mailing list is ambiguous, and the consequences of either doing too much or not doing enough are undesirable.

</comment></comment></comment><comment><USER11> As someone who's tinkering on an app to send marketing email, I constantly struggle with the field. On one hand, I really think it helps small/niche businesses survive, which I think is critically important nowadays. On the other, nobody thinks their shit stinks, and man does it stink.

I'm curious--what emails do you actually appreciate getting? Would you subscribe to marketing email with restraints? (e.g. only email me when items in my size are on sale). If you could change how email marketing works, how would you do it?

<comment><USER12> I only appreciate emails with special offers for complementaries of products and services I'm already using.

And whenever I receive marketing emails that I never subscribed for, I flag it directly as spam, although sometimes I ask the sender, just for fun, the source of my address. They rarely reply :-)

I keep my Inbox clean because otherwise I'm missing important messages. If it's not important, it doesn't belong in my Inbox. If I don't know the sender and it tries to sell me something, it's spam. If I don't remember subscribing, it's spam.

</comment></comment><comment><USER13> Unclear regulation? I am encountering this over and over again. Lets clear the unclearity...

If you have my data, you will handle them in same manner as you would handle yours. You are not selling yours to get higher prices when buying something online? You are not selling your email account to spammers to get a lot of worthless emails to your email account each day? ... Now you wont do it withy my data either. It is so simple, you don't need any clarification. No special law or directive, no studying of GDPR... it just works. Oh you want me to receive unsolicited emails for your profit? You want me to get tracked? ... I will personally take care you will get a punishment and/or sue you personally.

What is so complicated here? Act in best interest of you customers, regarding the personal data, and you are safe, over whole EU. I don't understand what is the problem unless you are NOT ACTING IN THEIR BEST INTEREST, then it becomes vague (you need a way to circumvent GDPR, but you can't as it is not an IRS list but a conceptual law). Anyone having a problem with GDPR already knows the answer that solves the "problem". But wants to continue his habits.

Just state your problem and I will answer to you with advice where you wont get punished for breaking GDPR, just ask. But you wont, right? You know the answer, but you need a way to avoid it. Wont work.

<comment><USER14> I fully agree with you, but there are many technical services/platforms that assume things that are not compatible with that thinking. Those will have to change, but they are still not up to speed.

Let me preface my question with the statement that I mostly love the GDPR, and I think it greatly improves privacy and digital rights and I will exercise some of those rights come May 25:th against companies that I feel have needlessly collected data on me.

That said I (as a data controller) think that in many cases that the guidelines are very weak or undefined on subjects like logs or backups. I (as a private individual) think that any deletion request should automatically apply to logs and backups, but also I (as a data controller and...) as a operator of a service see it as a problem to have backups be mutable and have large swaths of data need to be deleted from backups and logs.

Is there any way to reconcile these ideas?

<comment><USER13> Sorry for late reply. For old data, the easyest way is to burn the tapes and make new backups. Now about new backups, here it becomes nasty as typically they aren't organized granulary enough (but you also need this for exporting the data on user request, so you just need to do it). Instead of backuping the whole databases, backup each users data separately, maybe database partitioning, table inheritance (postgres) or something else, hard to be specific here. Once you did that, backup the data by encrypting them with random key (long enough, we are using 32 bytes of random garbage) for each user while storing those keys on simply modifiable storage, cloud, whatever in triplets. Once the user requests data deletition, just destroy the key. We did it this way and it is great solution (and we DID burn the tapes literaly, luckly we have business data separated physically from everything else from the start).

Logs are destroyed each week and the customer will be notified. Also we anonymize ips and reverse lookups by hashing them, while we still can identify the same visitor.

I hope I was helpful :)

<comment><USER15> If You're really destroying your logs each week you're not meeting a lot of regulatory requirements, such as PCI if you accept credit cards.

Most security-oriented regulations, and indeed so-called "best practice", requires keeping logs for security auditing purposes for at least a year if not longer. They're often the only tool you have to detect when and how a breach began.

</comment></comment></comment></comment><comment><USER16> I hope this stops Sparkpost. Most of spam I receive is traceable back to them. Sure I forward it to abuse@ and few times I received open ticket receipt. They followed up few times then completely ignored my further request for info or status updates. These days I don't even receive new ticket or any kind of confirmation. I started forwarding those to FBI and FCC, but I'm sure they too busy.

It seems that yahoo really loves Sparkpost spam that goes straight to my mailbox even when sender domain is no-existent, not to mention any DKIM or SPF records; gmail is much better at catching those.

This is my experience for the last 2 years.

</comment><comment><USER9> As someone who doesn't deal with Europe much... CASL here in Canada seems to have similar rules. Would following CASL automatically mean it follows GDPR?

<comment><USER17> The UK has had data protection laws for years, people aren't scared of GDPR because it finally provides laws, they're scared because they actually look enforceable.

<comment><USER18> I think people are scared because legislation is unclear.

</comment></comment></comment><comment><USER19> This is a frustrating article.

The regulation in GDPR is not new! It's a refinement of long existing law (in England this is the DPA, and PECR).

If it's illegal under GDPR it was probably already illegal under PECR.

All this stuff about "ZOMG we need informed consent before we send email"? You already need that.

</comment><comment><USER20> This is going to be fun when election times come in europe. Here we get a lot of unsolicited email from candidate MPs , and i m certain most of them bought/found the addresses from dubious/illegal sources.

<comment><USER21> The Canadian anti-spam legislation goes so far as to have a specific carve-out for political emails that solicit money [1]. I joined each of the big 3 parties during their leadership campaigns and they all have a practice of ignoring unsubscribe requests, and passing your email address around internally or signing you up to new lists. It's pretty gross.

[1] [https://crtc.gc.ca/eng/com500/faq500.htm](https://crtc.gc.ca/eng/com500/faq500.htm)

</comment></comment><comment><USER22> Isn't the simple way to get around GDPR is to send the email marketing from a foreign email company (in a non GDPR jusisdiction) asking if you are interested in being referred to a type of product or service?

</comment><comment><USER16> VERIFIED OPT-IN parts opens up beautiful opportunity to destroy your competitor for $5.

1\. Open DigitalOcean hosting for $5. With prepaid card they will let you do it, however port 25 will be blocked.

2\. You don't need port 25 anyways. Download few lists of emails from online search and setup php_curl every 30 seconds to your competitor's landing page subscription ajax call.

3\. Wait few months for them being slammed with $4MM fines as there will be unable to prove how they got that traffic in the first place :)</comment>


"Taxes that Work" proposal: eliminate income tax for  - <AUTHOR> http://www.samefacts.com/archives/taxation_/2007/09/wrongheaded_tax_policy.php

<comment><USER2>
      The best way to counteract inequality is by raising more
      revenue and then spending it on things that the middle and
      lower classes need, like national health insurance.

It ain't the job of the gubmint to "counteract inequality" in the economic sense (it's an impossible task in a free society, anyway)

      [The plan] also wrecks much of the political support for
      tax expenditures...

When you think of lowered tax rates, tax credits and deductions as "tax expenditures", you've got it completely backwards. That money isn't the gubmint's money. If my salary were to be lowered by my employer, it's not an expenditure on my part like it is if I reduce my expendable income by investing in a CD.

      And perhaps most important, his plan gives the American
      welfare state a much sturdier base for future expansion
      (esp. since the VAT is less transparent than other forms
      of taxation), especially since there's room here to add
      some form of carbon tax, which is necessary for dealing
      with global warming and would provide even greater
      revenue going forward.

Oh yes, that's brilliant.

1\. An expansion of the welfare state is a _horrifically_ bad idea. A smaller welfare state is an _advantage_ , in all kinds of ways: economically, demographically, and with respect to liberty and (little-r) republicanism.

2\. Making the functions of government _less_ transparent is a goal?

3\. Carbon taxes are not necessary to "deal with global warming". Assuming that whatever part of global warming we actually have any ability to affect is significant enough to matter (evidence suggests otherwise, and in any case is far from settled), capitalist market responses would be _vastly_ better at "dealing" with it than anything the gubmint could do with carbon tax revenues (which may not actually increase overall government revenues).

4\. I _hate_ the phrase "going forward".

<comment><USER3> Agree in general, with one note:

> capitalist market responses would be vastly better at "dealing" with it than > anything the gubmint could do with carbon tax revenues

Carbon taxes have two sides, the costs and the revenues. The costs _do_ work with and contribute to the capitalist market responses, by making more polluting products more expensive, relative to the less polluting.

On the revenues side, I agree again, but why not use that money for general expenditures? Why not tax the bad things (pollution & other negative externalities) more, and the good things less (sales, income, investment)?

</comment></comment><comment><USER4> however for someone that is "rich" or earns more than $250k a year, if you start taxing them say 50%, they can afford 1 person to think for 1 full year at $50k to figure out how to avoid $100k in taxes. keep that in mind, when you tax the people who can afford to protect their money, THEY WILL.

for example, Steve jobs earns $1 a year from apple as CEO, do you think he cares what income tax costs are?

instead he relies on stock sales, or capitol gains. so steve is wealthy enough that he can target his sale of stock to +/- a decade, in the mean time he can borrow money at below inflation rates against his assets, so he does not even have to sell a stock to reap its benefits. so someone like Steve will decide when the political climate is right to buy and sell stock(avoiding taxes, through credit). (high interest rates, make this unlikely, and puts pressure on stocks to beat interest rates) so in high interest rate periods, paying taxes might be cheap, compared to loosing gains. saving pulls money out of the economy and strengthens the currency, as the economy is less leveraged. strong currencies make it easier to buy forgien goods and resources, as people want to trade now on your currency, as they expect its value to increase in the short term.(this is one possible way to bring gas prices down through raising interest rates)

so, lets say that old Steve sells some stock, regularly, that means that the rest of us have the chance to buy it at a good price and we might see gains at 20-50% in a year, this is good for everyone. it increases trading volume and makes the market more fluid (and yes volitile). however volitile markets are how you make money. it makes much better use of your fiscal efforts, than labor based activities. So, its good for Steve to sell more often with fewer capitol gains taxes. adding taxes to capitol gains will probably reduce trader volume, and depress stock prices, effecting the leveraging power of companies, and resulting in a lower GDP.

so other devices that wealthy use include dividends which right now are taxed at 15%. (politicians want to raise this too) low dividends also keep people seeking investments that share their earnings, so wealthy companies "share" their wealth with investors inorder to keep the avalible levarage of their company up. again low taxes are better.

i think the single best way i have seen to generate revenue for the government was based on a proposal from warren buffet, whe he suggested a transaction tax of 0.3% on all transactons, or account transfers. he suggested that the tax could easily be added to all software, and if you buy a stock, you pay 0.3% on the cost of the purchase, if you sell it, you pay 0.3% on its sale. so as money moves around the economy, it generates revenue for the government. as a wealthy person, you can not escape it, you would need to move your money around to get better than 12% anual returns, so you would always pay the tax, and 0.3% on 1,000,000, is only $3k which might hardly be worth the effort of someone else to find a work around. the idea is that a single dollar might be party to several of these transactions a month.

taxing, or punishing the rich, does not make for a better economy.

</comment><comment><AUTHOR> hmm. HN needs to set the "maxlength" attribute on that "title" input field as well as the "size" attribute.</comment>


Receivd - beautiful, fast filesharing for everyone - <AUTHOR> http://receivd.com

<comment><USER4> Feedback:

> Receivd - beautiful, fast filesharing for everyone

Everyone who has a Mac? Or everyone? I only see screenshots of a Mac app. There are sketches of other apps, but I'm guessing those do not exist yet.

"Real time file sharing" as a tagline seems kind of meaningless. Two reasons I don't like it: a) real-time seems overly technical, and b) real-time is implied. No one expects that there would be a delay (beyond the amount of time it takes to send the file.)

Finally, this seems like it occupies a weird grey area between dropbox and email. All the examples you give are things that I can easily do with email. If email proves to be too much of a hassle, I have to convince people to use something else, and teach them how it works. Between this and dropbox, why do I choose this?

Minor point: To look at all 4 screenshots in large-o-vision, I have to first click one, then close it, click the next, and so on. Arrows within the lightbox would be appreciated.

Also, I realize this sounds kind of harsh, but I don't mean it too—I'm just putting out there the first things that come to mind. I think there actually _is_ room between dropbox and email, but you're going to have to work a little bit on positioning your product properly to define that space.

</comment><comment><USER5> _We're opening up Receivd to users slowly. If you'd like to get VIP access, please tell people about Receivd on Twitter and Facebook using the buttons below._

I got this message when signing up for access and I have to ask: Why would I tell people on Twitter and Facebook about this if I haven't gotten a chance to use it yet?

<comment><USER6> Do what I did: post to Facebook, and put in your message that you're testing it out and this is just a test post, not an endorsement. Then wait for the email (mine took ~30 mins).

Do this at late at night (like I did) and delete your FB post after you get the email (like I did).

Alternately, you can go the whole +hn route described below. It took longer than the FB route, but at least you're not participating in marketing of a sight-unseen web service.

<comment><USER7> I think the point is that asking a user to do this (even though they can get around it) is offensive. "Hey spam your friends and we'll give you free stuff" is starting to get too common.

</comment></comment></comment><comment><USER8> Sorry to be Debbie Downer, and a little bit off topic: when will the trend of dropping vowels from names end? Googling for "Receivd" autocorrects, and I doubt they're going to be expedient in fixing that. Domain names are now responsible for devolving English...

I need someone to hate for this. Was Flickr first?

<comment><USER9> I fear that in the future these mispellings will wind up being recognized as correct spelling of the words.

<comment><USER10> That's what Mark Twain (purportedly) said: <http://grammar.ccc.commnet.edu/grammar/twain.htm>

<comment><USER5> "Every quote, given enough time, will be attributed to Mark Twain" - <USER2> Twain

</comment></comment></comment></comment><comment><USER11> Uhm. So what is _the_ difference here? The ability to easily share with pre- configured lists of people? The rest of what's outlined on a linked page does not strike me as something that wasn't successfully done before.

Genuine question. Consider it a lead for _how to change the landing page for more clarity_ discussion.

<comment><AUTHOR> Thanks for your comment - much appreciated.

We're optimizing the landing page as we go along. The difference is the ease- of-use, and the ability for non-technical folks to pick it up really quickly and get started with sharing files with the people they care about in a few minutes. Not sure if you noticed, but the current landing page is not designed with hackers in mind (we understand we're posting on HN, but technical folks are not our primary audience).

</comment></comment><comment><USER12> I'm sure that it exists, but could someone describe for me the main point of differentiation between Receivd and a shared folder on Dropbox?

That is, what should be compelling me to download _another_ application to do that which I can (seemingly) accomplish with one already installed?

I'm not trying to be simply critical, but rather just pointing out that, at least from the landing page, there isn't anything necessarily motivating me to sign up. (That said, as it's on HN, I'd love to sign up, if only to provide feedback!)

<comment><AUTHOR> We built Receivd to not just share files with our friends and family, but to have them share back photos and other stuff with us. Our parents and our friends', for example, cannot understand Dropbox's shared folders, but they're using Receivd to share photos they took with their kids. That's a big win for us.

We love Dropbox and use it everyday, but we feel that Receivd does sharing the way that most consumers understand. For example, people move stuff out of shared folders all the time, which deletes it for everybody else - just a consequence of the tech. We do know that Dropbox is killing it, and as loyal customers we're very happy for them and wish them well.

<comment><USER13> I don't understand how people that don't understand Dropbox shared folders would understand Receivd, a more complex app?

However, some people I share folders with do use it as a transfer mechanism and remove the files from it.

<comment><USER14> Because it has a more explicit interface (vs. Dropbox's "implicit" interface.)

That is, for what I can see with Receivd the user explicitly "opens" an app (as they're used to doing for almost anything else on their computer), which in turn clearly states "drag and drop files here to share."

With Dropbox the user needs to understand the concept of a shared folder that "magically" syncs with other computers. For many users this is an alien concept and may confuse them. I know of many people who have a Dropbox account, yet use the service exclusively through the web interface, simply because typing a URL, logging in, browsing for a file, clicking and downloading it is a concept they "get."

There's no such thing as "one size fits all" in this world---specially when it comes to user interface design.

Edit: I should clarify I do not work for Receivd, nor have any connection with the team, so I don't speak for them. This is just my assessment as a UI designer.

<comment><USER15> Minusfive's point of view tracks with years of working with non-technical customers, _many_ of whom have a mental model suggesting that files "live in" the application as opposed to being in a folder somewhere.

I don't know how far dragging and dropping files is going to get them (many of my users could not locate a file if it bit them on the nose), but hey, it might work.

</comment></comment></comment></comment></comment><comment><USER16> If you guys sign up with +hn within your email, we'll make sure to give you access right away.

<comment><USER17> what do you mean, firstname.lastname@example.com +hn?

<comment><USER18> Gmail allows tagging at the end of the address like email@example.com. This is what he is referring to.

<comment><USER19> This is in the email RFC 5322, it's not GMail.

<comment><USER18> What do you mean? Just because it's in the standards doesn't mean they are treated as the same address, does it?

I don't see anywhere that email@example.com and firstname.lastname@example.com must send to the same mailbox. I only see that an address must "[contain] a locally interpreted string followed by the at-sign character."

Of course, this means that you can use - or ~ or # in your address but they don't have to be treated (and probably aren't) treated as the same address by most email systems.

</comment></comment></comment></comment></comment><comment><USER20> Been using the Alpha for the past few weeks, and the product is super slick and polished for the first release.

I'm not sure why people are talking about the landing page, domain name, and other irrelevant stuff. Dropbox is really good at syncing files across systems, but not that great for sharing files quickly and effortlessly. Receivd in my opinion is trying to solve this problem.

</comment><comment><USER21> The Mac app feels unfinished — connections, files, etc. aren't draggable, reshare doesn't work (can't even select the files), but all of that is stuff you'll obviously get around to.

I'm using it now, and it's _so close_ to being exactly what I need, with two necessary pivots that seem almost predestined to happen from your current position. The current basis of it being an inbox for files is already on the right track, and something I haven't seen attempted before, but it needs a little more verisimilitude to email:

1) The requirement to invite / induce signup / confirm "connections" needs to be ruthlessly optimized out — I should be able to just send a file to any email easily, forming a 'weak' connection. Maybe they have to do a 3 second signup to download the file, but it needs to be all in one motion. This obviously brings up a ton of issues like how to display the now 4 types of connections and unsolicited files, but it's essential to you getting traction.

2) 'sent' files need to behave like a outbox — remember that one of the biggest uses of email attachments is just sending stuff to yourself. Really you should just implement that straight up, it's the biggest use I'd have, and one I think you would find yourself as soon as you get the iOS client going. The dropbox iOS app's UI is terrible for it (pick one photo at a time and wait), plus really the filesystem metaphor is all wrong anyway! I actually _want_ the transactional inbox feel of email here. If you don't explicitly support this I'm just going to end up using it with a computers account and an iPhone account.

<comment><AUTHOR> Hmm, all of those things you described up top are working well for us - <USER3> you tell us what version of OSX you're running on? Also what version of Receivd are you using?

1\. You _can_ send a file to any email right now - once you invite an email, they'll be greyed out in the sidebar, then drag any files to them as you would with a confirmed user. They don't even have to signup to download the file. You can even add those folks to a list.

2\. Interesting point. We'll think more about this.

Thanks a lot for your comments. Can we reach you directly somehow to figure out why you're seeing these issues?

<comment><USER21> OS 10.6.6 (10J567), Receivrd 0.7.5 (16). I expect to be able to drag a connection to a list, and drag a file from inbox/connection/list activity/received lists and the "sharing activity" lists to all other targets (including outside the app). The Re-Share button has not lit up, which did not surprise me given that I can't really get a file into a selection state. If all this is working for you, I can just try updating / rebooting.

I can send a file to any email now, but there's a lot of sharp edges. You make me click through a modal sheet wizard to intentionally setup a contact first, there should really be an email textbox / fb friend selector / etc. in your app. The worst part about the current setup is that it forces you to first send a no-context invite email minutes before sending a meaningful "x has shared y with you" email. The first kind of email really needs to only ever get sent when a user is explicitly inviting other people to share with them, which is probably not going to be the default path.

My email is in the 'about' area of my profile if you want to reach me.

</comment></comment></comment><comment><USER22> Hmmm mac only for now. I'd be able to try this out with Linux and Windows versions. I have a mac, but it usually sends files to non-macs.

What's the advantage over Dropbox? Does your service go through a central server, or is it more p2p? Or both? Are my files encrypted on your server? Storage or bandwidth caps?

</comment><comment><USER23> > Unlike email, all photos and videos sent via Receivd are delivered at the > same high quality resolution that your camera supports. No compromises.

What? Whose email client does this?

<comment><USER24> That probably refers to the iPhone's options on sending a scaled version of the images it takes.

<comment><USER25> Windows Live Mail (the desktop client formerly known as Outlook Express) also offers resizing options when sending.

</comment></comment></comment><comment><USER26> Looks a bit different from Sendoid (<http://sendoid.com/>), but still, are they in competition?

</comment><comment><USER27> Do you also own recievd.com? I'm sure a lot of people forget "I before E, except after C."

<comment><USER16> We do, actually - it should be forwarding now.

</comment></comment><comment><AUTHOR> We don't have screenshots up, but there's also a really hot iOS app in the works.

</comment><comment><USER28> I've been hurting since drop.io shut. This sounds like a good replacement.

EDIT: Functionality is quite different than drop.io and less anonymous in terms of sharing (though powerful for sharing with small groups or individuals). It would be great to get a private URL for each item shared, a la google docs.

<comment><USER16> Thanks!

You can actually get a shareable URL for each item you share. Just right click on a file and select Copy Public Link.

<comment><USER29> Is there an option to share sensitive files w/out going through your servers? Just wondering....

</comment></comment></comment><comment><USER30> After 5 seconds on the site, I'll share my first impressions before they fade, because I think they could be useful.

In my nominal browser view, all I see above the fold is a logo the size of an elephant, a blank field labeled "Submit", and what looks like screencaps from iTunes.

<comment><USER31> That's not iTunes. It looks like their OSX client.

<comment><USER32> I think he knows that. He just saw "what looks like screencaps from iTunes" for the first 5 seconds, and afterward, he realized his mistake.

</comment></comment></comment><comment><USER33> I need to be able to upload files from the *nix/OSX command line. Is that supported?

</comment><comment><USER34> You need to label your input field better....all it is is an input field with a button that says submit. It's not very clear to people that they need to enter an email address

</comment><comment><USER35> Have signed up! Waiting for the invite. My bro is tech averse and hence Dropbox averse. I always find it difficult to share docs with him. Hope receivd solves it for me!

<comment><AUTHOR> Invited! Email us at email@example.com if you need anything at all :)

<comment><USER35> Thanks. I would have missed it. Google didn't consider it important enough for it to be in my priority inbox

<comment><AUTHOR> Thanks for bringing that to our attention, gotta fix that real quick :)

</comment></comment></comment></comment><comment><USER36> So.. dropbox + sync that always unions files from local source + interface for access control?

Seems like the functionality could be copied pretty quickly...

</comment><comment><AUTHOR> Thanks for your interest, excellent comments and support! We're still giving quick access to HN users - append +hn when you signup (ex: email@example.com)

</comment><comment><AUTHOR> Also, we're based out of Mountain View (no offices yet). Let us know if you'd like to sync up :)

<comment><USER37> Hit me up. Email in profile.

</comment></comment><comment><USER38> It's a awesome app. This is the way I want, for sharing files. BTW, is it created by Macruby?

<comment><USER16> We didn't end up writing the Mac client in MacRuby. We considered it, but to be on the safe side we decided to just go plain Objective-C.

</comment></comment><comment><USER39> Nice, but why would I leave Dropbox? Also, I like www.letscrateit.com for ad hoc sharing.

</comment><comment><USER24> Can't wait to get an invite to start playing with this!

</comment><comment><USER40> Will there be Windows support?</comment>


My business experience at 26: 3 successes and 26 fails - <AUTHOR> At 17, I sold my first business for $5,000. Since 2017, two of my other projects have earned $300,000 in net profit. These are POGUMAX - software for interior projection mapping, and HYLS - an online course of yoga, meditation and healthy habits for donation (45k participants).
I've written an article and described the launch history of three successful projects, difficulties, promotion cases, insights, 26 unsuccessful projects.
Original is here: https://vc.ru/life/134977 I can translate it into English and publish, if it interesting. Vote <comment><USER2> Thanks for sharing your interesting stories</comment>


What the Apple Watch Means for the Age of Notifications - <AUTHOR> https://medium.com/backchannel/what-the-apple-watch-means-for-the-age-of-notifications-1f0409a0c8e0

<comment><USER3> I agree that not all notifications are equal and it sucks that our options for notifications from an app are normally binary (on/off). Some apps allow you to pick/choose the types of notifications you get but the problem still remains that my phone treats them all the same regardless of if it's a CNN announcement of a shooting or magic piano has a new song (I've since disabled MP's notifications completely).

I say this all with no solution to put forth. I'm betting users would not be happy having to assign a priority level to each app and furthermore not all notifications from an app are of equal importance (ie. IFTTT SMS vs iMessage from my Mom).

One thing, unrelated, that I hate about notifications (that IMHo greatly diminishes my use of them in iOS) is the fact that in the notification center they are grouped by app which is one of the stupidest things I've seen in iOS. I want my notifications weaved together and sorted by time not by app THEN by time. Under iOS current implementation you NC could look like this:

      * GMAIL

        * 1 min ago - Blah blah blah

        * 5 days ago - Yada yada yada

        * 10 days ago - Bippity boppity boo

      * MESSAGES

        * 2 min ago - URGENT MESSAGE

        * 3 min ago - ANOTHER URGENT MESSAGE

That's just stupid, it should be:

      * GMAIL - 1 min ago - Blah blah blah

      * MESSAGES - 2 min ago - URGENT MESSAGE

      * MESSAGES - 3 min ago - ANOTHER URGENT MESSAGE

      * GMAIL - 5 days ago - Yada yada yada

      * GMAIL - 10 days ago - Bippity boppity boo

This is just a simple example but throw a few more apps in there and it's not unheard of to have something that happened 1-2 minutes ago buried a few swipes down to get past other app's notifications.

<comment><USER4> Android lets you manage what apps you consider to have "priority notifications" and filter accordingly, but managing notifications like that is a pretty awful user experience.

I don't have any alternative suggestions either. There's a mismatch in motivations - the app makers want you to open their notification, so you can trust them to prioritise notifications.

<comment><USER5> Android Apps already have the ability to prioritize notifications, and at least a couple get it right. Very few apps go out of their way to give themself a high priority they don't need (I've not encountered any as far as I know)

</comment></comment></comment><comment><USER6> I typically don't find notifications to be a problem. I don't use them for trivial items (such as an app like MLB like the authors complaint). You have to curate your user experience some on your _personal_ device. It isn't personal unless you do so. I looked at the list of apps that I allow to notify me and its about a 50/50 mix of on vs off. However, among those I have turned on only about 3 or 4 of them notify me very often. I think that the solution for this problem already exists.

<comment><USER7> I thought apple's whole value proposition was that they curated that for you. Very good user experience by default, that you tweak to suit your needs.

<comment><USER8> They do. Notifications are opt-in on an app-by-app basis. Each app has to ask you for permission to send notifications, and you can change your preferences in the Settings app later.

</comment></comment></comment><comment><USER9> The thing that is missing from notifications is receiver context. The time that the notification is pushed doesn't mean that is the time that I want to receive them. When I'm driving I don't need a notification to tell me the latest sports score, in fact it would be a distraction. But I do want the notification telling me that the meeting I'm driving to has been rescheduled or that there is an accident up ahead and if I leave at the next junction I won't be stuck in traffic for an extra 30 minutes. Bill payments are set-up on a Sunday evening so no point reminding me on a Tuesday morning, as I'll just forget by Sunday. Facebook you can prod me at lunch-time or in the evening, providing I'm at home and not engrossed in a film. Breaking News maybe I want to know, but if it's really just an update and you don't know anything more than you knew 2 hours ago then don't waste my time.

The challenge is that the notification publisher doesn't know my schedule or context so will never know whether it's a good time for me to receive the notification. The device I'm using might have some explicit or implicit insight into my schedule/context but it doesn't have any detail on what the notification is about and whether I would want to receive it now.

Putting the fine grained control for type of notification and time of day/day of week in the hands of the user might help, but realistically it's only a subset of users who will invest the energy in configuring this. A learning notification filter is the only way I can see that it would be simple enough for most users to benefit.

</comment><comment><USER10>
        We are about to experience a hyperdrive
        acceleration of notifications, propelling
        us to a crisis situation

The only crisis that's looming is a storm of 1 star ratings and uninstalls for app makers that send too many push notifications. If anything, the Apple Watch and other wearables will force app makers to think more critically about the shit they send out right now.

<comment><USER11> I get that on my Pebble. The app store is terrible for this. Every time an app is updated, I see nothing pushed to my phone but surely my watch will buzz. Stupid.

</comment></comment><comment><USER12> The problem that the author complains of--high noise to signal on notifications--can exist on the iPhone of a careless user.

The solution doesn't need to be as complicated as an AI for sorting your notifications, just a set of filters. Notifications for messages from my family and major stock moves, badges or silence for all else.

</comment><comment><USER13> 1\. Notifications on a watch do not need to be more disturbing. To the contrary, it is quite possible that the vibration on the wrist is easier to ignore than the phone, that it is easier to just check the display very shortly, instead of fumbling with the phone in your pocket. That was one of the reports from the android devices used.

2\. There is no age of notifications. There is one system where notifications seems to go a bit rampant, and even on iOS user have the option to just not opt into them (like several others mentioned here already).

3\. The great AI solution actually colides with the idea of seeing them as a feed, because there, the solution is rather to let the feed be filtered by humans. I don't let newspapers ping me, I follow friends who ping me with articles from newspapers. And there always where notification we let humans decide that we want to see them, they are called texts/sms/chat messages. A technical solution might be nice, but is probably not necessary - <USER2> since it is so much easier to just not accept anything unimportant to send you a notification.

4\. A really great notification concept is notify-osd on the ubuntu desktop. That concept behind the software defines notification as awareness things that users might want to see or might want to ignore, so they are in black boxes that can be transparent when the mouse hovers them, they are only shown shortly, they don't make a sound, they are not animated, they never expect any input. It's the perfect system for stuff that is not really important. If one really has to let every crappy app ping oneself, it should be with a notification system that adopts this principles.

</comment><comment><USER14> I feel like I have already been through this process without an Apple Watch, because I always keep my smartphone in my pocket with my wallet, so whenever it vibrates with a notification I can feel it buzzing on my skin.

I solved it by just going to Notification Center and turning off notifications for everything except Calendar alerts and SMS messages.

It doesn't always work. I sometimes get spam texts. But it's better than it was.

<comment><USER15> I have this exact same setup - except a) add a Pebble - iPhone isn't on me at all times (e.g. shower). b) I add alerts for work mail server, but none for personal mail c) VIPs bypass all my filters

All that said, Pebble still buzzes too much - e.g. Ascension game invites/turns don't show on phone, but cause my watch to buzz.

Maybe... the Apple Watch will resolve that issue.

</comment></comment><comment><USER16> The suggestion from the author is for development of AI to sift through notifications.

I am all for AI... yet, How about you don't install that app? How about you _just let your phone ring?_

No really... you don't have to check it. It's sad to see three people on a table, each on their own phone.

Cherish the moment with those humans. Forget about your phone.

<comment><USER11> >It's sad to see three people on a table, each on their own phone.

"Hey Becky, did you see that Facebook post the other day?"

"No"

"Go check it out! You too Ashley!"

 _all three of them go and look at it so they can discuss it_

Is that not enjoying moments with other humans? Who are you to say how people should enjoy themselves?

<comment><USER17> >"Hey Becky, did you see that Facebook post the other day?" >"No"

"Oh well check it out when we finish with our meeting. Oh right did I tell you about what happened the other day ..."

<comment><USER11> >Oh right did I tell you about what happened the other day...

"No, what?"

"Oh I went to the mall. Bought some new jeans."

"Oh that's cool..."

"Yeah..."

"..."

"..."

"So anyway, about that thing on Facebook..."

 _they all check the thing on Facebook_

 _cue 90 minutes of laughing and joking about the thing they now all share in common and may even still make reference to ten years down the road_

<comment><USER17> Well if the shit you read on Facebook is the most exciting thing to talk about .. then the issue aren't notifications and social media.

<comment><USER11> Like I said: Who are you to say how people should enjoy themselves?

</comment></comment></comment></comment></comment></comment><comment><USER18> The "notification" problem boils down to this: many want your attention; you only want to give attention to what is relevant to you, at a particular time. What is signal some times is noise others. It's not just NOISE Versus SIGNAL. It's NOISE Versus SIGNAL for USER X at TIME Y.

configuring preferences for this problem takes time and dedicated thought (when they're even available to be customized!), resources that users have been reluctant to spend on personal technology. The example in the article was really just due to wrongly selecting a "send me messages" option; THAT bad user decision produced enough frustration to write the OP. This will compound and be very frustrating eventually for 'bad operators'; if you don't have spam properly filtered out of this feed or that one, it compounds at the phone- notification level. In viewing technology, I don't really like saying "well, they're just a bad user", it smacks of Jobs' comment on the iphone 4: "you're just holding it wrong".

This problem is not distinct to an individual mobile user overloaded with notifications. We have this problem in every service we interact with that provides a regular stream of information. In email, this started as SPAM and NOTSPAM as the only measure. If it's from a human, it's relevant. There's one step further with this in Gmail with "important" notification (based on who sent it and what you open). Mobile notifications make a good example of the problem, because they are the final layer of "what is relevant" filtering between so many info services and a single user. Even with binary on/off for many of those streams, the problem remains, and compounds with expectedly careless users.

Automating relevancy measurement will be one of the more "creepy" technologies, I believe, because of the nature of human memory. We will find very quickly that a computer can be "better" (er... faster and more user- aware) at deciding what we want to look at than we ourselves are.

We will also find that the computer can, through micro-decisions, drive a particular thought or behavior pattern. By tending to ignore "health-related" notifications because you are personally uncomfortable with them and view them less than other notifications (as an example), it could exacerbate that problem by further obfuscating such notifications based on that preference. OR, the opposite: have a predisposition to ignore money notifications because of income apprehension? Computer knows and compensates, notifying for these personal weak points more urgently. These examples are strained, but they express the idea.

Compare to facebook's microdecisions to show you more or less of your ex- lover's posts soon after breaking up. These could vastly affect a person's behavior, and we trust in the "know by wire" system to do what is "best" for us.

We've outsourced the relevancy for news events to journalists, editors, and individual publications for hundreds of years ("I'm a WSJ man", for example). the ability for technology to challenge that relationship is just beginning to come into its own. The need for a journalist at the bottom level of "event or not" decision making is still necessary (this is changing too, but still necessary), but publications are now more like buckets of particular editorial opinion and style that are churned together on other platforms than they are decisions of "this stream or that" that users make.

What I see is a need that every technology user shares on many platforms, and a moral hazard of sorts for whoever develops a solution for that need. We already trust in facebook and google to show us what is most "relevant" to us, to not manipulate us for profit. As we rely more and more on automated measures of relevancy, we put more and more trust into their making such measures benevolently.

</comment><comment><USER19> _So what's the solution? We need a great artificial intelligence effort to comb through our information, assess the urgency and relevance, and use a deep knowledge of who we are and what we think is important to deliver the right notifications at the right time._

I believe the answer to this question is far, far simpler. The problem started in the second paragraph.

 _... MLB.com At Bat apparently deemed this important enough to broadcast to hundreds of thousand of_ users who had earlier clicked _, with hardly a second thought, on a dialogue box asking if they wanted to receive notifications from Major League Baseball._

MLB.com decided nothing. They asked if they could send messages. The author decided that anything MLB.com At Bat wanted to send him was important enough to interrupt, as he puts it, _" enduring a meeting, playing basketball, presenting to a book club, daydreaming, watching a movie, enjoying a family meal, painting their masterpiece, proposing marriage, interviewing a job candidate, having sex, ..."._

"MLB.com At Bat would like to send you notifications (Yes / No)" is pretty clear and to the point. They want to message you. If you want to be messaged, click yes. Sure, app operators can be reckless and bad citizens, but the ultimate control is in your hands. That last push notification giving you permission-granting-remorse? It's trivial to disable notifications on a per- app basis, or even restrict them to something non-intrusive.

There is no notification alert crisis. Apple and the other smart watch makers are not releasing wrist-mounted spam machines. You will be able to choose who you trust to interrupt you, and you will have decently granular controls over how these notifications present themselves. It is up to you to decide who to trust with the ability to grab your attention at any moment. If you place that trust poorly you have the ability to revoke that grant at any time.

Personally I feel like I have a lot of control over how distracting my mobile device is. I only allow alerts from Hangouts, iMessage, Slack, AlienBlue, and whatever turn-based game I'm playing at the moment. These are apps where I'm interacting with another human, where the notification will probably get me to interact with the individual. I restrict or deny notification permissions for every other app. Rarely do I press "Yes" to the request for notifications. When I do, at the first breach of trust that app gets permissions removed. Uber is a good example, I granted notifications thinking it would be nice to not watch the map when waiting for a car. Uber took the opportunity to send me a push advertisement for some weekend rate thing, so now Uber can never message me for anything. Map watching isn't very onerous, still better than waiting for a cab to show up, so I'll deal.

<comment><USER20> Not to nitpick the author and his anecdote, but the MLB At Bat app actually affords the user considerable control over what you do and don't receive in a push notification. There's the binary yes/no for news, and then you can add your favorite teams (wherein you'll only receive notifications that have to do with those teams). You can further enable/disable things like game start, lead change, score change, game end, highlights, and game news. You can do all of this sorted by team. TBH it's probably the best experience I have with an app that gives me notifications because I get _exactly_ what I want and not a push more.

</comment></comment><comment><USER21> It will be hard to avoid notifications on your watch.

<comment><USER22> But it will take much less attention to process them.</comment></comment>