Torus: A distributed storage system by CoreOS - <AUTHOR> https://coreos.com/blog/torus-distributed-storage-by-coreos.html <comment><USER2> It's hard to take this seriously: storage is an excruciatingly hard problem, yet this cheerful description of a nascent and aspirational effort seems blissfully unaware of how difficult it is to even just reliably get bits to and from stable storage, let alone string that into a distributed system that must make CAP tradeoffs. There is not so much of a whisper as to what the data path actually looks like other than "the design includes the ability to support [...] Reed-Solomon error correction in the near future" \-- and the fact that such an empty system hails itself as pioneering an unsolved problem in storage is galling in its ignorance of prior work (much of it open source).

Take it from someone who has been involved in both highly durable local filesystems[1] and highly available object storage systems[2][3]: this is such a hard, nasty problem with so many dark, hidden and dire failure modes, that it takes _years_ of running _in production_ to get these systems to the level of reliability and operability that the data path demands. Given that (according to the repo, though not the breathless blog entry) its creators "do not recommend its use in production", Torus is -- in the famous words of Wolfgang Pauli -- not even wrong.

[1] [http://dtrace.org/blogs/bmc/2008/11/10/fishworks-now-it- can-...](http://dtrace.org/blogs/bmc/2008/11/10/fishworks-now-it-can-be-told/)

[2] [http://dtrace.org/blogs/bmc/2013/06/25/manta-from- revelation...](http://dtrace.org/blogs/bmc/2013/06/25/manta-from-revelation- to-product/)

[3] [http://dtrace.org/blogs/dap/2013/07/03/fault-tolerance-in- ma...](http://dtrace.org/blogs/dap/2013/07/03/fault-tolerance-in-manta/)

<comment><USER3> I totally agree with you. I also liked how they said the motivation is to make Google infrastructure for everyone else. How did Google do this? They basically imitated and improved on clustered filesystems developed in HPC. There were a lot of lessons to learn for emerging cloud market in all tooling done in HPC. Some was FOSS.

Whereas, many companies seem to be doing the opposite in their work on these cloud filesystems. They don't build on proven, already-OSS components that have been battle-tested _for a long time_. They lack features and wisdom from prior deployments. They duplicate effort. They are also prone to using popular components, languages, whatever with implicitly-higher risk due to fact they were never built for fault-tolerant systems. Actually, many of them often assume something will watch and help them out in time of failure.

If it's high-assurance or fault-tolerance, my manta is "tried and true beats novel and new." Just repurpose what's known to work while improving its capabilities and code quality. Knocks out risks you know about plus others you don't since they were never documents. Has that been your experience, too? Should be a maxim in IT given how often problem plays out.

<comment><USER2> Yes, that's absolutely been my experience -- and even then, when it comes to the data path, you will likely find new failure modes in "tried and true" as you push it harder and longer and with the bar being set at absolute perfection. I have learned this painful lesson twice: first, with Fishworks at Sun when we turned ZFS into a storage appliance -- and we learned the painful difference between something that seems to work all of the time and something that _actually_ works all of the time. (2009 was a really tough year.[1]) And ZFS was fundamentally sound (and certainly had been running for years in production) before we pushed it into the broad enterprise storage substrate: the bugs that we found weren't ones of durability, but rather of deeply pathological performance. (As I was fond of saying at the time, we never lost anyone's data -- but we has some data take some very, very long vacations.) I shudder to think about building a data path on much less proven components than ZFS circa 2008, let alone building a data path seemingly in total ignorance of the mechanics -- let alone challenges -- of writing to persistent storage.

The second time I learned the painful lessons of storage was with Manta.[2] Here again, we built on ZFS and reliable, proven "tried and true" technologies like PostgreSQL and Zookeeper. And here again, we learned about really nasty, surprising failure modes at the margins.[3] These failure modes haven't led to data loss -- but when someone's data is unavailable, that is of little solace. In this regard, the data path -- the world of persistent state -- is a different world in terms of expectations for quality. That most of our domain thinks in terms of stateless apps is probably a good thing: state is a very hard thing to get right, and, in your words, tried and true absolutely beats novel and new. All of this is what makes Torus's ignorance of what comes before it so exasperating; one gets the sense that if they understood how thorny this problem actually is, they would be trying much harder to use the proven open source components out there rather than attempt to sloppily (if cheerfully) reinvent them.

[1] [http://dtrace.org/blogs/bmc/2010/03/10/turning-the- corner/](http://dtrace.org/blogs/bmc/2010/03/10/turning-the-corner/)

[2] [https://github.com/joyent/manta](https://github.com/joyent/manta)

[3] [https://www.joyent.com/blog/manta- postmortem-7-27-2015](https://www.joyent.com/blog/manta-postmortem-7-27-2015)

<comment><USER3> That was a pretty humble and good read. I don't think I'd have seen the autovacuuming issue coming. Actually, this quote is a perfect example of how subtle and ridiculous these issues can be:

"During the event, one of the shard databases had all queries on our primary table blocked by a three-way interaction between the data path queries that wanted shared locks, a "transaction wraparound" autovacuum that held a shared lock and ran for several hours, and an errant query that wanted an exclusive table lock."

That's with well-documented, well-debugged components doing the kinds of things they're expected to do. Still downed by a series of just three interactions creating a corner case. Three out of a probably ridiculous number over a large amount of time. Any system redoing and debugging components plus dealing with these interaction issues will fare far worse. Hence, both of our recommendations to avoid that risk.

Note: Amazon's TLA+ reports said they model-checkers for finding bugs that didn't show up until 30+ steps in the protocols. An unlikely set of steps that actually was likely in production per logs. Reading such things, I have no hope that code review or unit tests will save my ass or my stack if I try to clean-slate Google or Amazon infrastructure. Not even gonna try haha.

<comment><USER4> Anyone who's worked with Postgres at scale would guess autovacuum.

Postgres doesn't have many weaknesses, but most of them relate to autovacuum.

<comment><USER3> "Anyone who's worked with Postgres at scale would guess autovacuum."

Well, there's knowing it's autovacuum-related then there's the specific way it's causing a failure. First part was obvious. The rest took work.

"Postgres doesn't have many weaknesses, but most of them relate to autovacuum."

Sounds like that statement should be on a bug submission or something. They probably need to replace that with something better.

<comment><USER5> It's known to the postgres developers, and we are working on it. This specific issue (anti-wraparound vacuums being a lot more expensive) should be fixed in the upcoming 9.6.

<comment><USER3> Awesome! I already push Postgres and praise its team for the quality focus. Just extra evidence in your favor. :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Good... Good... Let the hate flow through you.

An open source OS company just _blogged_ about a new open source project that they are putting resources into. They did not release a commercial product that competes with any existing storage solution.

How exactly would one expect a _new_ project to be announced? Tell me more about how far away from done you think they are.

Sorry if that's a bit sarcastic, but seriously wouldn't it be nice if these comments where more along the lines of: "Interesting new project, here's some issues we've run into vis a vis reliable storage that you should look out for..."

<comment><USER7> Yes, I have to agree. There's a little too much negativity for my liking, especially from people who have a vested interest in downplaying the efforts of others.

</comment></comment><comment><USER8> I love CoreOS and they've done some super impressive engineering. But really, a new storage system? Rewriting in Go and using etcd for central state management makes things easier, but this is still a _hard_ problem.

Some things that need to be solved sooner or later: data replication so that N faults of X entities are protected against (X can be disks, enclosures, racks, data centers, regions, ..), recovery from failed disks, scrubbing, data management, backups, some kind of storage orchestration and centralized management.

If you look at Ceph which IMHO represents the state of the art in software defined storage, it took many years to get it to a point that it was usable. I hate to be cynical but in this case I would be surprised if CoreOS can pull this of. I wish them all the best though and would be happy for them if I am wrong.

<comment><USER9> It's like any time we take a step forward in one area we have to reinvent the last 50 years of computing to support it.

Persistent storage is a "hard problem" Really?

<comment><USER10> Persistent storage isn't a hard problem. Distributed, well performing, scalable and consistent storage is a hard problem.

<comment><USER11> Actually, persistent storage is fairly hard in itself. Look at what ZFS does to ensure data integrity in the face of phantom writes, dropped writes, bad controllers, and other implicit, non-fatal failures.

<comment><USER10> ZFS still also have the issue of having to perform well. You have a point, but ZFS is still trivial compared to a proper distributed filesystem, and you could achieve the same reliability much easier than ZFS if you sacrificed the performance.

<comment><USER12> The ClusterHQ guys behind the FlockerHQ found this out the hard way [0]. Initially Flocker was meant to provide a container data migration tool on top of ZFS, now it is a front-end to more established storage systems like Cinder,vSan,EBS,GPD and so on.

[0] [https://docs.clusterhq.com/en/latest/faq/#what-happened- to-z...](https://docs.clusterhq.com/en/latest/faq/#what-happened-to-zfs- support)

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> The CoreOS team is excited to make this initial release and start collaborating with folks that want to tackle distributed storage.

tl;dr this is a new OSS distributed storage project that is written in Go and backed by etcd for consistency. The first project built on top of Torus is a network block device that can be mounted into containers for persistent storage. It also includes integrations out of the box for Kubernetes "flex volumes". Get involved! :)

[https://github.com/coreos/torus](https://github.com/coreos/torus)

<comment><USER13> What's the current largest-scale deployment?

<comment><USER14> It's described as a prototype, so I'm pretty sure there's no deployment at scale at all.

</comment></comment></comment><comment><USER15> Storing container images is a great use case for an object store, not a block store. They should get sucked down whole to ephemeral storage on the compute nodes.

Mounting containers to a distributed block system is an anti-pattern. This is going to go poorly.

Ceph has had some really smart people working on distributed block for a lot of years, and they still have significant issues. It's not because they're dumb, it's because performant, scalable, and available distributed block is either hard or impossible.

<comment><USER16> This is about providing data volumes to containers, not about hosting container images or mounting containers themselves.

</comment></comment><comment><USER11> > At its core, Torus is a library with an interface that appears as a > traditional file, allowing for storage manipulation through well-understood > basic file operations. Coordinated and checkpointed through etcd's consensus > process, this distributed file can be exposed to user applications in > multiple ways. Today, Torus supports exposing this file as block-oriented > storage via a Network Block Device (NBD). We also expect that in the future > other storage systems, such as object storage, will be built on top of Torus > as collections of these distributed files, coordinated by etcd.

Am I understanding correctly that this is a file-based API? Distributing a POSIX filesystem effectively is very challenging, particularly since most applications that use them aren't written with CAP in mind; they don't expect a lot of basic operations to block for extended periods and fail in surprising ways, and they very often perform poorly when operations that are locally quick end up much slower over a network.

To be concrete:

> Today's Torus release includes manifests using this feature to demonstrate > running the PostgreSQL database server atop Kubernetes flex volumes, backed > by Torus storage.

It will be interesting to see how well this performs and how it behaves in the face of single-node failures and network congestion.

<comment><AUTHOR> It isn't a POSIX filesystem API. It is like a single file or a big distributed tape.

<comment><USER11> Virtualized, network block devices have all the same problems I described -- even worse, because the abstraction coneys even less about what an application is trying to do.

<comment><USER17> No. The difference is, that with network block devices you are usually only allow accessing the block device once. That's an easier problem than mapping POSIX file system semantics!

</comment></comment></comment></comment><comment><USER18> I don't understand why this is being used over say librados + librbd from Ceph. This seems like an awful lot of work to get the same functionality that Gluster/Ceph already have. Is there something I'm missing here?

<comment><USER19> I kind of see this as a general trend where CoreOS is concerned they reimplement everything in house, even when open source solution might already exist - rkt, etcd, now this.

<comment><USER20> what is the existing alternative for storage in distributed container environments?

<comment><USER21> Ceph, OpenStack Swift, GlusterFS, OrangeFS, Lustre.

[https://en.wikipedia.org/wiki/Clustered_file_system#Distribu...](https://en.wikipedia.org/wiki/Clustered_file_system#Distributed_file_systems)

[https://en.wikipedia.org/wiki/Object_storage](https://en.wikipedia.org/wiki/Object_storage)

<comment><USER22> If Lustre is the answer, you're probably asking the wrong question. Unless that question involves short life-span, massively parallel swap-outs from one weapons simulation to another.

I write this not as storage religion (of which there's far too much), but to warn away those who haven't experienced the many kinds of data (and stomach lining) loss that come with being a Lustre admin.

<comment><USER23> [https://www.csc.fi/web/blog/post/-/blogs/the-largest- unplann...](https://www.csc.fi/web/blog/post/-/blogs/the-largest-unplanned- outage-in-years-and-how-we-survived-it)

</comment></comment></comment></comment></comment></comment><comment><USER24> Okay, so from reading the developer docs at [https://github.com/coreos/torus/Documentation](https://github.com/coreos/torus/Documentation), I'm inferring that:

* Single reader/writer. If you try and set up more, Bad Things happen (even if it's single-writer, you don't get read-after-write consistency).

* It sure sounds like network partitions will allow all kinds of badness.

* If copy 1 goes down, you can keep operating on copy 2, then lose copy 2, have copy 1 come back up, and then warp back in time? Maybe that's prevented because of append-only and you just lose the data entirely because it was only replicating to one node due to the "temporary" failure.

Most egregiously, the Architecture description of an Inode implies that persisting a write-to-disk requires persisting the "INode". INodes are persisted in etcd. Which means your entire cluster's write-to-disk throughput is limited to what you can push through a Raft consensus algorithm.

Look, there are all kinds of reasons one could legitimately decide that none of the existing scalable block storage systems satisfy your use case. Maybe containers really are different enough from VMs. But the blog post claims that there just _aren 't_ any solutions; the research papers cited in the Documentation page are mostly old and are about systems in a very different part of this sub-space; and what developer documentation exists does not encourage me that this is a good idea.

Granted, I've been working on Ceph for 7 years and am a bit of a snob as a result.

<comment><USER25> They accept data loss, yes, and haven't figured out consistency yet. But given that it's an early prototype I would expect them to evolve and change things a lot in a couple of years, possibly throwing away raft and etcd.

Good work on Ceph, by the way. I've been following your work since it was a PhD if I remember correctly.

<comment><USER24> "haven't figured out consistency yet" is the issue. Seemingly very small decisions have huge impacts that you don't expect, and the state of the art is far enough along that you don't get better than the existing solutions except by exploiting newly-identified workload characteristics (or acceptable ways of losing data, loosening consistency, etc based on the workload) that you've planned out to make use of ahead of time.

One example: Both Gluter and Ceph have erasure-coded storage. Gluster's looks just like the replicated storage, only it involves more nodes and less overhead. Ceph's is severely limited in comparison: it's append-only, it doesn't allow use of Ceph's omap kv store, "object class" embedded code, etc. The reason is because distributed EC is subject to the same kind of problem as the RAID5 write hole: if a Gluster client submits an overwrite to 3 of a 4+2 replica group and then crashes, the overwritten data is unrecoverable and the newly-written data never made it.

Torus won't hit that particular issue because it is log-structured to begin with, which has all kinds of advantages. But garbage collection is _really hard_! Much harder than seems remotely reasonable! Getting good coalescing and read performance is _really hard_! Much harder than seems remotely reasonable! There's one big existing storage log-structured distributed storage system which has discussed this publicly: Microsoft Azure. They have a few papers out which hint at the contortions they went through to make block devices work performantly — and Azure writes first to 3-replica and then destages to the log! They still had performance issues!

[https://github.com/coreos/torus/blob/master/Documentation/re...](https://github.com/coreos/torus/blob/master/Documentation/research- links.md) points to a bunch of HDFS research and replacements; HDFS is designed for the opposite (large files with high bandwidth and nobody-cares latency) of what I presume Torus is targeting (high IO efficiency, low latency). Mostly the same for the Google papers they cite. There's no mention of Azure's storage system papers, nor of Ceph, nor anything about the not- paper-publishing-but-blogging stuff from Gluster or sheepdog; nor from academic research into VM storage systems (there's _tons_ about this!).

Can they fix a bunch of this? Sure. But the desires they list in eg [https://github.com/coreos/torus/blob/master/Documentation/ar...](https://github.com/coreos/torus/blob/master/Documentation/architecture.md) go towards making things _worse_ , not better. They aren't talking about how etcd can be in the allocator path but not the persistence path[1] and how every mount should run a repair on the data to deal with out-of-date headers. They talk about adding in filesystems, but not any way of supporting read- after-write (which is impossible with the primitives they describe so far, and _really hard_ in a log-structured system without synchronous communication of some kind). They discuss network partitions between the storage nodes, and between the client and etcd; they don't discuss clients keeping access to the disks but losing it to etcd.

[1] Using etcd for allocation would be a reasonable choice, but putting it in the persistence path is now. Right now a database in your container would require two separate write streams to do an fsync:

1) data write. It doesn't say in docs and I didn't look to see if replication is client or server-driven, but assuming sanity the network traffic is client->server1->server2->server1->client, with a write-to-disk happening before the server2->server1 step.

2) etcd write. Client->etcd master->etcd slaves (2+)->etcd master->client, with a disk write to each etcd process' disk before the reply to etcd master.

This is a busy-neighbor, long-tail latency _disaster_ waiting to happen.

<comment><USER26> These are very good points, but also probably much more constructive as GitHub issues, where they can be either answered or addressed. In the meantime, hopefully I can talk to a few of these:

>haven't figured out consistency yet I don't recall this being the case, but I'm not the authority on the matter.

>I presume Torus is targeting (high IO efficiency, low latency) The best- possible performing storage solution is _definitely_ not our primary goal, though we'll take it where we can get it. The most important goals for the project are ease of use, ease of management, flexibility, and correctness when the use-case desires it. Please note that the block device interface is only one of many planned. The underlying abstraction was designed (and will be improved) to support other situations.

>Using etcd for allocation would be a reasonable choice, but putting it in the persistence path is now. Right now a database in your container would require two separate write streams to do an fsync In Torus today (with the block device interface specifically), and with the caveat that I'm not the authority, so I may be slightly wrong, calling sync(), fsync(), and friends result in what I think you would consider an "allocation". Writes happen against a snapshot of the file (in this block storage case, the block volume is the "file"), and then a sync() makes those changes visible as the "current" version. Syncs hit etcd, writes do not.

I would really encourage you to submit feedback like this in GitHub issues. The project is still in _very_ early stages, and legitimate feedback can actually make a difference.

</comment></comment></comment></comment><comment><USER27> Where's the formal specification of the system? If we want these distributed systems to be reliable how can we be sure our algorithms and processes work if we do not first model them?

I'd be happy to work on a specification with the community.

<comment><AUTHOR> We haven't gotten that far down the path. We would love the help. I believe there are some issues related to documenting the architecture and failure domains for the v0.1.1 release: [https://github.com/coreos/torus/milestones/v0.1.1](https://github.com/coreos/torus/milestones/v0.1.1)

This is one of those classic "release too early or release too late" sort of things that got cut in preference to get early feedback and community participation.

<comment><USER28> Coming from the company that raised hell about Docker's lack of a spec, this is an interesting approach.

</comment></comment></comment><comment><USER21> Is the goal to provide POSIX file access as NFS and AFS do? The mention of object storage as a future direction feels like the early "the sky's the limit" euphoria of a new project that has yet to adopt specific end goals.

<comment><AUTHOR> The project doesn't provide a POSIX filesystem directly. But, instead provides a block device (think AWS EBS).

The initial goals that are in this release:

1\. An abstraction for replicated storage between machines in a cluster.

2\. A block device "application" on top that an ext4 filesystem could be run on. This is like an EBS.

In the future someone might build other applications like "object storage" or filesystems and we would love to get the feedback on the API to do that. But, that isn't in the initial goals and we will be focusing on the storage and block layers for the time being.

<comment><USER21> Will each disk in the system be weighted so that different size disks can be used in the system over time? OpenStack Swift offers this and is useful to not lock-in disk size at design time.

<comment><AUTHOR> Yes, see the admin documentation: [https://github.com/coreos/torus/blob/master/Documentation/ad...](https://github.com/coreos/torus/blob/master/Documentation/admin- guide.md#modify-my-cluster)

</comment></comment></comment></comment><comment><USER29> Seems really nice. Would love to see it how it compares to Ceph, we were considering Ceph for our container storage. For starters, it seems much more simpler to deploy.

<comment><USER30> In my old firm we managed to get ceph running on coreos using ceph in docker. The storage was exposed using a s3 gateway.

[https://github.com/ceph/ceph-docker](https://github.com/ceph/ceph-docker)

<comment><USER31> Docker supports native RBD since version 1.8. Which should be both faster and more reliable than using the S3 gateway (though that has other benefits and becoming pretty good too).

</comment></comment></comment><comment><USER32> I'm always in awe and grateful for the teams like CoreOS that tackle these tough problems.

But when it comes to container storage I'm confused.

The 12 factor app methodologies work very well for every service I have written and supported.

Processes/containers get ephemeral storage. When you need persistence you delegate to a stateful service, generally Postgres and S3.

Id much rather write my apps around these simple and well understood constraints then depend on magic file systems.

<comment><USER33> The average 12 factor app doesn't really need this, but as you mentioned, sometimes you delegate to Postgres, or S3, and today those are available as hosted products via Amazon, Google, Heroku, etc.

However, when your running in your own datacenter, you don't have the luxury of using Amazon's hosted products, so Torus exists to help provide some building blocks to build your own S3, or potentially even RDS.

</comment></comment><comment><USER34> Congrats on the relaase! How does this compare to GlusterFS (loved it, was free - I used it before they got acquired by RedHat) and Isilon (expensive but fast and reliable). Is there an upper limit on number of nodes(?) or total size?

</comment><comment><USER35> So is this the building blocks for an in-house EBS?

<comment><AUTHOR> Yes, that is the idea. Torus block is like EBS. Torus "library" could be used by other applications like an append only log, object store, etc.

<comment><USER36> How do you say that with conviction? I looked at the code and it has no bearings of being able to build a fault tolerant elastic block storage system.

</comment></comment></comment><comment><USER37> No mention of Jepsen tests, I look forward to a @aphyr call me maybe talk/blog post detailing all the issues.

</comment><comment><USER38> This reminds me of the SheepDog storage architecture ([https://sheepdog.github.io/sheepdog/](https://sheepdog.github.io/sheepdog/)) SheepDog provides safe distributed block devices and uses corosync or Zookeeper instead of etcd.

</comment><comment><USER39> Despite all the negative sentiment here, I am super excited about this. I use CoreOS heavily and really like how everything just works. Running Kubernetes on it, is the first cluster solution for me that works without configuration orgies and is robust against machine outages. Torus seems to be the missing piece. For now we use local volumes with sidecar containers for r/o storage and nfs volumes for r/w storage.

All other solutions are not practical. GCE and EBS are only single mount. iSCSI is unsupported in the cloud. Leaving only Ceph and Glusterfs, both mentioned here, but needing heavy configuration.

<comment><USER40> Gluster does not need heavy configuration: just add peers and create and start volume - 3 commands. Not harder than lvm. Of course, gluster has tons of options to fine tune cluster for various kinds of loads, which is not bad, but confusing for newbies. But if you are newbie, why you need to change defaults?

<comment><USER39> That's interesting. When I was looking at it, it seemed overwhelming. Do you have some pointers for the basics?

</comment></comment></comment><comment><USER41> Watch out, next we'll be seeing CoreOS writing their own encryption protocols. As someone that's designed and built storage clusters from the ground up - take it from me: storage is not as easy as it seems if you care about your data and performance at scale. The number of people I've seen using CoreOS and then moving away from it is quite alarming, I feel like CoreOS will become the Ubuntu of the container world.

<comment><USER42> They're all-in on Kubernetes, which is quite apparent given (a) who's funding them and (b) product direction, including this one. They don't have the resources or backing to go after Docker; notice that changed? Docker has something like 15x the funding and is pulling away in a lot of ways, so CoreOS hitched on the Kubernetes wagon.

Better bet for destiny: Google is letting them build stuff off the books by just funding them, and at some point they'll get quietly bought with all their stuff added to Kubernetes, and that'll be that.

</comment></comment><comment><USER43> So a userspace fs, I'm guessing this will use fuse to actually expose a POSIX fs? Small sync writes will absolutely kill performance and will require all sorts of hacks like glusterfs has had to implement due to the amount of context switches. CoreOS really should have added whatever needed to ceph, filesystems are not something you just hack together overnight.

<comment><USER26> We had a POSIX interface early on (via FUSE), but decided to expose a a block storage interface first instead. This is not a "filesystem", it's a storage abstraction, and we've spent more than 6 months on it. Seems like a lot of folks are quickly jumping to conclusions, which is expected from "the internet", but I would have hoped for better from HN.

<comment><USER43> 6 months is overnight in terms of a filesystem, there are literally millions of dollars of work that was done with glusterfs and probably a magnitude less man hours. I understand coreos wants.to innovate but what does torus bring to the table besides negatives and NIH?

</comment></comment></comment><comment><USER44> Funny, just this week I was looking for an easy solution to cluster storage for containers.

I found SXCluster and posted it here [https://news.ycombinator.com/item?id=11812566](https://news.ycombinator.com/item?id=11812566) \- not directly geared towards containers, but very easy to setup (no need for etcd and the like).

</comment><comment><USER45> I just hope the performance is visibly better than of gluster. Especially for small files.

<comment><USER20> are there any performance metrics for Tarus yet?

</comment></comment><comment><USER46> You could also check out EMC's Unity midrange storage at the EMC Store. [http://bit.ly/1SIZ9N7](http://bit.ly/1SIZ9N7)

</comment><comment><USER47> How is this different than Tahoe-LAFS?

</comment><comment><USER48> Is it possible that Facebook haystack will be open source?

</comment><comment><USER42> I love that I know who this is at CoreOS by the choice of username. Pick better throwaways if you're going to blow up a thread like this. (ideal0227 upthread is also a CoreOS employee and one of the etcd developers.)

On topic, yes. It is quite trivial to lose data with etcd and pretty much everybody I know who runs it has experienced problems. Try backing it up and restoring. etcd is compsci great yet operationally burdensome; it is _extremely_ difficult to operate, tends to make assumptions and explain to you how to operate your fleet, and the developers are not very receptive to these and other operational concerns. This stems from CoreOS culture, to be quite clear -- CoreOS seeks to eliminate operations as a discipline and replace it with software, and tends to disregard or devalue operational concerns as a result.

Launching down the storage path with Go software and disregarding several decades of operational experience in the industry on how to do distributed storage correctly (not to mention the "looking down" on etcd-related infrastructure decisions from this anonymous employee) should indicate to you that I'm not making this up, despite how it may appear.

90% of folks I know on etcd, I'd estimate, have (a) reverted to Zookeeper or (b) moved on to Consul. It is the single piece of software that is holding Kubernetes back, too, and it's fairly obvious from roadmap direction that Kubernetes is now _the_ etcd customer. Plan accordingly.

<comment><USER49> Cockroachdb is built on top of etcd. So far seems to work very well for their product.

I think this whole thread is a bit harsh and unfair to CoreOs and their solutions.

<comment><USER10> I think that reflects the complexity of a storage product more than CoreOS itself. I use CoreOS. I like it. I love the update mechanism, and the tight focus on running as much as possible in containers.

But I also have spent enough time using it to run across certain warts, and Etcd clusters refusing to start without manual intervention etc. have been a frequent enough problem that when they go after something incredibly hard such as distributed storage, and it's relying on Etcd, I see that as a scary combination.

</comment></comment></comment><comment><USER50> It's threads like these where you start to wonder just how many marketing teams are arguing with each other in the comments. I take the FUD with a grain of salt: there is a lot of financial incentive to create FUD, where there is financial incentive and little to no regulations a market will naturally arise. It's going to get worse as the bubble pops and companies become more and more desperate.

<comment><USER51> I don't think it's marketing teams. There are at least a half dozen CoreOS employees/contributors commenting here, plus a couple of known allies. On the "other side" there's Bryan, me, and one person who seems to be an ex-employee. AFAIK _none_ of them are in marketing, or even in coordination with marketing. Certainly, I often get flak from my company's official mouthpieces for saying things that conflict with their preferred talking points. They _wish_ they could control what I say.

Mostly I think this is a matter of people naturally standing up for their friends and colleagues, which is a wonderful thing, vs. people who have specific concerns about The Right Way to do either technical or non-technical things. No coordination or collusion is necessary. You can see exactly the same thing happen for _every_ company or project that's discussed here, or on Twitter, or wherever. Try criticizing a YC portfolio company some time. Not all of the people arguing with you will admit their affiliations, and of course the downvotes are all anonymous anyway. Just remember, the more skin someone has in the game, the more tempted they'll be to cross that vague line into astroturf. All the rest of us can do is admit our affiliations and biases, and hope that people will get their heads out of the _ad hominem_ gutter enough to reach rational conclusions about the facts being presented.</comment></comment>


Ask HN: OSQA like system written in PHP?  - <AUTHOR>

I need a OSQA like system written in PHP. OSQA is excellent but is written in Django/Python. <comment><USER2> [http://meta.stackoverflow.com/questions/2267/stack- overflow-...](http://meta.stackoverflow.com/questions/2267/stack-overflow- clones)

(I run a OSQA site, so can't recommend any of these, other than OSQA :-) )</comment>


2012 Screen Reader User Survey Results - <AUTHOR> http://webaim.org/projects/screenreadersurvey4/

<comment><AUTHOR> Noteworthy is that only 1.4% of screen reader users had JavaScript disabled.</comment>


Thunderbolt 3 Unblocker - <AUTHOR> https://github.com/rgov/Thunderbolt3Unblocker <comment><USER2> From the TB3 enabler GitHub:

"For unknown reasons, Apple decided to block the support for some categories of Thunderbolt 3 peripherals under macOS and when you connect those Thunderbolt 3 peripheral, you simply get an "Unsupported" message under Thunderbolt Device Tree. After digging around, it turns out the block is implemented in software level and it is possible to bypass the block by patching the related kext. This patch modifies IOThunderboltFamily and allows "Unsupported" Thunderbolt 3 peripherals to work under macOS Sierra."

I wonder why they did that. Stability, probably?

<comment><USER3> Security issues, most likely. Like Firewire, Thunderbolt suffers from the same security issue in that to achieve its high speeds, it gives connected devices direct memory access. In other words, any connected device can read and write directly from your RAM, bypassing any kind of access control built into the OS.

[https://en.wikipedia.org/wiki/Thunderbolt_(interface)#Vulner...](https://en.wikipedia.org/wiki/Thunderbolt_\(interface\)#Vulnerability_to_DMA_attacks)

<comment><USER4> Thunderbolt is basically external PCIe, so that's not surprising.

Quite frankly, I'm not worried about physical access being a "vulnerability" \--- I'm of the "if you physically own it, you should logically own it" mindset.

<comment><USER5> While I mostly agree with this, I don't agree here. Maybe you need to connect your notebook to a projector and they hand you a thunderbolt to hdmi cable.

They don't phsically own your device and yet you're possibly fucked.

<comment><USER6> Couldn't a simple OS "permissions" system mostly fix that loophole (as much as it can be fixed)?

"[device name] is requesting [scary sounding permission] Allow?"

It won't protect users from themselves, but it will protect "bad-usb" style attacks.

<comment><USER7> No, because when you have full access to RAM, there's no actual distinction in those messages for thunderbolt devices.

Unless maybe you meant just the same scary message for every thunderbolt device, in which case people become acclimated to ignoring the message. And that may be after they've wasted support time asking why some device wants this scary access.

<comment><USER8> IOMMU? You still get to lock down access per device...or the Mac laptops don't have an IOMMU?

<comment><USER9> Many laptops ship with a disabled IOMMU or without one.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> The code for xnu_override (the library the author wrote to patch functions in the kernel) is wrong for a number of serious reasons.

It assumes (with no verification) that the first twelve bytes of any kernel function you could want to patch are all one basic block and do not contain a jump target or a relative branch. These reasons _ARE_ why live patching is hard and why proper support for it is basically required in original code (making sure each func starts with a basic block at least twelve bytes long). Without that, it's quite a dangerous game to play in kernel space.

Furthermore, it assumes RAX is safe to clobber. This may not be so. Compilers are generally free to ignore standard ABI when calling provably-static provably-leaf functions. Who said RAX doesn't have a useful value on entry to patched function?

Additionally, it edits cr0 and cr3 with interrupts enabled, allowing it to be preempted during this action. If said preemption happens, another edit to those registers could happen in between. Scary, since what's being done is a non-atomic read-modify-write on CPU control registers.

Strongly advise anyone from running this on any system.

<comment><USER11> > Strongly advise anyone from running this on any system.

Oh come on. Your para 2-4 are serious conceptual defects with xnu_override in general, but this is the kind of hacker shit you'd expect to find in something that will live patch your kernel code. At the point in time you find yourself turning off SIP to load some code to hot patch your kernel, you've long since crossed the Rubicon of caring about running particularly good code.

The odds are good that the functions this thing is set up patch do hold with your first two invariants (RAX is scratch, there's 12 bytes in the entry BB). The third is the race, and you'll probably win this race most of the time.

"You'll probably win this race most of the time" shouldn't be in any serious engineering design document and is probably grounds for demotion or firing if you bring it up as a defense during a design or code review, but that isn't what's going on here. This is some hacker shit posted to github.

<comment><USER10> And if this project was one monolithic thing, I'd agree. But the page sells this _xnu_override_ as a "reusable library" which might lead someone to try to reuse it in another place. This is scary and hence my warning. Hell, tomorrow's apple 10.13.0.0.0.0.0.0.minor.insignificant.0.1 patch could change these "invariants"

<comment><USER11> You're right, and I was reacting poorly to the "don't run this on anything" statement. It's some hacker magic off github, caveat empor. You're also right that future updates could break it. It's brittle. I won't be running it myself.

I bet, though, that 99% of the time, it works every time, and for its use case, that's "good enough." Hot patching the kernel is scary, though, and hopefully no one walks away from looking at that github thinking "oh yes, I know so many problems that can be solved by hot patching the running kernel" because honestly getting the act of patching the physical code itself is only the start of your troubles.

</comment></comment></comment></comment><comment><USER12> FYI, this block was found back at the launch of the first Thunderbolt 3 Macs to be because Apple (for whatever reason) didn't wish to support the older Thunderbolt 3 chipset, according to peripheral manufacturer, Plugable:

 _The version of OS X on the new MacBook Pros (late 2016) will not work with existing Thunderbolt 3 docks and adapters that were certified for Windows prior to the release of the MacBook Pro. These existing devices use first generation of TI USB-C chipset (TPS65982) in combination with Intel's Thunderbolt 3 chipset (Alpine Ridge). Apple requires the 2nd generation TPS65983 chipset for peripherals to be compatible. Certification of solutions across different device types is still in-progress for this 2nd generation chipset. From the Plugable product line, our dual display graphics adapters for DisplayPort and HDMI (TBT3-DP2X and TBT3-HDMI2X) are affected… We've also postponed our TBT3-UD1 Docking Station to update to the TPS65983 chipset and re-certify to make this docking station MacBook-compatible. Our Flagship TBT3-UDV dock with Power Delivery /Charging was already planned to use the next generation controller chip from TI, and will be compatible with the 2016 Thunderbolt 3 MacBooks._

I still don't think we exactly know why, but the allegations that they're doing it for profit or for some attempt to control the ecosystem don't seem to hold up, especially since most of these peripheral manufacturers since updated their devices with the new chipset to work with macOS just fine.

</comment><comment><USER13> Why would anyone (specifically developers) buy another apple product if they now start blocking hardware from working on PCs?

Them blocking battery extenders with headphone jacks on the iPhone is already bad enough, but doing it to PCs is a whole other level of walled garden.

<comment><USER14> What if the hardware _doesn 't_ work on the PC though? Isn't that the point?

<comment><USER15> FWIW, the current case seems to be "hardware works just as the spec says, OS blocks in anyway."

Mind you, I've seen many types of HW, some of which indeed wouldn't work "with a PC." The expected course of action was always "complain at peripheral vendor," this "we know what you want, but nope because FU that's why" has historically been reserved for vendors trying to keep out competition.

So, while it is theoretically possible that an OS-enforced soft block is there to protect the user, a statistically more probable version is that it exists to protect the OS maker from competition; IMNSHO the ball is in Apple's court to prove otherwise.

<comment><USER4> _FWIW, the current case seems to be "hardware works just as the spec says, OS blocks in anyway."_

I've noticed a disappointing and somewhat repulsive trend increasing over the past few years, one which not only Apple participates in (but they are one of the most notable), of moving from "not supported" meaning "we won't help you with it" (fair enough) to "we will actively stop you from doing it" (FU for even trying to do something we didn't think of/don't want/etc.).

IMHO it's perfectly reasonable for a company to not offer support for literally every hardware combination out there, because that's a huge space; but to deliberately sabotage attempts at doing things outside of that boundary is incredibly hostile. I have no definitive evidence but this trend seems at least somewhat correlated with the rise of authoritarianism.

As a developer myself, and also someone who has done many "unsupported" things with no ill effect, I constantly try to fight this position from others, but it's difficult.

</comment></comment></comment></comment><comment><USER16> That is great, I found Thunderbolt the best solution for using notebooks at home and work (multiple monitors, USB ports). I only hope in the future you can connect external GPUs in some way. I have tried via PCIe with an external board but it is not the best solution (performance, unsupported), it is just a hack.

<comment><USER17> I've been using eGPUs for gaming (on Windows though) on my MacBook Pro for a few years now. macOS introduced official eGPU support and now there seem to be numerous compatible enclosures: [https://egpu.io/macos-high-sierra-official- external-gpu/](https://egpu.io/macos-high-sierra-official-external-gpu/)

<comment><USER18> Which GPU are you using?

<comment><USER17> nvidia gtx 970. My setup is pretty diy with an external 200W power supply. Today I would get one of those nice enclosures mentioned in the article. You loose around 10% performance compared to an internal GPU.

BTW: Windows 10 also has pretty solid eGPU support on my MBP at least.</comment></comment></comment></comment>


Hacker News Reader is Free Today - <AUTHOR> https://itunes.apple.com/us/app/hacker-news-reader!/id532402679?ls=1&mt=8

<comment><USER2> What problem does this app solve? I find using browsers for websites is more usable for me than using an app. I can have many tabs open, I can copy anything I need, press on links and do what I have been doing on the web for 20 years. All those apps brake the web, and make it actually harder to use it...

<comment><AUTHOR> I personally hate browsing HN in a browser, even on computer. I use a plugin for chrome that does something similar to the apps where I can easily flip back and forth between the comments and the article. Especially with the rapidly expiring links, I find flipping back and forth to be useful.

This also uses the bigrss to get several pages on stories instead of just one.

<comment><USER3> which plugins?

<comment><AUTHOR> Hacker News OnePage - [https://chrome.google.com/webstore/detail/hacker-news- onepag...](https://chrome.google.com/webstore/detail/hacker-news- onepage/plldnnbdlbgbiknjebohmlggcbicghlj)

</comment></comment></comment></comment><comment><USER4> Has pg ever said why he doesn't just fix the stylesheet so it's usable on an iPhone? If you're not familiar, it looks like this: <http://i.imgur.com/Mzrk6l.png>

The site design is responsive to the width changing, but Safari just picks far too wide a width for a phone.

<comment><USER5> Well, all you need is a meta viewport tag, like so:

      <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

With that, viewport width will match CSS pixel width of the screen.

<comment><USER6> All you need is a couple fields in your profile that lets you use a custom CSS and a custom JavaScript file... You know, like the current `topcolor:` field.

</comment></comment></comment><comment><USER7> I always thought that phones powerful enough to have apps are also powerful enough to have browsers.

How is this app better than just using a browser?

<comment><USER8> The iphone doesn't zoom like the android browser and makes hacker news more difficult to use in portrait mode. (Tiny text and tiny buttons)

<comment><USER9> The tiny text problem, at least, can be solved with a bookmarklet for increasing the font size. In fact, having such a thing around is almost essential when using mobile Safari, at least if you're over a certain age.

<comment><USER10> You don't find it a problem (read: annoying) that you need to run a bookmarklet on every single page load?

<comment><USER9> Yes, yes I do. It would be far better if mobile Safari had a configuration for a minimum font size.

<comment><USER9> And even better if people stopped writing stylesheets that created unreadable pages on small screens. It's not that hard to write CSS that makes your site readable on any size screen. A page with no CSS works fine. If your text gets too small on a phone, you've added CSS that breaks it.

</comment></comment></comment></comment></comment></comment><comment><USER10> Anyone have any suggestions for an Android HN app?

I have this one currently:

[https://play.google.com/store/apps/details?id=com.airlocksof...](https://play.google.com/store/apps/details?id=com.airlocksoftware.hackernews&hl=en)

Just curious what else you guys might be using.

<comment><USER11> This one - [https://play.google.com/store/apps/details?id=com.rickylaish...](https://play.google.com/store/apps/details?id=com.rickylaishram.hackernews) and this one - [https://play.google.com/store/apps/details?id=com.manuelmaly...](https://play.google.com/store/apps/details?id=com.manuelmaly.hn) are the best ones currently imo.

</comment></comment><comment><USER12> I like: ihackernews.com

I'll check out the app though.

<comment><USER13> same here, i tried many apps and web apps a few months ago and i ended up saving ihackernews.com on the springboard. It is perfect (except that you can't see the older posts).

</comment></comment><comment><USER14> Shameless plug ahead. I checked all the HN clients for iOS (I think) and I didn't like any of them. Their designs are usually rough and they don't add much functionality to the site. That's why I started writing a better HN app and I'm documenting the process. If you're interested about it, you can see my last post about the app here <http://www.rainycape.com/post/300/building- hackerful-day-2/> (currently writing post about day 3).

</comment><comment><USER15> I don't like it. Pages seem to load slower than in Safari, it seems to think that people will read articles before the HN comments (target area is way larger), I find the text layout worse (it wouldn't change my opinion, but can't the "Show WebView of comments" toggle stick?) and there is no "Reader" functionality on web pages.

</comment><comment><AUTHOR> Dev here, just getting it out there hoping to get some feedback and reviews. Note, there is a known bug in the iPhone version that cuts the last line off on a few comments, but it will be fixed soon (and you can also just go to the web version instead of the custom comments).

<comment><USER16> It's a little unfortunate that there are two apps named "Hacker News Reader" and "Hacker News Reader!". Nice app though.

<comment><AUTHOR> yeah, i had an "!" in the title at one point and it got modded out. Guess they didn't look into it enough to realize that's the name of the app (and as you noted is important)

</comment></comment></comment><comment><USER17> I use the HackerNode iPhone app and find it to be a fantastic way to read HN from my phone. Not free though but I would definitely recommend it.

<comment><AUTHOR> I liked HackerNode but the lack of the ability to comment (and lack of threaded comments) made me make this. Have they added commenting yet?

<comment><USER17> No commenting but I'm a read-only lurker on my phone so that hasn't bothered me. I'll give your app a try.

</comment></comment></comment><comment><USER18> Flipboard is pretty good for accessing Hacker News. To be totally honest, this app doesn't seem totally useful.

</comment><comment><USER19> how much did it normally cost?

<comment><AUTHOR> it started at 1.99 and has fluctuated between .99 and 1.99

</comment></comment><comment><USER20> when searching for "hacker news" on the app store this app is 16 or 19. Very difficult to find even when you use the full name. Searching for 30 south doesn't help either. Apple's app store search is miserably bad.

<comment><AUTHOR> Agreed. In this case there just aren't that many exciting things to call a hn reader app, but in general the search is really difficult since the changes this summer. They lowered the importance of the name a lot, so even finding an exact match is tough.</comment></comment>


Show HN: Automatically turn your blog posts into emails for your readers - <AUTHOR> https://blogsend.io/?ref=hackernews <comment><USER2> Have you considered the recruitment use case? Many staffing agencies advertise jobs on their website but don't have the job alert functionality a job board would offer. If you turn this into a "alert me", the next time they post a job matching the criteria you'll email candidates for them. Same technology, just a different niche - one that I think will be more profitable.

<comment><USER3> t's not the same technology. Job page doesn't have RSS. It will be hard to read and get an update from different agencies/company career page.

<comment><USER2> That's an easy one to solve by running it through fetchrss or feedity.

</comment></comment></comment><comment><AUTHOR> Hi HN! I made Blogsend.io.

It's a way for blog owners to handle emailing their readers automatically whenever they make a post. I built this because I actually needed it first, but decided to try it out as a product as well.

Technically it's not terribly complicated – it has a widget which collects emails of your subscribers and it listens to your RSS feed and emails those people when you post. The goal is to make it as hands-off and hassle-free as possible to email out your content to your readers.

Built it in Node.js and Postgres, with Bulma providing the CSS defaults. I vowed to move fast in this project and to that end I decided to stay away from frontend frameworks and build the old fashioned way, with <form> tags! It was actually super fun. I've had my hands on React, Vue, Angular and Ember before, and in all of them I actually never really felt as if it felt as natural as this approach.

Anyways, I'd love to hear any feedback you might have :)

<comment><USER4> Do you store the list of emails per blog?

<comment><AUTHOR> Yes, I do! At present they're not exportable though. I think one part of the product that's nice in that way is that as a subscriber, you know you don't need to think about your email getting spammed - communicating with subscribers is only done as a post is published.

<comment><USER5> > At present they're not exportable though.

You may want to think this one more through. As a publisher, I would definitely be in the market for your product but the mailing list is the most valuable (and expensive) data set I will be working on for years, and to have it not exportable in the case that you might go out of business or if I decide to use another product is a definite showstopper for me.

<comment><AUTHOR> That makes sense!

Just out of curiosity, what size is your blog?

I spoke to several blog owners of various sizes. The smaller ones tended to not care about this, but indeed the bigger ones took more of an interest in the actual emails they collect. Just wondering if you'd fall into one of these categories.

</comment></comment></comment></comment></comment><comment><USER6> Not to detract from the project, but is there much demand for this? Anything that's emailed to me is generally lost in my mailbox or purposely ignored... when I want to read a blog, I'll go to the blog.... I don't want more emails.

<comment><USER7> Why don't you clean your inbox and setup some filters? I'd say 99% of the emails I get is relevant. Blog posts from blogs I actually signed up for is great to get. And if you start to dislike it it's 1 click to unsubscribe.

</comment></comment><comment><USER8> Here is our version of this concept, except it also works with your social media accounts: [https://www.gibbonwire.com](https://www.gibbonwire.com)

plus we take privacy very seriously, which is an added bonus.

edit: here is a link to our FAQ: [https://www.gibbonwire.com/about/faq/](https://www.gibbonwire.com/about/faq/)

and our privacy policy: [https://www.gibbonwire.com/about/privacy/](https://www.gibbonwire.com/about/privacy/)

<comment><AUTHOR> Good luck :)

</comment></comment><comment><USER9> Doesn't WordPress already do this? Not sure this is very helpful for WordPress users. Although, I do see the use for static blogs and those that do not already send emails when posts are published.

<comment><AUTHOR> You're right, WP does have several plugins that do this, the most popular being from providers like ConstantContact and Mailchimp. I think I'd expect a few WP users but you're right, my target are probably people that are using something like Jekyll.

</comment></comment><comment><USER10> Dig it! But I'm clicking on Preview and getting a I need to subscribe?

[https://cl.ly/709c92e9ca7b/Image%202018-11-20%20at%2011.12.4...](https://cl.ly/709c92e9ca7b/Image%202018-11-20%20at%2011.12.45%20AM.png)

Get this:

[https://cl.ly/eac5828099c5/Image%202018-11-20%20at%2011.13.2...](https://cl.ly/eac5828099c5/Image%202018-11-20%20at%2011.13.22%20AM.png)

<comment><AUTHOR> Oh no! I'll check it out! Could you message email@example.com and I can check out the problem!

</comment></comment><comment><USER11> My blog is obviously published as an RSS feed but I also have a JSON feed ([https://jsonfeed.org](https://jsonfeed.org)). Don't laugh, there are dozens of us. Dozens!

Considering you've already done the hard work of parsing RSS, do you plan to support JSON feeds at some point?

<comment><AUTHOR> Hello!

Looking at the spec, it appears to differ from RSS/Atom in very small ways (besides being in a different data format). It would be pretty trivial to build in support for this.

It's difficult to find usage stats since searching for "json feed" unfortunately gives a whole lot of unrelated things!

I think if I can find a paying client that wants it, I will build this in. Wanna be the first? ;)

</comment></comment><comment><USER12> Does it also do emails to blog posts? I have a number of carefully considered emails I'd like to publish.

<comment><USER13> We have a site to do that, [https://www.fwdeveryone.com](https://www.fwdeveryone.com)

In addition to sharing on social, it also supports embedding email threads directly within Reddit threads, Medium posts, etc., e.g.:

[https://medium.com/to-the-best-of-our-knowledge/were-not- ok-...](https://medium.com/to-the-best-of-our-knowledge/were-not- ok-2b87b413dee2)

Here are some video tutorials showing more functionality:

[https://www.youtube.com/playlist?list=PLJAEYmnEjIu4-- bpq5LSK...](https://www.youtube.com/playlist?list=PLJAEYmnEjIu4-- bpq5LSKZ9LS44po_NUx)

And we'll have a Gmail add-on submitted to Google within the next couple weeks, so hopefully that will be live by the end of the year. (Since I know a lot of HN folks have reservations about OAuth and have been asking for another way to upload content.)

</comment></comment><comment><USER14> Nice project! I currently do the same with Mailchimp, I have a mailing list and they support creating newsletters from RSS. Does your project have advantages over that?

<comment><AUTHOR> Yes indeed, MC does this pretty decently. I tried it myself, but I found the product is very much catering to marketers. Which is fine, of course, but for people that only want this functionality, it seems like overkill.

Blogsend is meant as a much simpler, more focused alternative. This is all it does, and the features to be added in the roadmap are almost exclusively to help bloggers, not marketers or a general audience.

<comment><USER14> I see, that makes sense, thanks!</comment></comment></comment>


Uber Hires Veteran NASA Engineer to Develop Flying Cars - <AUTHOR> https://www.bloomberg.com/news/articles/2017-02-06/uber-hires-veteran-nasa-engineer-to-develop-flying-cars <comment><USER2> I feel this is additional evidence for my theory that Uber is in big trouble and is trying to spin a bunch of pixie dust into some kind of dumb money exit for its increasingly angry investors.

<comment><USER3> The "self driving car" meme is petering out, so they are attempting to create some hype in order to keep the Ponzi investment flows churning.

<comment><USER4> By which you mean, coming to fruition?

I'm not the biggest fan of Uber on a number of issues but I think the biggest problem they have is that their investors were planning on having a monopoly on livery. That is clearly not going to happen. Flying cars and self driving cars are actually significantly more sound than their business model and are much more likely to provide a decent ROI. If and when Uber runs out of steam and its assets are liquidated the most valuable part will be research.

Lets check back on this in 10 years.

<comment><USER5> Which could be true if Uber's self driving car project lasted longer than a week. Other companies have already logged tens of thousands of road hours into their self-driving car research.

At this point, it seems more like these announcements into researching more solid business models are a red herring, like the previous commenter said. You don't need to see the end product to know if progress is being made.

<comment><USER4> They did hire some of the top self-driving car (can we call them autocars?) researchers.

>"these announcements into researching more solid business models are a red herring"

My view is that they are attempting to develop a more solid business model. Why wouldn't they try? Even if they fail to turn it into business it creates value that can sell to the companies that do succeed. Besides if they need a shiney for their investors, faking it seems to be just as much work as actually doing it.

</comment></comment></comment></comment></comment><comment><USER6> This strikes me as pretty pathetic effort by Uber to make itself appear forward looking

Flying is not a valid commute solution and never will be because of physics. All things equal, the energy requirements to change elevation will be too great to justify adoption.

The system only becomes useful/safe if you "stack" vehicles on roadways. What happens when one has a failure and crashes?

This is either market signaling or another thing Uber will bleed money onto.

<comment><USER7> If we approach the problem from a perspective of "How to make flying a viable commute option?" we can come up with atleast a few scenarios

1) Cities near water bodies like SFO, NYC, Miami or even Chicago can benefit from over water flying cars. This provides some safety while making more areas livable within a short commute to downtown.

2) Places like Mumbai, Bangalore where car ownership has outpaced the governments ability to create roads causing massive traffic jams can be served by a vertical stacking approach you mentioned and leap frog the western worlds freeway model. By creating pre-defined paths in the air and having only autonomous flying cars, the governments can save money on building roads, parking etc and also provide a level of safety against rogue drivers and airspace violation

<comment><USER8> > Cities near water bodies like SFO, NYC, Miami or even Chicago can benefit > from over water flying cars"

I don't agree. While there may be less impediments to putting that traffic over the waterways the challenges are still significant.

Just using NYC as an example:

A high density of bridges.

A High volume of existing commercial and passenger boat traffic.

A number of busy heliports are located along the waterways.

And although the rivers could help you navigate you up or downtown they provide no efficient means of going across town further limiting this as a viable option.

</comment></comment></comment><comment><USER9> There's a flying car company, Terrafugia (1), that came out of MIT ten years ago and they have working prototypes. There's an old YouTube video here (2) and apparently they have "persuaded the FAA to grant Terrafugia a waiver of weight and stall-speed limits so it can self-certify its Transition flying car as a light sport aircraft" (3).

Not sure of the safety implications or the commercial viability of flying cars for Uber, but there are people out there who want to be able to own or fly an aircraft and not have to deal with various hassles that come with traditional aircraft ownership.

1\. [https://www.terrafugia.com/](https://www.terrafugia.com/)

2\. [https://www.youtube.com/watch?v=nnF2yua4KIw](https://www.youtube.com/watch?v=nnF2yua4KIw)

3\. [http://aviationweek.com/technology/faa-light-sport- aircraft-...](http://aviationweek.com/technology/faa-light-sport-aircraft- waiver-clears-way-transition-flying-car)

<comment><USER10> Unfortunately, not really out in production yet, just like the amphibian Icon A5 [1].

With aviation certification requirements, it takes huge amounts of time and capital to get it from concept to prototype/first flight, and unfortunately also from prototype/first flight to production and customers.

[1] [https://en.wikipedia.org/wiki/ICON_A5](https://en.wikipedia.org/wiki/ICON_A5)

</comment></comment><comment><USER11> Projects like this will eventually produce an airborne commuter vehicle, but:

* It will resemble a scaled-up quadcopter (for which there's already an early prototype: [http://www.theverge.com/2016/6/8/11882458/ehang-passenger-qu...](http://www.theverge.com/2016/6/8/11882458/ehang-passenger-quadcopter-test-flights-nevada))

* It will only happen when batteries are much better than they are now -- more energy density, longer lived, more efficient.

The bottom line is that this idea, like so many around us, is waiting for decent batteries to be invented.

<comment><USER10> I don't quite get that the eHang commands such a presence in the public mind. With the rotors where they are, it looks almost designed to maim and cripple passengers and bystanders.

The Volocopter, by way of contrast, has the rotors above the cockpit, has 18 rotors for redundancy, and is actually already flying for both unmanned and manned test flights, with preliminary approval by German authorities. In my view, it's a more sensible concept that's also further advanced already.

[http://volocopter.com/](http://volocopter.com/)

<comment><USER11> > I don't quite get that the eHang commands such a presence in the public > mind.

I think it's only because they're the first, their craft is really a rough prototype, and in five years they'll likely be forgotten.

> With the rotors where they are, it looks almost designed to maim and cripple > passengers and bystanders.

That's my reason for thinking they're engineers who aren't thinking of the needs and vulnerabilities of end users -- they aren't really thinking about end-user issues.

> The Volocopter, by way of contrast ...

* Much safer design, better protection against any single-point failure (the eHang can't afford to lose even one of its props and remain controllable).

* But soo ugly. :)

To me, the first large quad with protective shields around its props will signal a level of seriousness consistent with the supposed goal (end-user acceptance, non-prototype).

</comment></comment></comment><comment><USER12> When does Uber run out of money? I think the timeline on their R&D projects is longer than their runway, and I'm not sure how many more people would be willing to write Uber a 10 digit check.

<comment><USER6> They've raised _a lot_ of money in the last 18 months (likely in anticipation of an evolving financial environment) so I suspect they actually have a couple years left if they're careful.

It's like they're stumbling forward blindly hoping that at some point their books balance.

<comment><USER12> I also forgot they bought Otto. I think self-driving cars are a long way away, but I believe self-driving trucks could potentially be viable in a few years - and the market is _massive_. Maybe they can commercialize that soon enough to save themselves. Apart from the technology though, I'm sure the regulations (something Uber loves) and customer relations could be a tough nut to crack.

<comment><USER6> You are completely right. Shipping automation has a lot of potential.

I don't think Uber has any strong competitive advantage in the self-driving space relative to all of the other players. I've been wrong about these things before. The thing that may put them at a disadvantage is their history of ignoring regulations compared to companies like Google. Even if they develop the technology quickly enough they may be screwed because a competitive technology is the one that gets licensed by the DOT because other companies play by the rules.

<comment><USER12> I'm biased because I used to work with many of the people Uber hired for self- driving cars, but they bought _really_ great engineering talent. Great talent doesn't mean much though if the management is bad, and after their whole illegal testing in California debacle it makes me cautious.

That said, I didn't realize just how far ahead Google/Waymo is with regards to self-driving cars. If you check out the report of autonomous vehicles (HN "discussion" here [https://news.ycombinator.com/item?id=13554796](https://news.ycombinator.com/item?id=13554796)) they are already operating on highways with zero problems! The data is limited, but that is promising.

<comment><USER6> > I'm biased because I used to work with many of the people Uber hired for > self-driving cars, but they bought really great engineering talent

I think that's a fair perspective and I don't discount the capabilities and talents of the people working at Uber, it's the people running Uber that I don't believe are capable of steering the ship.

A friend of mine who has been an executive at a couple tech companies have chatted about the "no grown-up" phenomenon. Essentially, you can have a ton of super-talented people but if there isn't a 'grown-up' (irrespective of age) providing the managerial/fiduciary discipline (think Steve Jobs and Elon Musk as the epitome of this) then the challenges of running a successful business will quickly outpace the efforts of the talented engineers working there.

The number of teams/engineers that unilaterally work on overlapping projects or projects that don't serve any purpose can rapidly drown the needed forward progress.

</comment></comment></comment></comment></comment></comment><comment><USER13> I don't get it, is this hiring like a PR stunt?

I would thinking creating your own maps and autonomous driving platforms is already fairly ambitious, but flying cars?

</comment><comment><USER14> Rant: Since when did VTOL == 'flying car'? A flying car is a hybrid, dual-use vehicle. Is a helicopter a flying car? A Harrier jet? VTOL would be great for an actual flying car and it would be great for small, personal aircraft, but that doesn't make an aircraft that has it a flying car.

<comment><USER10> Absolutely. We need a new word for this new category of aircraft (like the eHang, Volocopter, etc.), but "flying car" it is not.

I'm in favour of Personal Electric VTOL Aircraft, PEVA.

</comment></comment><comment><USER15> This could have equally been an article on the Onion

<comment><USER16> They are gonna hire this guy next to research Compost Fueled Cars: [https://youtu.be/DkGMY63FF3Q](https://youtu.be/DkGMY63FF3Q)

<comment><USER15> Actually I remember an article from the local paper here (on Kaua'i) where someone on the island had made just that.. though I think the compost was making biofuel of some sort

</comment></comment></comment><comment><USER17> Flying cars of the 2010s seem striking similar to solar cars for the 1980s. A few expensive prototypes here and there, some great pontificating from leading engineers, but no serious consumer model.

<comment><USER18> Depends... I think we are really close to have viable "flying cars" in the form of electric or even hybrid gas-electric multirotors... Give it 5-10 years Im sure we will begin to see multirotors being used for personal transportation.

<comment><USER10> Predicated on advances in battery tech. The rest is there, agreed.

</comment></comment></comment><comment><USER19> Flying cars aren't the future. I think it's much more in Jetpacks. Once power density is better than Jet-A, we'll be in business for some great options.

Probably dangerous as all get out, but that's part of the fun.

[https://medium.com/@6StringMerc/for-christmas-2016-i-give- th...](https://medium.com/@6StringMerc/for-christmas-2016-i-give-the-world- the-parawing-fcc63fb3c20d#.w6iatgr2j)

</comment><comment><USER20> Interesting, I didn't realize Mark had resigned from LaRC. I'd be interested to hear a more complete take on his opinions about NASA Aeronautics, because he sounded a bit disillusioned.

The big caveat when comparing to industry efforts, of course, is that NASA Aero, by design, is not an airline, or an airframe manufacturer, or a regulator. It's an R&D outfit that still very much follows the structure of its predecessor, NACA.

</comment><comment><USER21> And this is why giving cruddy companies insane amounts of cash works out well for humankind. Few sane companies would have been able to put significant resources behind something like this without a solid lead in hand.

</comment><comment><USER22> I never understood the point of flying cars. Do we not already have those but call them airplanes?

Flying cars seem infeasible to operate for the mass consumer as they (like airplanes) require extensive training to operate safely.

<comment><USER23> The idea is that they are self-driving flying cars, so consumers don't need training.

While the physics of flight are more complex than driving, the environment is far less complex (straight-line between points, few obstacles along the way). Flight is better suited to automation, and in fact we already have very reliable autopilots and computer-enhanced airplanes.

That said, I think the fuel costs of flying mean it will never be competitive with driving, based on cost.

<comment><USER22> I still do not understand the difference between a flying car and an air plane.

<comment><USER10> Terrible name, indeed.

Flying car sort of implies:

* can fly and drive on the road.

While there are concepts along those lines (e.g. the Terrafugia Transformation), that's not what this is about.

What distinguishes this new bred of aircraft is:

* VTOL, so not bound to a traditional airport

* simple control ("like a car"), so don't need a pilot. That distinguishes them from a helicopter, which is harder to fly than a fixed wing plane.

* maybe also autonomous flying and collision avoidance, so more like next-gen cars.

I find PEVA (personal electric VTOL aircraft) more descriptive.

</comment></comment></comment></comment><comment><USER18> Can we all please agree that if something is not designed for road use, it can hardly be categorized as a car?

Im all for stuff like this, but "flying cars" implies something that is practical as both a road vehicle and a flying contraption of sorts...</comment>