Migrating to CockroachDB - <AUTHOR> https://www.openmymind.net/Migrating-To-CockroachDB/ <comment><USER2> > if you want password-authentication, you must configure your cluster to > communicate using certificates. You can start each node in "insecure" mode, > but then can't use password authentication. There's a issue about this. Like > the people posting there, our network is already secured, so this > requirement is just an unnecessary nuisance.

Eh, I'm not sure how I feel about this. Obviously you should secure your network as best you can, but how confident are you really that a bad actor will _never_ find a foothold anywhere in your network? I think I would advocate for your services to all communicate securely (TLS et al), even internally, and if your database supports mutual client/server auth like it sounds like Cockroach does then you should use that as well. Particularly if you're depending on at-rest encryption handled transparently by the DBMS to protect your users' data - that won't do you much good if someone can just sniff/MITM network traffic and wait until a bunch of your data has been queried.

<comment><USER3> See, I do infrastructure work in companies that often don't even expose their services. They just want a database and a frontend for a very small internal app that will never get out. And yet, a lot of the tools are designed "for the web" and, rightfully so, enforce https, ssl, certs, you name it.

Now the issue is that they are a real PITA to implement in some of those networks, where you have to use tons of private registries and repositories, and to get everything working with certificates. Because those companies are not used to issue certs, often times it take them months to do so, as they have a very rigorous process, so everyone wonders why this tiny app never goes to "production" and why we are blocked by such restrictions.

Now I'm not even accounting for the fact that you sometimes actually need to go online to install some of those stuff. Recent example was node-sass, dependancy from a composer package, which you would expect to just install from the composer registry ; but no, it has a setup script that goes to nodejs.org or something, which of course you cannot fake because it will have to check certs as well. So again, something designed for the web that'll never work seemlessly elsewhere. (There are workaround of course but I'd love to spend my time elsewhere).

To give you an idea, the cloud I'm working on right now is pretty much entirely offline, for both egress and ingress. There is a nexus mirror-proxy to dockerhub and that's it, even this is in the LAN. So really, having to configure cockroachdb certs (or anything else for that matter, like etcd...) is useless, time consuming, frustrating, and counterproductive.

<comment><USER4> > very small internal app

Is cockroachdb the right DB for this use case, though? A more traditional Postgres/MySQL or evening sqlite (depending on needs) is often plenty good for a small internal app without requiring this extra security

> the cloud I'm working on right now is pretty much entirely offline

I fee like "cloud" has lost all meaning it may have once had

<comment><USER3> > Is cockroachdb the right DB for this use case, though? A more traditional > Postgres/MySQL or evening sqlite (depending on needs) is often plenty good > for a small internal app without requiring this extra security

It's not, but the app I described is just an example. Some use cases involve bigger dataset and resilience.

It's not cockroachdb specifically but any of those "web" tools that, used internally in a closed environment, become a PITA to manage. Language tooling, repositories, dbs... Etcd for example is used as an internal storage for tenants (for example to store DNS information) and the team that manages it cannot upgrade right now because they have to figure out first how to implement ssl.

> I fee like "cloud" has lost all meaning it may have once had

Well, how would you call an on-prem install of OpenStack ? It's their internal cloud. Cloud has not much to do with the web or with the fact that it is publicly exposed, it's a tool to manage infrastructure. And to be perfectly honest, "cloud" isn't a proper term to begin with. Years ago when the word came out one of my professor said they basically renamed "grid" to "cluster" then "cloud". They used to administer hundreds+ of machines and multiples of that of jobs using pssh. They didn't wait for kube, and they didn't need ssl between each and every API and services.

Now don't get me wrong, the public cloud provides a great service and tooling coming out now should be web ready and web safe. Just, please, pretty please, give us a --insecure and --offline even if it means having it all over my ansible code, so we can get the job done without having to spend my days working around those tools.

<comment><USER5> > Just, please, pretty please, give us a --insecure

Have you taken a look at [0]? (All caveats around running an insecure cluster apply)

[0]: [https://www.cockroachlabs.com/docs/stable/deploy- cockroachdb...](https://www.cockroachlabs.com/docs/stable/deploy-cockroachdb- on-premises-insecure.html)

</comment></comment></comment></comment></comment><comment><USER6> The lack of useable backups in the open-source version is confounding to me. They are effectively limiting adoption of both their free and paid version.

If they would have proper backups in free, more people would start using it, and then discover that they'll need some of the premium features (data locality, Role-Based Access Control or follower reads).

Also, why is there no public pricing page? How will I know that pricing won't change every year? Or be arbitrary based on how much they can squeeze us? (less likely with public, listed prices)

Why would a company with low marginal cost (software) limit their growth like this?

<comment><USER7> This is a problem with Postgres as well (for different reasons). I currently have a server sitting without proper DB backups (only full backups and dumps) because figuring out how Wall-E (or whatever the latest tool is) works is more effort than "here's my bucket and encryption key, please start backing everything up".

<comment><USER8> WAL archiving is really helpful for (any) point in time restores and backing up large databases. If neither of those apply, pgdump is great!

Also WAL-E has been replaced by WAL-G (written in Go):

[https://github.com/wal-g/wal-g](https://github.com/wal-g/wal-g)

</comment></comment></comment><comment><USER9> I'm glad for the honest writeup of Cockroach. I like CockroachDB, and hope they continue it in the future, but the performance and backup capabilities are something to consider. Still, I'd rather deploy it to Kubernetes whenever I can over Postgres to ensure that I can do rolling upgrades and whatever else. At least the Helm chart they provide lets you use Cert Manager with it, so the TLS isn't as much of a pain.

</comment><comment><USER10> > But going from our single PostgreSQL instance to a 3 node cluster, > performance is worse. This is true even for simple queries involving 1 node > (say, getting a record by id or running locally with a single node). Still a > few extra milliseconds of latency is a fair price for better availability. > But the performance characteristics aren't the same as a relational > database.

I think this should be true to any distributed database: you can't really beat an optimized single-machine software with a networked app, no matter how hard you try.

My experience with CockroachDB showed that increasing the cluster size from 3 to 6 and more provided more benefit than I've expected. 1 -> 3 seems to be a meaningless metric for such setup with a replication factor of 3.

</comment><comment><USER11> There is no free lunch with databases. I often argue the position that master- master is overrated, because you have to design your application around the constraints to see the benefits and avoid the pitfalls.

Same thing with CockroachDB Postgresql "wire protocol support". The fact that it only supports "serializable" isolation level when Postgresql utilizes Read Committed OTB should be a screaming red flag to just about anyone that you are in for some challenges on the performance front. Some well-worn modeling techniques are simply infeasible under typical workloads at that isolation level, for one.

Starting with bog-standard RDBMS you have to be deliberate to get great performance. Adding in extra, fancy scaling properties means even more considerations to take into account to maintain consistency while achieving gains.

</comment><comment><USER12> High-availability is always an issue that I am concerned with self-hosting MySQL or PostgreSQL. I am always worry about the fail-over failing, in case any step in the setup was not properly configured. Let me know if someone has some magical formula for easy replication.

I still find CRDB's replication interesting, since all data is replicated 3-way on every commit, and data also auto-rebalances as needed. With some exceptions, as long as you are connected to any node, you should have access to all data.

<comment><USER13> We used Percona's XtraDB Cluster for HA and have had very few issues.

Just fronting it with haproxy to read/write to a single node at a time (setting others to "backup") has worked well for us, since we're not doing a huge number of queries.

<comment><USER14> I concur, but consider ProxySQL instead of HAProxy. Percona with XtraDB/Galera failover fronted by ProxySQL is very resilient and performant.

</comment></comment></comment><comment><USER15> I've tried CockroachDB as well as YugabyteDB. A distributed ACID transaction Database sounds like the holy grail, but it comes with a price. For CockroachDB the performance penalty was high, even though I'm sure I could have squeezed out more performance if I was to design my DDL specifically for CRDB.

There was an inconsistency issue when deleting a ton of (self referencing foreign key) rows in rapid succession. The results of a "select count(*) from table" returned different results than a "select count(id) from table". After some delay the results came back in sync again.

As I was trying to shoehorn an existing PostgreSQL database into CRDB, I had to find solutions for SELECT .. FOR UPDATE, finding alternatives for triggers, issues with the query builder I use (knex.js). In the end it was too much, not even considering lack of backups and the risk of a VC funded company going belly up. I do hope they gain a solid foothold and these NewSQL databases gain more traction.

<comment><USER5> We take our claims if correctness very seriously. The behavior you're describing, depending on exactly what you're doing, is likely an allowed serializable history, which is the isolation level we claim to support. See [0] for a deeper dive on the subject. Either way, you should show us what you found (if you haven't already).

As for your performance analysis, do share your results and methodology. We published an in-depth, reproducible comparison with YugaByte here [1] in addition to publishing our TPCC-100k numbers[2]. If you're seeing performance that doesn't line up with the above, let us know.

[0]: [https://www.cockroachlabs.com/blog/consistency- model/](https://www.cockroachlabs.com/blog/consistency-model/)

[1]: [https://www.cockroachlabs.com/blog/unpacking-competitive- ben...](https://www.cockroachlabs.com/blog/unpacking-competitive-benchmarks/)

[2]: [https://www.cockroachlabs.com/blog/tpcc-100k/](https://www.cockroachlabs.com/blog/tpcc-100k/)

<comment><USER15> Thanks, "serializable history" could well have been the root cause. I did discuss this in the forum but was unable to resolve it at that time.

</comment></comment></comment><comment><USER16> We use Elixir and Postgres at work, I didn't realize cockroachdb didn't have geospatial features, unfortunately that makes it a non-starter for us. We need it to track pathing from mobile phones (ios and android).

Thank you for writing this up and putting your experiences out there, clear as day. It's very valuable.

</comment><comment><USER17> What a world of difference good tests and coverage can make. Great write up.

</comment><comment><USER18> What new class of RDMS?

Cluster scalability has always been a thing with Oracle, SQL Server, Sybase, Informix, DB2.

<comment><USER19> Scalability is relative. To some people a 10TB database is large, but some teams I work with regularly see databases in the 10PB range. Going to that kind of scale you tend to see different design tradeoffs.

<comment><USER18> Those kind of dataloads were already typical 15 years ago in the telecommunication domain, for example Nokia NetAct report processing for the whole operator network, based on HP-UX with Oracle OLAP support.

Or the CERN infrastructure to deal with non-stop handling from accelerator data and storage for posterior data analysis.

Sometimes this industry feels like a continuous reboot.

<comment><USER20> I'm not sure how OLAP relates to CockroachDB. The use case for CockroachDB is quite simple. You have millions of applications that runs Postgres. A large chuck of them have to process more and more data which a single master cannot handle and there is no easy way of scaling. Any option you pick does require a lot of changes to the codebase. This is were CockroachDB should come in handy. The dream is to have a drop-in replacement for Postgres that would scale, so all SELECT .. JOIN still works.

<comment><USER18> Because that specific example made use of Oracle distributed cluster scalability, already 15 years ago.

Postgres still has a couple of tricks to learn from big boys databases.

</comment></comment></comment></comment></comment><comment><USER21> Does CockroachDB work with Ecto in Elixir? Can it be used as a drop-in replacement for PostgreSQL?

<comment><AUTHOR> The Postgrex driver works with cockroach as-is, and they're open to making tweaks as necessary (I made a small patch to Postgrex to support the way cockroachdb encoded empty arrays and it was quickly accepted).

I don't think Ecto works as-is, you'd have to check.

For simple apps, it might be a drop-in replacement, or require very few tweaks. In addition to the issues mentioned in the OP, the cockroachdb documentation has a detailed list of differences (1).

(1) [https://www.cockroachlabs.com/docs/stable/detailed-sql- suppo...](https://www.cockroachlabs.com/docs/stable/detailed-sql-support.html)

<comment><USER22> Would you mind providing a link to the empty arrays patch? Is this something CRDB should fix?

<comment><AUTHOR> It was fixed in CRDB also: [https://github.com/cockroachdb/cockroach/issues/42942](https://github.com/cockroachdb/cockroach/issues/42942)

</comment></comment></comment></comment><comment><USER23> Anyone have experience with rqlite and how it compares to cockroachDB?

</comment><comment><USER20> Has anyone tried to run CockroachDB with GCP TrueTime?

<comment><USER5> TrueTime is not a publicly available GCP service.

</comment></comment><comment><USER24> Seems like overkill, and just a shiny new toy to play with for this use case. Stackoverflow.com uses SQL. You aren't exceeding stackoverflow. In a couple of years they'll migrate back, or to something else.

The whole point of cdb is to enable things like Google scale. But all I ever see on it are random devs spinning up three nodes and being so proud.

Which is too bad because I'd love to see someone pushing its limits, and betting mission critical services on it that nothing else could handle.

<comment><AUTHOR> Sorry if I wasn't clear, but the goal of the migration wasn't for scale, it was for HA.

I find it difficult to setup PostgreSQL without having a single point of failure and introducing downtime for things such as upgrades.

We aren't doing life changing stuff, true, but a downtime in our system can result in employees (often at the bottom of the economic ladder) not being able to get to work. That gnaws at me. I guess the better alternative is to go for a hosted approach, but that isn't without its own complications and challenges.

<comment><USER25> Out of curiosity what do you feel so hard to set up with PostgreSQL?

Having a master with a read replica and then making the read replica the master in case the master goes down seems to be a very well known methodology. You can use a virtual ip with both servers behind it (keepalived) or a 0 ttl DNS solution (consul). Is you case more complex than that? With master to master replication I feel there are couple of gotchas and you need to design your data in a way to avoid conflict as much as you can. A solution like bdr seems to be an off-the-shelf one that is proven.

I haven't really upgraded postgres in a rapid fashion, usually just keep with whatever version comes with the OS package manager forever.. or whatever docker image I start it with. I guess the same approach for HA can be used to upgrade one at a time, am assuming postgres doesn't break backwards compatibility very often.

You can either automate the failover or have a system to SMS you when it happens. I like to do it manually personally since I have seen the automated failover fail more times than the manual procedure.

I have probably managed over 30 postgres instances in my services that have grown up to 1/2 PB of data and I've never had a single issue with it... postgres, redis, rabbitmq, consul are technologies that you RTFM, set it up, and it just works!

I wouldn't trust a technology that isn't established and widely adopted which is how I see cockroachdb a the moment.

<comment><USER26> >Having a master with a read replica and then making the read replica the master in case the master goes down seems to be a very well known methodology. You can use a virtual ip with both servers behind it (keepalived) or a 0 ttl DNS solution (consul).

>You can either automate the failover or have a system to SMS you when it happens.

I don't want to be writing code to make things do automatic failover. I'm not a database engineer. I want a database where its standard setup does this stuff for me.

(I would find it really interesting if there was a tightly-integrated "distribution" of Postgresql plus related services that had a setup process as straight-forward as CockroachDB and had automatic read-replicas and failover.)

<comment><USER25> You usually don't have to write stuff it is all pretty much yum install and config a file

</comment></comment></comment></comment></comment><comment><USER27> I'm sorry but the name of the database needs to change. It's like if I named my company "Disgusting Bug Company" then expected everyone to ignore the fact I named my company after a gross creature. Call it "RockSolidDB" and you get the same point across without invoking gross bugs.

<comment><USER28> I used to agree, but the name actually seems to have an ironic marketing benefit: Everyone ascribes Cockroach's success to its technical merit. I've never heard Cockroach criticized as another dumb web dev trend caused by marketing and fads.

<comment><USER29> I propose that they further strengthen this effect by dropping the word "roach" from their name, and rebrand as just "Cock DB". /s</comment></comment></comment>


Jupiter has 10 more moons we didn't know about - <AUTHOR> https://www.nature.com/articles/d41586-018-05725-6 <comment><USER2> Is it time to have a Pluto moment for moons? In the end, all such labels are arbitrary, but an calling an object of 2 or 3 km radius a moon feels wrong to me, somehow.

Plus, without a lower boundary, pretty much every single chunk of ice in Saturn's rings would count as a moon, so having a formalized definition might come in handy. ;-)

<comment><USER3> well, I certainly think pluto is more of a planet than these are moons

<comment><USER4> I think they mean to make a formal definition of moon that would exclude some of the new Jupiter moons, similar to the new formal definition of planet excluding Pluto from being a planet.

</comment></comment></comment><comment> cup-of-tea I wonder if discovering lots of tiny moons will lead to a new definition for moon like what happened with the planets.

<comment><USER5> Yes, perhaps these will be reclassified as satellites and moons will have to be large enough to form spheres under their own gravity. That would mean that Mars would have no moons though.

<comment><USER6> Can you expand a bit on why a satellite would need the planet's gravity to keep a spherical shape ?

<comment><USER7> No, we're talking about a satellite forming into a sphere under its _own_ gravity, not the planet's. Given a large enough lump of matter, gravity will overcome structural strength and force the material to flow into a roughly spherical shape. On Earth, this happens with the maximum mountain height being a few kilometers, but on Mars' moons, which are much smaller, the gravity is far too weak to overcome the rock's structure, and they are basically strange shaped rock things.

<comment><USER8> Given enough time [1], all solid objects become perfect spheres, no matter their size.

[1] 10^65 [https://en.wikipedia.org/wiki/Timeline_of_the_far_future](https://en.wikipedia.org/wiki/Timeline_of_the_far_future)

<comment><USER9> I am comfortable with a definition of "moon" that does not involve extrapolating 10^65 years into the future.

<comment><USER10> Well, then you'd need to define an arbitrary time limit for how long it would take to form a sphere. In which case, you might as well have just defined a size/mass limit in the first place.

<comment><USER11> I think the definition would be that they're presently a sphere, that was formed under its gravity. That way objects could possibly become moons in the future, but the timescales are so long it's mostly irrelevant

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> The eighth moon sounds exciting... "it travels in the same region of space as the retrograde moons, but in the opposite direction... That means it could easily smash into the retrograde moons, pulverising itself into oblivion."

Setting up science nerds for thousands of years of suspense!

<comment><USER13> Wouldn't it be a spooky experience to stand on one moon, and have another pass close enough for you to touch it? If they were of similar mass my intuition is that you'd go weightless while it was happening. Might be best to be harnessed during something like that.

While I'm thinking about spooky space stories. Jupiter creeps me right the hell out. Thinking about being on one of its moons where Jupiter takes up a HUGE percentage of your visible sky and is just screaming radiation at you. Personify the gas giant a little and you've got a decent proxy for hell. [https://solarsystem.nasa.gov/resources/184/simulated-view- fr...](https://solarsystem.nasa.gov/resources/184/simulated-view-from-europas- surface-artists-concept/?category=planets_jupiter)

<comment><USER12> That would be amazing, it looks like Europa travels at 14 km/s or 31,000 mph. With no atmosphere there would be no sound or pressure wave. And the bodies are probably small enough to not shatter because of the gravity interaction. Escape velocity is about 36,000 mph. So you could see a huge Manhattan size chunk of rock flying at you at rocket speeds. Would be amazing to see it pass by 100 feet above you.

<comment><USER14> > Would be amazing to see it pass by 100 feet above you.

At those speeds, and the necessary angle, I'm not convinced you would be able to see it at all.

The actual encounter would last for too short of a time to see (like trying to see a bullet in flight), and their motion would take them out of your field of view, unless the planet you were on was rotating very fast.

<comment><USER12> Cool question/objection.

The two moons are going in opposite directions, so the speed guess is really 28 km/s.

For a full second it would be 2km across and 14km or less away (.5 seconds before and .5 seconds after it's overhead). That's 16 degrees of visual angle. The moon is .5 degrees of visual angle, so you get a full second of something 32 times bigger than our moon.

It would be moon sized about 230 km away, which is over 8 seconds away. So you'd get to see a moon grow quite large over 8 seconds.

So you would see it, but would it be awesome?

Lets say our eyes can see things in the 100 ms range (really a bit better). For that 100 ms, the object would be < 1.4km from you and subtend fully 71 degrees of your vision. That's about 2 basketballs width at arms length.

Which sounds cool though maybe not awesome. But hey you are looking at a moon rushing at your face like a basketball, I would totally go.

[https://wolfe4e.sinauer.com/wa03.01.html#Calculator](https://wolfe4e.sinauer.com/wa03.01.html#Calculator)

<comment><USER14> You have to check the angles. Think of the X I posted earlier. Imagine your goal is to stand at the point looking "up" (because that's were they will meet), both moons were traveling bottom to top, and you were on the moon that started bottom left. (So your gaze is at an angle to direction of travel.)

For most of the time the moon will be behind you to your right out of sight. After the interaction, if the other moon was larger than yours, you might be able to see it.

But if it was smaller, then your own moon's horizon would block your view of it.

But it's worse than that - even if you could see it, you would have to turn your head to the left to see it, but it's moving so fast you just don't have enough time to physically move your head before it's far far away.

i.e. It would whiz by _across_ your field of view, not toward it. Which makes it really hard to see.

Edit: I'm tying to visualize how it would look, but I may have missed some detail. It would be cool if someone could program a visualization of this.

<comment><USER12> In the spirit of "could this be awesome" rather than is it likely...

So pretend we stand on the side of the moon facing toward Jupiter so that Jupiter is straight overhead. And pretend the other moon will pass just inside our orbit 100 ft above our head.

Imagine 2 quarters overlapping, representing the orbits. Pushing the top one up a bit keeps the inner moon completely above the horizon for the whole approach. You can also push the top quarter to the left a smidge so that the inner moon orbit is coming from high and to the right. This would be a moon sized object popping up high and right in the sky and then disappearing in a blink below the horizon to your left.

Speed isn't an issue because for the 8 full seconds before it's right overhead it will be our moon sized and bigger. Up to a 2 "arm's length basketballs" when right overhead. You could definitely track the moon across the half the sky over 8 seconds.

<comment><USER13> I was thinking a bit more on this today. Specifically If a moon did this past earth and assuming it was made out of something that wouldn't just break apart in our atmosphere.

The amount compression on the air around you might set you on fire.

That would be interesting.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Other recent articles I saw about this said twelve, this one says ten. Anyone know why?

[https://www.npr.org/2018/07/17/629396121/galileo-would-be- st...](https://www.npr.org/2018/07/17/629396121/galileo-would-be-stunned- jupiter-now-has-79-moons)

If you look at Wikipedia's list and sort by year of discovery, 10 would include the ones in 2018 and 2017, while 12 includes two more in 2016. All 12 are discovered by Sheppard et al. Perhaps those two aren't news anymore?

<comment><USER16> Right, they are talking about roughly the same discoveries but some include 2 more: [https://astronomy.stackexchange.com/questions/26999/did- we-d...](https://astronomy.stackexchange.com/questions/26999/did-we- discover-10-or-12-new-moons-of-jupiter)

</comment></comment><comment><USER17> Title should read "At least"

A planet as massive as Jupiter will have captured lots(1) of objects.

1\. Terry Pratchett Troll counting system.

<comment><USER18> I'm gonna paste this in because pterry was a god (small G), and there should be no point ever where a reference to him shall go unnoticed.

 _'Everyone knows trolls can't even count up to four!'

(In fact, trolls traditionally count like this: one, two, three, many, and people assume this means they can have no grasp of higher numbers. They don't realise that many can BE a number. As in: one, two, three, many, many-one, many-two, many-three, many many, many-many-one, many-many-two, many-many- three, many many many, many-many-many-one, many-many-many-two, many-many- three, LOTS.)_

<comment><USER19> Presumably because trolls only have 4 fingers on each hand?

<comment><USER20> There are finger-based counting systems with more than one number per finger. Our average of 10 fingers can handle a lot more than 10 numbers. [https://en.wikipedia.org/wiki/Finger- counting](https://en.wikipedia.org/wiki/Finger-counting)

<comment><USER21> And less than a digit per err digit (thumbs included)

If I recall there was a Native American base 8 counting system which utilized the space between fingers holding sticks or strings as place holders.

Made for a handy octal abacus.

</comment></comment></comment></comment></comment><comment><USER22> This reminds me of the discussion(s) on BBC QI (Quite Interesting) about 'how many moons does the earth have'.

A video from QI official channel: [https://www.youtube.com/watch?reload=9&v=7qZWM8Aatb8](https://www.youtube.com/watch?reload=9&v=7qZWM8Aatb8)

And a compilation of updates over the years. It has Stephen Fry discussing the topic but in recent seasons Sandy Toksvig discussed the topic as well : [https://www.youtube.com/watch?v=sGgmmX- dzgU](https://www.youtube.com/watch?v=sGgmmX-dzgU)

</comment><comment><USER23> Awesome, new Eclipse IDE versions!

</comment><comment><USER24> For reference: [https://en.wikipedia.org/wiki/Moons_of_Jupiter](https://en.wikipedia.org/wiki/Moons_of_Jupiter)

Its gravitational field is so huge that it is basically the vacuum cleaner of the solar system. Also a lot of things that are not in orbit of it are occasionally significantly perturbed in their solar orbits, by Jupiter, such as a number of long period comets.

<comment><USER25> How exactly does that work though?

An orbit can't _just_ change from a parabola into an ellipse, without the influence of external forces. Is something slowing down these object so that Jupiter "catches" them? (Rings?)

<comment><USER26> You're right that in a two body system this is totally impossible. But for a three body system of Jupiter, the Sun, and our new moon then ballistic capture will _baaaasically_ get you into orbit and then it only takes a small further interaction with another planet, another moon, or whatever to get into orbit the rest of the way.

<comment><USER14> > the Sun

Jupiter is in a stable orbit about the Sun, I don't think you can use the Sun as the 3rd body, you need another moon or other object.

<comment><USER26> The important thing, as far as I understand it, is the way the future moon slowly transitions from being primarily under the gravitational influence of the Sun to primarily under the orbital influence of Jupiter.

We've actually used ballistic transport to transition probes from orbit around the Earth to basically in orbit around the Moon with just a tiny braking burn at the far side instead of the serious burn that a Holman Transfer orbit would require. And the Moon is in a stable orbit around the Earth and doesn't have any further satellites.

<comment><USER14> > slowly transitions

There is no slowly for capturing an incoming object - you get one chance, unless that object is in a similar orbit to your (and in that case the Sun doesn't help).

> with just a tiny braking burn

Tiny is relative here. It's still enough of a burn to significantly change the momentum of the rocket. Moons don't have anything with that kind of energy, except other moons.

<comment><USER26> No, slow is a fair description of this. You need to start out in nearly the same orbit as the target and while you're both traveling around the Sun or Earth fairly quickly you approach the target much more slowly than you would in a Holman Transfer. Then when you're captured you'll be going around the target a number of times before other gravitational perturbances eject you again, giving a long time to do a burn to get into a stable orbit. So yes you have to change the momentum of the rocket but the burn could take place over the course of a week and won't have to be very large. Your burns to achieve an intersect giving you a ballistic capture would typically be much larger than with a Holman transfer but the slowness of the final approach means you're free to use an ion drive, solar sail, or other high efficiency and low thrust drive.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> "All the newfound moons are small, between about 1 and 3 kilometres across. Seven of them travel in remote orbits more than 20 million kilometres away from Jupiter"

As a layman with no astronomical credibilty I'd call those satellites, not moons.

<comment><USER28> Natural satellite == moon.

"Natural satellite" is a subset of "satellite", thus what you'd call them is correct by definition. Let's let the astronomers with credibility play and name the objects.

<comment><USER29> So does Saturn have billions of moons?

<comment><USER30> Saturn's rings are a bit more special. They are actually only 10-100m thick. There's still a difference (however small it may be) between those and kilometer sized rocks.

</comment></comment></comment></comment><comment><USER24> Fun fact, asteroids in the belt can also have natural satellites:

[https://en.m.wikipedia.org/wiki/87_Sylvia](https://en.m.wikipedia.org/wiki/87_Sylvia)

</comment><comment><USER31> Incredible.

</comment><comment><USER32> There goes the neighborhood.

</comment><comment><USER33> This is a great example of the hubris of scientists that non-scientists can't stand.

Most believe Pluto is a planet. Moons are rocks in orbit around planets, etc.

Why the _obsession_ to change these names? Brings painful thoughts of our future if this is what they care about.

<comment><USER26> It's in the nature of science that sometimes things that were previously believed to be true turn out to be false. The notion that there are nine planets matched what astronomers knew for many decades but with better measurements turned out be be an unsupportable idea.

For centuries there were just 6 planets. Then people discovered Uranus and there were 7. Then people discovered Ceres and there were 8. Then people discovered lots of other main belt asteroids and there were 7 again. Then Neptune and 8 again. Then Pluto and 9. And now we've found all these other big bodies in the outer system and we decided to demote Pluto rather than add a half dozen new planets, just like we did with Ceres.

I don't think anybody particularly objected to the demotion of Ceres since it hadn't been around that long when it happened. But I can only imagine the furor that must have arisen when people tried to claim that Uranus was a planet despite the number of planets being fixed since the time of Homer. Well, Ok, there was the bit when they turned Earth into a planet and we all know about the people like Giordano Bruno literally burned at the stake over that.

So this demotion of Pluto is just another event in the long, long history of scientists revising what counts as a planet. I understand that it's uncomfortable to have to unlearn things you learned as a kid but science is all about changing our view of the world.

<comment><USER34> A great { essay / blog post } about this constant churn:

\- [Ontological remodeling | Meaningness]([https://meaningness.com/eggplant/remodeling](https://meaningness.com/eggplant/remodeling))

My favorite bit:

> Jupiter doesn't go around the sun, and therefore is not a planet by the 2006 > definition.

> Don't believe me? In Newtonian mechanics, two bodies orbit their barycenter, > or center of mass. If they have equal masses, the barycenter is the midpoint > between them. If one is heavier than the other, the barycenter is closer to > it. If one has much greater mass than the other, their common barycenter is > located within the larger body, and the smaller object goes around that > point. Only then is the smaller body said to orbit the larger one. > Otherwise, the two form a binary system.

> Jupiter is ludicrously heavy: it has 2.5 times the mass of everything else > in the solar system combined, apart from the sun. The sun is much heavier > still—but the barycenter of their mutual orbit is outside it. Jupiter and > the sun are a binary system. Their barycenter is, to be fair, quite close to > the sun, and informally it may be reasonable to say Jupiter goes around it. > But in terms of the formal definition, it doesn't, so by the IAU criteria, > Jupiter is not a planet.

<comment><USER26> I'm not sure why you would say that A isn't orbiting B if the barycenter of the orbit isn't inside B? We normally talk about Jupiter orbiting the Sun or Charon orbiting Pluto despite this not being true. I can see why someone might want that to be the definition for aesthetic reasons but I'm not seeing any evidence that it's the accepted definition. Wikipedia certainly says "near or within" instead of "within" in the most relevant section and I couldn't find anything clearer or contradicting that.

<comment><USER34> > I'm not sure why you would say that A isn't orbiting B if the barycenter of > the orbit isn't inside B?

If A and B were roughly the same mass then you'd say they're orbiting each other. We _don 't_ tho say or write that about Jupiter in my experience.</comment></comment></comment></comment></comment>


Twitter and Facebook Both Quietly Kill RSS - <AUTHOR> http://www.staynalive.com/2011/05/twitter-and-facebook-both-quietly-kill.html?q=1

<comment><USER2> The title is a bit of a double entendre. At first reading, I thought the claim was that (once again), Organization X had killed RSS (figuratively). As it turns out, Facebook and Twitter have, quite literally, killed RSS within the scope of their own systems by removing it from key locations.

In my view, RSS was never the right tool for this application anyway. Social networks are naturally conversational in nature, but you're only a part of the conversation while you're there.

In the economy of my attention, both Facebook and Twitter occupy a different position than the RSS feeds I follow. Facebook and Twitter are there for when I want them, but my RSS feed is the kind of thing where I try to at least review every headline. I don't want to miss anything in my RSS reader, but I'm perfectly content to miss out on large portions of what occurs on Facebook and Twitter.

Twitter is probably closer in function to RSS -- for me, anyway -- than Facebook, in that I rely on it for "pushed" information that I rarely respond to (think service outages and status updates). This could, theoretically, have been fulfilled by RSS, but for some reason, I never made use of RSS that way.

I think one of the reasons for this (Twitter as a popular info-push medium) is that you _can_ respond easily through Twitter, even if I only do it on rare occasions. Status blogs provided an avenue for response, but you often had to have an account on every blog... Ugh. Twitter is also great because you don't have to dig through the service website; you simply search Twitter for their company name and follow.

Facebook, on the other hand, is purely entertainment for me. I use it to keep up with friends and converse in a time-shifted manner. I cannot imagine collecting RSS feeds from Facebook streams.

Based my usage patterns, I don't see this as "killing" RSS for me in a general sense. Maybe I'm a dying breed though.

<comment><USER3> A common "general" application of RSS is news, but this will hopefully be slowly phased out as well. The New York Times offers an awesome API (<http://developer.nytimes.com/>) which gives you a much better view of their data than their RSS feeds (which they still provide).

<comment><USER4> So I'm supposed to use the NYT's custom API to get articles when RSS has, for years, provided a "general" method for me to do the same thing for not only the NYT but millions of other sources?

<comment><USER5> Yes, and they want you to install a special 'NYT app' for that, which will be future: every company and news outlet will have its own app.

<comment><USER6> haha, and they'll all be based on frontend UI accessing RSS.

<comment><USER5> I was thinking more along the lines of 'accessing their own specific API (maybe even a private, undisclosed API)'.

</comment></comment></comment></comment></comment></comment><comment><USER7> I have no interest in ever directly using Twitter and there's not enough content on there for me to bother with any workarounds. I currently use RSS for the one Twitter feed I bother to follow and if they ever really do get rid of RSS for their site, I'll just stop using even that. Not that much of a loss.

My habits are such that if you don't offer RSS feeds of information I want, I probably won't ever get your information _or_ go to your site. You're not going to sucker me into "visiting regularly", just because you don't offer an RSS feed. I'll just move on and get what I want somewhere else.

<comment><USER3> I don't think getting you to visit their site is the point of the switch. Twitter and FB are in a position where they have enough addicted users that they can get away with messing with standards. RSS is outdated and slow, and FB and Twitter want people to use their data in a more sensible way, via their flexible APIs.

It's trivial to build an APP that generates an RSS stream for an arbitrary twitter account with their API, and I'm sure someone will soon. But the point is that they are setting a new standard for what can be done streams of structured data, and hopefully more sites will move away from RSS into cool APIs.

Of course, for now at least, there's no standard to this kind of API, but I think that people will eventually figure out what works and converge to it (as is the case with OAuth).

<comment><USER7> I hear people talk about how "RSS is so slow" all the time, but I still don't understand it. I see no failings or complaints about RSS. I get a headline and then I get an article. Are we talking speed between when content was generated and when it appears in an RSS feed? Honestly, I couldn't care less about that. I'll live if it takes fifteen minutes for an update to reach me. It's not like I'm getting a patient's live EKG readings via RSS.

As for yet another standardized API to replace it. As long as it provides the same functionality to all current services/devices that use RSS, that's fine with me. I would _really_ prefer that it not be driven by Facebook and Twitter, however any more than I want Microsoft Office development teams driving the standardization of ODF.

<comment><USER3> If you read about FriendFeed's Simple Update Protocol ([http://blog.friendfeed.com/2008/08/simple-update-protocol- fe...](http://blog.friendfeed.com/2008/08/simple-update-protocol-fetch- updates.html)), you'll find some good arguments about why RSS slow. The crux of the problem is that to get _any_ updates on a feed, you have to request and download the _entire_ RSS feed (which is filled with summary text and is of nontrivial size).

Consider the case of a real-time news aggregator. A news source doesn't update very frequently (say once every 5 minutes), but you want to provide updates super fast (say within 30 seconds). That means on average you have to scrape the entire RSS feed 10 times for a single update. On the other hand, FB/Twitter APIs give you the flexibility to do things like "download items after this id," which hog much less bandwidth.

<comment><USER8> Looking at that article, the slowness doesn't come from having to download the entire RSS feed. As commenters there note, If-Modified-Since means you only need to download the feed when there are changes. So in your second paragraph, you have to ping the RSS feed 10 times, but you'd only need to download it once.

Rather, the slowness seems to come from the lack of an aggregation mechanism. If I want to follow 100 feeds (e.g. 100 Twitter users), I have to query each of those 100 feeds. Even though I can query them efficiently using If- Modified-Since, that's still slower than firing off one query to find out which of those 100 feeds have updates, and then querying only those which do.

<comment><USER3> Good point. The reason I didn't bring that up is that the comments suggest that if-modified-since isn't universally adopted by RSS publishers.

</comment></comment></comment></comment></comment></comment><comment><USER9> _UPDATE: Dave Stevens shared a hack around this in the comments that you can use with the Twitter API._

Using a well-documented feature of the API? That is not a hack.

<comment><USER10> You must have missed the memo where now almost anything you do on a computer is hacking.

Got a form to submit in Django? I'm hacking. Got nodejs to run a sample app? I'm hacking. Integrate a shitty jQuery plugin? I'm hacking. RTFM - nope, hacking.

<comment><USER11> Ironically, you seem to have forgotten that "hack" is also a synonym for "kludge" -- long before programmers started referring to themselves as hackers as a term of pride and endearment, many writers were referred to as "hacks".

I don't see any problem with referring to using a documented workaround as a "hack" in that context.

<comment><USER9> But that's my point -- it's not a hack OR a kludge OR a workaround.

It is the straightforward and recommended way of doing something. Twitter didn't kill RSS, they just stopped advertising it to end users.

</comment></comment></comment></comment><comment><USER3> This is super interesting. RSS has gotten really slow and bad, and a few years ago FriendFeed started to push for improvements with the Simple Update Protocol (<http://en.wikipedia.org/wiki/Simple_Update_Protocol>).

The world should move to a _standardized_ JSON-based, OAuth compliant API, but unfortunately having two big, competing players like FB and Twitter means that things will probably partition into camps. It'll be interesting to see what happens when new websites start publishing content: will they define their own new standard or clone FB/Twitter?

</comment><comment><USER12> 99.99% of users will not notice or care.

I can see a use case where devs who don't feel like building for Twitter's API will want to use RSS to quickly get a user's recent tweets. But I don't think this will significantly impact anybody's anything, and it feels like the author is exaggerating the consequences.

<comment><USER13> Most users of anything don't care about RSS. Most users also don't explicitly care about the API, either — those are both at a lower level than what users notice. That doesn't mean they're worthless.

</comment></comment><comment><USER14> And thus become even more walled gardens (a proprietary API doesn't count as "open").

The pendulum is still swinging towards "closed", but I sense it's about to swing back.

<comment><USER15> > but I sense it's about to swing back.

I'm curious what you're seeing that gives you this sense? From where I'm sitting I see things swinging even further toward closed which is rather disheartening but I'd be happy to be shown otherwise.

<comment><USER14> Just the fact that we're so extremely closed now. It just _has_ to swing back at some point.

</comment></comment></comment><comment><USER16> Does anyone have an example of RSS being used/picked up by the mainstream user?

<comment><USER17> Google Reader didn't die like Wave; that implies some level of success.

</comment></comment><comment><USER18> RSS is awesome, and standards help ease integration, but we need to stop freaking out about RSS.

What we need to focus on is making sure companies endorse open data. Allowing users to get data in, and out in a reasonable format.

</comment><comment><USER19> At least from the FB standpoint, "kill RSS" is quite an exaggeration. The code correcting the oversight that removed the links for RSS feeds in latest redesign of Page profiles should be going out tomorrow (5/10/11). Note that the feeds themselves (e.g. [https://www.facebook.com/feeds/page.php?id=9445547199&fo...](https://www.facebook.com/feeds/page.php?id=9445547199&format=rss20)) were unaffected.

/me added back the Page RSS feeds during a hackathon last year

</comment><comment><USER20> For me, reading rss is like reading newspapers or books: Better contents polished for readers, but less interactions(with authors or others readers).

On the other hand, reading facebook status or tweets is like listening or chatting with friends. Contents are impromptu and often out of context. However, it's much easier to interact.

</comment><comment><USER21> 140 char posts are not what one should should subscribe to via. a RSS reader. Though I stand for open standards I don't really think that it is crucial that Twitter and Facebook supports RSS. Especially not Facebook - Twitter because it is more like the medias that I subscribe to; blogs, newssources and so on...

</comment><comment><USER22> > Unfortunately, it seems #2 was not accidental, as it was never fixed.

That does not necessarily follow.

</comment><comment><USER23> I notice that Google Reader still lets a person subscribe to someone's Twitter.

</comment><comment><USER24> speaking of rss feeds, does anybody have a list of "social" or "information" (weather like) sites with support for rss feeds ?

</comment><comment><USER25> "Embrace and extend."

Way to support open source, guys!

</comment><comment><USER26> I've fully abandoned Google Reader for Twitter.

</comment><comment><USER27> Rss was just a transitional technology. Twitter and Facebook have made it far easier for normal people to get news updates, and for developers there are API's. RSS will live on for podcast delivery (at least until apple dropsre- invents" podcasting) but other than that, let's just let RSS rest in peace.

<comment><USER28> news in 140 chars ... reminds me of that movie where they water their plants with soda.

<comment><USER29> It's <http://en.wikipedia.org/wiki/Idiocracy></comment></comment></comment>


Ask HN: Am I releasing code artifacts or docker images? - <AUTHOR> At our company we use java with maven and Jenkins to build our components (Spring boot fat jars), push them to a private maven repo and then build a docker image of that build and push that to a private docker repo. We do this for every build.
Our docker images get auto promoted to our dev environment and later manually promoted to QA, UAT and PROD if passing all tests (Both automated and manual). We do not want to rebuild the docker image at any point past the first time.
Now, since any of the builds can ultimately become the prod release, depending on passing all tests, does that mean every maven build should also do a maven release?
I instinctively know they are two separate releases but I am tempted to just start thinking in terms of docker images rather than code artifacts. <comment><USER2> Otherwise you have to manually do a release after it gets promoted.</comment>


HappyBase, a developer-friendly Python library to interact with Apache HBase - <AUTHOR> http://happybase.readthedocs.org/

<comment><USER2> I've been playing around with HappyBase for a couple days and I'm really impressed. You can tell a lot of thought was put into design - API is very clean and intuitive.

Hoping lots of people give it a try.</comment>


Ask HN: Experience with Azure Cosmos DB? - <AUTHOR> I'm diving in to Azure Cosmos DB. Do you have any experience to report? <comment><USER2> Documentation is still a bit all over the place, at least for the .Net space. Some information is only apparent after diving into GitHub issues, it's not a total disaster but could definitely be better. Especially when some documentation recommends entirely incorrect dependencies or misleading support for some gremlin commands.

However the product itself is great, I've been using the gremlin interface almost exclusively. I will say however, that having used JanusGraph and other TinkerPop compatible products some of the more advanced gremlin features like branch and choose are absent which can make porting some applications problematic.

A further potential annoyance or benefit is the fixed usage of GraphSON for the documents in Cosmos which can make the results of your query appear radically different to what you would expect if you're familiar with other TinkerPop graphs.

<comment><USER3> I wonder how good is their geo-replication.

If I increment a counter in 2 regions/cross Atlantic, at a rate of 1000 RPS.

am I going to observe consistent count in both region on each read request?

I am not sure how they defeat laws of physics.

<comment><USER2> They have "bounded staleness" as an option for consistency, but obviously it doesn't break the laws of physics, latency is latency and you still have to account for that somewhat but at least you have the bounds to work with.

</comment></comment></comment><comment><USER3> I am also thinking about using it for one project which needs 100M inserts per day.

We'll be using range query only.

But since we are using GCP, it seems Mongo Atlas with sharding in one region with multiple availability zones is better and more cost-effective solution for us.</comment>


Remme API – store public key of your certificate on the PKI (d) blockchain - <AUTHOR> https://docs.remme.io/remme-core/docs/apis/rpc.html <comment><USER2> Link to Introduction, [https://docs.remme.io/remme- core/docs/introduction.html](https://docs.remme.io/remme- core/docs/introduction.html)

<comment><AUTHOR> Thanks! Did you try it? Can you share your feedback is this API useful for your goals?

<comment><USER2> I didn't try it because I'm not trying to solve that problem. Not that I didn't find the solution interesting after reading the introduction. This kind of project is difficult "to sell" because it requires a high level of technical commitment by potential users and the pool of potential users is already relatively shallow.

Good luck.</comment></comment></comment>


The $300 Million Button - <AUTHOR> http://www.uie.com/articles/three_hund_million_button <comment><USER2> It must be Best Buy. Their checkout copy is quoted in this thread from 2005, and matches the article exactly.

<http://www.webmasterworld.com/forum22/4423.htm>

"you do not need to create an account to make purchases on our site. Simply click Continue to proceed to checkout." "to make your future BestBuy.com purchases even faster, you can create an account during checkout."

Best Buy had close to $25 bil in revenue in 2004 and 2005.

</comment><comment><USER3> I think an implicit lesson in this story is that you should have a system to track the way visitors travel through your site (access patterns). With this in place they could have seen at a glance from the very beginning that a lot of customers balked at the register form immediately before checkout.

<comment><USER4> I agree, but knowing the bounce rate of that page doesn't necessarily indicate that the login form was the issue.

You may be surprised at how high the the "cart abandonment" rate is on ALL e-commerce sites. With nothing to compare to, perhaps they just assumed those people were a lost cause.

</comment></comment><comment><USER5> This vividly shows how important usability is.

<comment><USER6> It also shows how much users resent registration - which comes up here at least once a day.

<comment><USER7> Yep. Why does it seem like most new sites still roll their own reg? OpenID/Facebook Connect/etc all seem like better choices.

<comment><USER8> It's a usability nightmare. No one knows what OpenID is, I only heard of Facebook Connect from you mentioning it.

I once tried to establish an OpenID account for stackoverflow, but it was too much of a hassle so I didn't bother registering. Users can just reuse names and passwords. It sucks from a security perspective, but unless you can make it transparent to the user, you're pretty much boned because no one will want to use it.

</comment></comment></comment></comment><comment><USER9> This must be Amazon right? I mean, you can count on one hand the number of retailers that do $300m total online, let alone some multiple.

<comment><USER5> Amazon wouldn't make such an obvious mistake. besides they would have found it in a/b split tests which they use extensively.

<comment><USER10> Uhh...wrong. Check this out: <http://www.grokdotcom.com/2008/02/26/amazon- shopping-cart/>

<comment><USER5> You're right...

I went out and searched for screenshots myself, and found this one: <http://www.webreference.com/programming/xul/amazon.png> which shows it more clearly.

Damn, I hate being wrong :-(

And sorry Matt for being so persistent in being wrong. :-)

<comment><USER9> Ha, no problem.

</comment></comment></comment></comment></comment><comment><USER11> I appreciate the core point about the form. But something in me hesitates to believe that this all happened quite the way he says.

<comment><USER5> I did usability for a retail site two years ago that had pretty much the same problem, and solved it the same way.

I would say that the numbers are probably accurate.

<comment><USER11> Ok, you I trust. That other guy... :)

</comment></comment></comment><comment><USER12> The site this article is referring to could be "Coolhorse" [http://www.coolhorse.com/mm5/merchant.mvc?Screen=OINF&St...](http://www.coolhorse.com/mm5/merchant.mvc?Screen=OINF&Store_Code=COOLHORSE)

EDIT:Although I do not see how this one could make 25B a year.

<comment><USER13> I think there's some shopping cart software that has similar language in it. That's probably what Coolhorse is using.

<comment><USER14> looks like some good ol' oscommerce

</comment></comment></comment><comment><USER15> It's interesting that users assume you are jerks when you ask them to register no matter what your brand, or the information you ask for.

Registration has become synonymous with ways to sneak in email newsletters.

</comment><comment><USER16> Jared Spool is also a fantastic speaker. If you ever get a chance to see him give a talk (e.g., at a SIGCHI meeting), by all means do.

</comment><comment><USER17> This is more about obstacles in their checkout process rather than any button, but still important lessons to keep in mind.

</comment><comment><USER18> Wow this is very interesting but makes perfect sense. People just don't like registering for websites.</comment>


The road of tomorrow is electric - <AUTHOR> http://news.volvogroup.com/2013/05/23/the-road-of-tomorrow-is-electric/

<comment><USER2> Typically, the answer is overhead wires, which stay much cleaner. [http://en.wikipedia.org/wiki/Trolleybus](http://en.wikipedia.org/wiki/Trolleybus)</comment>


Dr. Wright and the Case of the Time Traveling PGP Key - <AUTHOR> https://medium.com/@checksum0/dr-wright-time-travelling-pgp-key-899a5b6c207b <comment><AUTHOR> Concluding Paragraph:

> _" There is no way a security conscious individual wouldn't update his key > in 2011 following SHA1 debacle, which explains why the key that signed the > document uses a default parameter that did not exists at the time of the key > creation."_</comment>


How to Get a Job at Cloudera - <AUTHOR> http://www.cloudera.com/blog/2010/07/how-to-get-a-job-at-cloudera/

<comment><USER2> "How to Get a Job" would have been an acceptable title. I also would have accepted "Cloudera Shows Hiring Practices That (Probably) Don't Suck".

I'm encouraged to see that at least some companies have sane hiring practices, and Cloudera is effectively communicating "hey, HN reader, it maybe won't suck if you work here". Well played :)

</comment><comment><USER3> I have worked with Mike Olson. One of the best people I've ever worked with. He's just remarkable. If you have the stuff, you should work at Cloudera.

</comment><comment><USER4> Hmm.... I know it's just an example, when they talk about about not wanting to go through recruiters, they mention what they'd have to pay on top of an 80K salary... and go on to say that a developer would have to be "astonishingly good" for a 100K price tag. I checked, and this company is in palo alto. are these realistic figures for what cloudera expects to pay for top programming talent?

<comment><USER5> That is under-market for top talent, maybe they make up for it in options (if you're the gambling type.)

</comment></comment><comment><USER6> It's interesting that they give out stock vs. cash as an employee referral bonus. I've never seen companies do that before, I'm not sure why.

It's usually either a meaningless amount of cash ($250? Really? For a $100k/yr engineer? At that point just give a non-cash gift), or a substantial amounts of cash (approximately 25-50% of a contingency recruiter's fee).

<comment><USER7> Facebook gave options to employees for referrals early in their history. At some point we'll switch to cash at Cloudera as well.

<comment><USER6> What's the reason for shifting?

</comment></comment></comment><comment><USER8> I get the general gist of the article - Recruiters/Head Hunters are a drain on the industry, and their fees are ridiculous.

One thing the author should know is that recruiter commissions are highly negotiable. What started as "1/3rd of first years salary, paid on day 1 regardless of whether the employee works out or not" can easily be negotiated into "We'll pay you a flat rate of $10K after a 6 month probationary period where we determine if the employee is a good fit or not."

Recruiters do have a large database of resumes, but more often than not I suppose people are using LinkedIn to find jobs, so recruiters are becoming irrelevant. If you're having a hard time finding someone with the right experience though, a good recruiter can come in handy. It's just hard to find a recruiter that actually knows the industry and isn't a trained monkey that rattles off acronyms.

<comment><USER9> How are recruiters a drain on the industry? I've never seen a place that used recruiters that didn't have a direct path for candidates to get into the hiring process.

People here seem to be shocked, shocked at the fees recruiters charge employers. I presume, unfairly perhaps, that those people have never had to scale hiring. The fully loaded _drag_ of an employee --- including their fully loaded cost and the lost productivity across the team to ramp them up and the risk-adjusted cost of lost productivity in the months leading up to you firing them --- that drag is _huge_. Recruiters are expensive, even in comparison to that cost, but they're not ridiculous; they're priced roughly where the on- paper value they bring to the process says they should be.

(We don't use recruiters, but that's not because of ideology).

<comment><USER10> Recruiters work when the most suitable candidates don't know who you are (or wouldn't consider applying for any jobs you might have)

Otherwise any value they might bring to the screening process is limited by their desire to send the number of possibly suitable candidates that maximises their chances of getting paid (which unless they really know their stuff or know they're a preferred supplier means lots).

<comment><USER9> Do I agree that recruiters are usually ineffective? Of course. But that doesn't necessarily make them a drain.

</comment></comment></comment></comment><comment><USER11> I can understand why they do it that way. But over time, I fear, people will busy themselves more with managing contacts and promoting themselves than with any deep technology work.

I mean you can see it today if you read blog comments. There is a growing share of people spending their day making flattering pseudo insightful comments on other people's blogs.

And using open source projects for self promotion is very damaging for those projects. The difficulty for many open source projects is that they need maintainers over many years even when the technology isn't the hype of the day any longer. Having self-promoters pass through a project until they got the job they want is a horrible idea.

Also, I'm not sure that it's a smart move on the part of cloudera to actively ask people to hit on their employees. Is that really the best use of their time?

<comment><USER12> If you're doing "deep technology work" in a field that's as relatively new as Hadoop, chances are your name will start to make it out there -- and that you'll start to become known to others in the community, including Cloudera staff, because you're bound to be on IRC, asking questions, submitting patches, going to conferences, etc etc etc. I agree that there are plenty of people who don't really know the technology, but who make a habit of commenting on others' blogs and so on, but those people are pretty easy to spot as soon as you start really talking to them about the technology. (Disclaimer: I work at Cloudera.)

</comment></comment><comment><USER13> This article is just more of a sign that its not what you know, but who you know that will at least get you in the front door. I really hope you don't have to get an employee referral to be considered for an interview.</comment>


Ask HN: Would you go cold turkey (quit) with half baked ideas at age 30+? - <AUTHOR>

So I am almost 31, worked for almost a decade in the corporate world, got good technical background and always had the itch to go on my own.
I have been really struggling lately with many of my half baked business ideas that I never seem to be able to lock down. So far, the good things that I have been able to do it teach myself enough python, the flask framework to build simple apps, understand what tech. stacks are used in 2012, read and contribute plenty on HN.SO I know where to start if I have an idea locked down.
The bad things: not able to lock down on any of my half baked ideas. I got plenty but just cannot make up my mind.
Is this because I am still employed (self employed consultant btw), get paid well and just lazy enough to not get to the next step ? I can tell you that I am not scared taking the plunge (done that plenty of times in my life already at times when things were in comfort zone). Then why is it so hard? What if I quit 100% and then focus on an idea even if half baked ? Would you recommend this ?
EDIT: I have a wife and she has a decent job. No kids yet but planning one soon (next year or so). We can live off her income if we live within our means which we are doing already. I have more than 18 months of savings based on our current cost of living. <comment><USER3> The 40 hour experiment...

<http://mojolabs.me/2012/03/28/the-40-hour-experiment/>

For each of these ideas, create a marketing campaign to see if you can sell the idea. Do this before you even start coding. If you can't sell the idea, then move on to the next one. Repeat until you have a winner, then start coding.

</comment><comment><USER4> I'm 30 and am in an identical decision - except, a few weeks ago, I made the decision to resign from a well paid job at a startup.

A close friend of mine from university are going it alone for the next 18 / 24 months.

I'll be relying solely on my wife's income and our substantial savings over the next 12 months.

The main propellant that started me on my journey has been the unequivocal support I've received from both my wife and parents (my own and my in-laws).

Once I was clear in what I wanted to do & had a rough plan for how I could achieve those, I waited a few months - thought about the various scenarios day & night and once I had convinced myself it was a risk worth take - I took it to my best friend (and tried convincing him).

Luckily, we have been coding together since university and we decided we could take the plunge together since our temperment and personalities are matched well.

One of my current colleagues encouraged me & said to me it'll be the best experience of my life, to paraphrase his words... "the worst that could happen is that I'd have to go back to working for someone else. There's no chance that I could die, so why the hell not give it a try?".

When I mustered up the courage to tell my CIO (who I report to...) that I was leaving - he was sad to see me leave, but overjoyed that I'd be taking the risk and trying a new venture (he co-founded the company I currently work for and previously sold an earlier startup to Cisco).

So with the encouragement of others around me, I could only blame myself for not following through and giving the 'startup' thing a try.

... and thus, I would impart this advice to you.

Over the last few months, we've been brushing up our coding skills by building prototypes of ideas we like (and launching them - this is the hard part). One of them is a project we call Math Brain (<http://mathbrain.com.au/exam>) which helps students in Grade 12 practise for their university entrance exams (or as we call them in Australia - High School Certificate).

The site is not supposed to be a billion dollar business - but it seems to have started getting some regular usage (we were plesantly surprised). Hopefully, we can continue building upon this and perhaps build a proper business out of it (we would never had known if we had never launched it).

At the same time, we're not putting all our eggs in one basket - like you, we've got a number of other app ideas we'd like to prototype and release - <USER2> built & following the lean startup methodology.

The difference though, is that we have external pressure to deliver - i.e. both my wife and parents are invested our venture succeeding and therefore we demo things to them regularly. They got sick of me telling them about the ideas I have and showing them mockups of products - now they tell me to shut up and build it.

The trick, I believe is to make sure that you have someone outside of you and your co-founders putting the pressure on to deliver the project. Once you've got this external pressure, your likelihood of success is already higher.

The fact that you have a wife that can support you financially is great - because now you are both invested in making sure you are successful - this is great motivation!!!

I hope this helps in getting the fire started... god speed!

</comment><comment><USER5> Why just not pick one idea and make an MVP during your free time? Get some experience in making an MVP, marketing it, etc. Even if it does not amount to anything, you will have learned something about the proces which you can use for your next MVP.

</comment><comment><USER6> Do you have a wife or kids to support? Do you have any savings? I would recommend at least 6 months of savings before you do something like this. A year would be even better.

<comment><AUTHOR> I have a wife and she has a decent job. No kids yet but planning one soon (next year or so). We can live off her income _if_ we live within our means which we are doing already. I have more than 18 months of savings based on our current cost of living.

<comment><USER6> then go for it!

</comment></comment></comment><comment><USER7> I'm right there with ya. Finally committed and am spending a couple hours every night hacking towards a MVP. Because why? Because coding is fun, so why not.

</comment><comment><USER8> You need a partner (seriously). I wouldn't quit until you're quite a bit farther along, especially if considering being a parent soon.</comment>


Diversity Crisis in AI, 2017 edition - <AUTHOR> http://www.fast.ai/2017/08/16/diversity-crisis/ <comment><USER2> I don't see how you plan on targeting diversity within AI by hosting classes in a concrete location (that too in the US). You want diversity? Send your employees abroad to 3rd world countries. I lead a university society in Pakistan that focuses on raising awareness and expertise in AI. We're in dire need of mentors and experts.

</comment><comment><USER3> > guess what the diversity stats of the Google Brain team is? It is ~94% male > with 44 men and just 3 women and over 70% White.

She has a point complaining about the 94% males, but as the US is 72.4% white, I don't see what's wrong with "over 70% White".

> just 3% of Google's technical employees are Black or Latino

Heh, Asians are once again not diverse enough to even mention.</comment>


Show HN: Playlists for learning - <AUTHOR> https://jooseph.com/ <comment><AUTHOR> Hello Hackers,

This is basically a platform where you can learn with a collection of links to different resources.

Me and my co-founder really like to learn new topics but nowadays, we can't escape from fluff content. Especially when you are just starting off a new topic. We were frustrated by the search process which is time wasting.

Search Google, find the right keywords and then dodge all the fluff/seo content. Voila, we have just spent 2 hour and still have dozens of tabs opened in our browsers, hours of content waiting for us to read or watch them. Yet, still we don't know which content would be valuable for us and which won't.

So we created a place where you can share playlist for learning and others can rank them.</comment>


New EC2 Standard Instances and Price Reductions - <AUTHOR> http://aws.typepad.com/aws/2012/10/new-ec2-second-generation-standard-instances-and-price-reductions-1.html

<comment><USER2> Always nice to see new options, but I wish Amazon hadn't dropped the ephemeral disk. Quite apart from the issue of surviving EBS failures, ephemeral disks are wonderful for swap space -- since the lifetime of swap exactly matches the lifetime of ephemeral disks. I even have a startup script in FreeBSD which autoconfigures swap space striped across a few GB of ephemeral disks.

<comment><USER3> All of the disk performance benchmarks (iops, throughout, buffered vs unbuffered) I ran on ephemeral disks matched exactly the performance of ebs. That could be a coincidence that my tests ran only on machines without local disk, but more likely ephemeral disk was really just EBS in most cases. So it always seemed like a silly or even misleading product to me.

<comment><USER4> Ephemeral is definitely not EBS, as its I/O does not contribute to saturating an instance's network connection (I've tested that). EBS has a ceiling for throughput (the instance's network throughput) and competes with your other network traffic, which is interesting for database workloads for obvious reasons. In addition, every single benchmark I've seen run reports ephemeral behaving very differently from EBS.

As ephemeral disappears when you stop an instance (presumably, when the system is given the opportunity to relocate your instance to another machine), I've always suspected that ephemeral storage was part of local disks on the virtual host's chassis -- as opposed to a SAN or other kind of network-attached storage. As such, you probably end up with the same unpredictable performance you find in all virtualized resources, since it is very unlikely that Amazon is giving you your own storage.

Before benchmarking ephemeral storage you have to pre-warm it, which might have contributed to your findings. Ephemeral is worlds better than EBS, particularly in outage scenarios; if I could convince everybody on planet Earth to stop using EBS, it would be a noble cause.

<comment><USER3> Hmm, I have been told by Amazon and confirmed in my testing that EBS is immune to the network throttle (unlike machismo to machine tcP, s3, etc)

<comment><USER4> It's not a throttle, it's the "physical" capacity of the "interface". If you're on an instance with gigabit connectivity and you're doing 1 Gb/s of EBS I/O, other network chatter will suffer, probably fairly dramatically. That's why the high-I/O instances have 10 gigabit connectivity, as I understand it.

Happy to be proven wrong but this is based on a year or so of experience dealing with EBS. You can't see the EBS traffic in your tools (at least that I've been able to find), which complicates things.

It's all kinds of different on VPC instances, therefore I suspect the network interface model -- and possibly EBS connectivity -- is different on those. So, who knows? I'd kill for Amazon to be more forthcoming here so I could understand the infrastructure running my fleet, but, I don't and they aren't.

<comment><USER5> Hey, you got hellbanned for posting on this thread: <http://news.ycombinator.com/item?id=4743954>

</comment></comment></comment></comment></comment></comment><comment><USER6> For some reason that is currently unknown to me (did not investigated) this instances are much better with Redis since they are able to fork using more or less 1/20th of the time needed before. (4.4 GB process forked into 44ms instead of 1 second).

EDIT: it's just hardware virtualized mode (HVM).

</comment><comment><USER7> That's strange. Did they upgrade the Micro instance's memory too?

> The Micro instance (1.7 GB of memory) is great for lower throughput > applications and web sites.

I assume its a typo. Otherwise, fenomenal news!

<comment><USER8> The Micro instance remains at 613MB according to: <http://aws.amazon.com/ec2/instance-types/>

So looks like a type-o sadly.

<comment><AUTHOR> Typo - Fixed, but only because my back went out and I can't sleep.

</comment></comment></comment><comment><USER9> Too bad they didn't reduce any of the reserved instance pricing. So effectively they've reduced the cost for things that scale up and down a lot, but not for permanent base load.

<comment><USER10> I'm not sure why this is a surprise. Real 'clouds' in my opinion are all about elasticity. If you can't benefit from that then you're wasting your money and should host elsewhere.

<comment><USER11> Because most users have a relatively fixed base load, and buying reserved instances for that base load lets you save lots of money. The ideal setup would look like: reserved instances for base load, on-demand for natural daily curves, and spot instances for batch or price sensitive work.

</comment></comment></comment><comment><USER12> Having never used EC2, I wonder what an instance would cost me each month if I were to use it for something like hosting my website.

I'm very happy where I am now, but I'm interested in learning more about Amazon's offerings and moving my blog over would probably be a good first step.

<comment><USER13> I'm in your same position exactly but unfortunately the options aren't good for what you (and me) will use it for. Before this price drop a Micro instance would run you about $15 - $20 monthly. About the same as I pay for my 512 Linode. But I've heard horror stories about the CPU on the Micro not being able to handle any amount of moderate traffic for more than a few minutes at a time. So then there's the Small instances which are, at least for me, ideal anyway but when you do the math on a Small instance running 24/7 each month with just 10GB of EBS storage you get up past $60 a month. Okay, not bad - pretty close to VPS pricing, actually. But $60/month is a lot for me to spend on hosting a blog so its not for me. I don't know about you. Maybe your priorities and financial situation is different. I'm not hurting at all financially, I'm just cheap (which is why I'm not hurting financially).

<comment><USER14> You can run a small reserved instance for $11.71/month (with $195 prepayment for the year). That comes out to about $27.96/month, plus data transfer fees. If you prepay for 3 years, it averages out to about $17.85/month.

<comment><USER12> I'd rather pay $20 a month for a VPS that I don't have to prepay for.

</comment></comment></comment></comment><comment><USER15> Pity they didn't introduce smaller High I/O instance types. I would love High I/O but don't need anywhere near 60Gb RAM :/

<comment><USER16> It would be interesting if they could get rid of "types" concept, and just let people pull together memory, ecus, and storage on demand.

<comment><USER2> That would be a provisioning nightmare, trying to figure out which partially- used-up box to place a new instance into. Much easier to slice them into fixed sizes.

<comment><USER17> Someone's provisioning nightmare is someone else's business opportunity ;-)

<comment><USER18> Cloudsigma (www.cloudsigma.com) is trying to grasp that opportunity.

</comment></comment></comment></comment></comment><comment><USER16> I like that the new instance types have a more balanced ratio of memory to ecu, but wish they had included some smaller sizes. I'd like an m3.medium with 4gb and 4 ecu (or an m3.large with 8gm and 8 ecu).

</comment><comment><USER19> Does new instances work with regular paravirtual AMIs or do you need to build an HVM AMI like for cluster compute?

</comment><comment><USER20> It's great that EC2 prices are coming down, but they're really not keeping up with the general trend in computing prices.

If you're running a static set of computers on EC2 today, and not making use of other features like S3, ELB, etc, you're almost certainly overpaying by a significant amount compared to even high end dedicated server providers, let alone the cheap end of the market like OVH and Hetzner.

<comment><USER21> It's always been true that running a static set of computers on EC2 means you're overpaying. It's not a case of not keeping up with the trend.

<comment><USER22> Reserved Amazon instances are not the cheapest but are competitive with some hosting providers. I've seen some comparisons where they're right in line with Rackspace's cloud offerings, with each winning on slightly different specs. And of course Amazon has different bandwidth pricing than almost anyone else (charging per-request as well as per-GB) so that might make the difference.

</comment></comment></comment><comment><USER23> EC2 is still kind of expensive.

<comment><USER24> Compared to what?

<comment><USER25> ... linode

</comment></comment></comment><comment><USER26> No price drop for US West California though?

<comment><USER24> We switched to Oregon once we discovered the price advantage and never looked back. Granted, our deployment is relatively small right now.

<comment><USER27> What's your latency experience been? Is yours a public-facing deployment?

</comment></comment></comment><comment><USER28> I heard a rumor that Compute Engine was supposed to open to everyone today, I suppose this reinforces that rumor.

<comment><USER29> Can you provide more details / URL?

</comment></comment><comment><USER30> Its good for small users. How does 0.0x effect a saving for others.

<comment><USER31> No matter how small or how big a customer you are, it is a price decrease. The cost of these first-generation standard on-demand instances has been reduced by 18.75%.

<comment><USER32> Well, except micros.

<comment><USER31> Micro instances (t1.micro) are not part of the group of first-generation standard instances (m1.*).</comment></comment></comment></comment>


Team Foundation Service by Microsoft - <AUTHOR> http://tfs.visualstudio.com/

<comment><AUTHOR> While upgrading to Visual Studio 2013 you're prompted to create your account on the new Team Foundation Service site.

Overall feels pretty interesting. One thing of particular interesting is "cloud based load testing" service available for hosted repositories.

[http://tfs.visualstudio.com/en-us/learn/load- testing](http://tfs.visualstudio.com/en-us/learn/load-testing)

I'm a huge fan of Github and I don't see myself changing anytime soon, but I'm interested to see what features end up a part of the TFS offering.</comment>


Biologists Decode an Ancient Cancer-Fighting Antibiotic - <AUTHOR> http://motherboard.vice.com/read/biologists-decode-an-ancient-cancer-fighting-antibiotic

<comment><USER2> Bad day for a almost too good to be true drug, but the scientific articles look legit.

Linked at the article (March 31 2015) [http://www.nature.com/ncomms/2015/150331/ncomms7402/full/nco...](http://www.nature.com/ncomms/2015/150331/ncomms7402/full/ncomms7402.html)

Another previous article (August 2012): [http://www.ncbi.nlm.nih.gov/pubmed/21678129](http://www.ncbi.nlm.nih.gov/pubmed/21678129)

<comment><USER3> Why bad day?

<comment><USER4> Hahahah April Fools! You fell for our joke about live saving drugs!

<comment><USER3> Well, I'm not sure this is a good joke for April Fools.

<comment><USER5> You must not have cancer.

<comment><USER3> I still don't get it. I'm not an English native speaker, so sometime reversed speaking sarcasm is a bit hard to graps (if any was involved).

I hope I didn't offend anyone. I thought the article was about a newly discovered cure, the joke being that it was all fake. And I just thought, bad taste for a joke.

<comment><USER6> To clarify -- the joke in Vraxx's post is the absurdity of the contrast between the tone of the speaker ("Hahahah!") and the seriousness of the subject matter ("life saving drugs"): no-one would actually make such a statement.

The joke in nugget's post is the absurdity of his implication that a cancer patient would find it _funnier_ if the article were a joke; when in fact a cancer patient would be the most offended.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> In what form was it used in China? (Wikipedia and Google--and the article itself--weren't illuminating in this case.)</comment>


Ask HN: What are some examples of machine learning startups in defense sector? - <AUTHOR> Modern defense capabilities are highly dependent on use of computers to assess preparedness, cyber attacks, terrain analysis, operations management, etc. I am curious whether there are any startups that convinced the govt. to address these needs or it is the usual high level contractors (lockheed or such) competing for these services? What kind of military needs can be civilian* in nature, for example I know Amazon helps army with supply chain management for food supplies and other such things.<p>* - forgive me for the lame word placement here <comment><USER2> There are way way way too many applications to list. DARPA has been putting money into ML for a long time as far as I can tell.

A search on google scholar yields 58000 research articles from the search "machine learning darpa" alone

[https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_vis...](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_vis=1&q=machine+learning+%22DARPA%22&btnG=)

check out their research page for more [https://www.darpa.mil/our- research](https://www.darpa.mil/our-research)

one example that stands out to me recently is their challenge for automatic frequency/spectrum allocation

<comment><AUTHOR> that's very insightful, thank you :)</comment></comment>