PostgreSQL as Schemaless Database [pdf] - <AUTHOR> https://wiki.postgresql.org/images/b/b4/Pg-as-nosql-pgday-fosdem-2013.pdf <comment><USER2> Where is taligent when we need him? MongoDB is pretty impressive considering its age to the market, but I'd still go with the relational version of PostgreSQL anyday. Not because of the benchmarks, but in general, most use- cases are covered well enough by our relational db's like MySQL and PostgreSQL. The most expensive mistake one could make is using a NoSQL db where an SQL db fits perfectly. I admittedly made this mistake and learnt a LOT. Now, these benchmarks are just some extra spice for sticking with my lovely PostgresSQL :)

<comment><USER3> Why do you need me ? I am just that crazy troll who believes that no database is perfect for all use cases and that making blind generalisations like "all NoSQL is bad" is stupid. Especially since MongoDB is as different from Cassandra or Riak as it is from Spanner as that is from the other 120+ odd NoSQL databases.

And if benchmarks impress you I suggest you take a look at Cassandra sometime.

[http://techblog.netflix.com/2011/11/benchmarking- cassandra-s...](http://techblog.netflix.com/2011/11/benchmarking-cassandra- scalability-on.html)

<comment><USER4> I keep hearing Cassandra touted as fast, but I just don't see it. Instacluster brags about 17ms at the 99th percentile being fast, but where I come from that gets you thrown out of the running.[1] The Netflix benchmark linked above sees 10,000 ops/sec/node. Again, not impressive.

Skip to slide 22 of this comparative benchmark with Couchbase Server[2] and you'll see that <5ms latencies at 99% is totally achievable. We can't take too much credit for these stats, most of the blame goes to the fact we use memcached at the front end, and it's been optimized for consistent low-latency for a long time.

I don't want to make this a speed contest, as there are a lot of good things about Cassandra. It just strikes me as odd that people sell Cassandra based on low latency. If you want consistent high-performance, don't use a database that runs on a language virtual machine. There are lots of great reasons to use Cassandra, standout benchmarks aren't one of them.

[1] <https://www.instaclustr.com/solution#tco> [2] [http://www.slideshare.net/renatko/couchbase-performance- benc...](http://www.slideshare.net/renatko/couchbase-performance-benchmarking)

<comment><USER5> What else would you say is good about Cassandra?

I find the datamodel inflexible and often hard to work with for real world use cases. I think the tooling is immature and the hector API leaves me feeling depressed (pycassa and astyanx are quite a lot better).

All that said, I find cassandra to be fast, incredibly fast. 99th percentile on our 9Tb time series store is around 20ms on below par hardware.

Another thing i like is we've never had downtime, not even scheduled. We've had nodes fail, we've had datacentres isolated during disaster recovery tests, yet our cluster has continued on regardless (2 x sub clusters per region). Whatever way you stack it up, that's impressive. Remember with cassandra there's no load balancers or other shenanigans involved.

Garbage collection / virtual machine concerns are a red herring due to the mechanics of a cassandra query (certainly for CL < RF)

<comment><USER4> I'm glad you are happy with 20ms, to me it is slow...

Probably the best thing about Cassandra is that it is written in Java, so it is easy to fork and add custom behavior. Of course this is orthogonal to performance.

</comment></comment></comment></comment></comment><comment><USER6> So, the problem that I have with PostgreSQL isn't that it doesn't have every datastore under the sun, but rather the lack of automated distribution, and fault-tolerance.

This is not a hard thing to build though (continue reading please). I don't mean that it's an easy task, but that the semantics by which you may be able to build a fault-tolerant, distributed database on top of Postgresql are pretty straightforward.

1\. Partitioning: So, Postgresql has multiple notions of partitions. Ideally, this would be done at the database level in Postgres, but the idea is that instead of having your entire database fail at once, a transparent layer in which you SET PARTITION_KEY=X before querying would make this pretty straightforward. It would be nice if there was a semi-transparent proxy layer that this.

2\. Replication / Fault-tolerance: Postgresql today has built-in synchronous master/slave replication. It would be nice if someone built a automated failover system on top this. Alternatively, it would be really interesting if someone had multiple postgresql instances, and then build a WAL log per partition, and performed 3PC between N replicas, and you could have a fully peer-to-peer postgres. I imagine these transactions would either need to be PL/SQL, or fully written-out, staged serializable SQL queries.

3\. Co-processors: One of the biggest benefits a distributed, fault-tolerant Postgresql setup would give you is the ability to collocate your application with the data. Instead of having to do read, modify, write, you can write complex business logic that can run on whichever Postgres partition's master node, and the data doesn't have to go over the network.

I feel by introducing some basic wrappers around postgres to support partitioned, master-slave topologies, and perhaps a slightly different query interface to support this transparently, PostgreSQL could effectively replace most modern NoSQL databases.

<comment><USER7> Hot Standby in postgres is pretty damn good for replication (like you said making failover occur properly takes a little bit of effort but its not too hard).

Also pgpool2 helps a lot with partitioning and parallel queries.

For a relational db, postgres does a pretty good job at distribution and fault tolerance.

<comment><USER8> Postgres won't let you perform any sufficiently long (read-only) queries on slaves. For example, I was unable to run pg_dumpall on a slave to have DB snapshots.

This is [well-explained]([http://www.postgresql.org/message- id/201102272005.00234.jens...](http://www.postgresql.org/message- firstname.lastname@example.com)), but still inconvenient. This way, slaves are only useful for failover and possibly offloading very short read- only transactions, which is pretty limited.

<comment><USER9> This used to be the case, but it's not anymore. You can run long-living queries, run pg_dump, etc on slaves.

<comment><USER10> Do you have a reference in which pg version this got fixed? We still pg_xlog_replay_pause() our slaves, it would be nice to drop that.

<comment><USER7> [http://www.postgresql.org/docs/9.2/static/runtime-config- rep...](http://www.postgresql.org/docs/9.2/static/runtime-config- replication.html#GUC-MAX-STANDBY-ARCHIVE-DELAY)

</comment></comment></comment></comment></comment></comment><comment><USER11> This is pretty much the reason we're using Postgres for our document storage. Admittedly, our doc store isn't really extensive and it's mostly for internal use (with limited client access), but we just didn't see why it was necessary to go with a new database when the existing stack worked well.

YMMV, but we've yet to come across an instance where a pure document DB was necessary or desired, even to store documents (which from my perspective seems nothing more than a flattened EAV model).

<comment><USER12> Do you have any thoughts on expression indexes? Assumed I would have to pull out fields from the document to index reasonably on pgsql, but this presentation at least shows that simple expression indexes perform well. I wonder about compound, nested, etc.

<comment><USER11> I'm not too familiar with the current schema, but I do know we make extensive use of partial indexes ( _WHERE (vnnk IS NOT NULL)_ , _WHERE (fhs > 0)_ etc...) and compound indexes in btree. In GIN, for example, this isn't possible, but GIN itself works reasonably well for partial matches (since it's also btree) so it's not too much of a deal. We use a combination of hstore, GIN, conventional fields and multiple indexes with the schema abstracted out on our app to provide a document storage and search engine with a separate app for client login. I believe some metadata is stored in conventional fields (varchar integer etc...).

Performance compared to the conventional schema is of course noticably slower, but not unacceptably so for a few thousand docs and growing (this is compared to hundereds of millons of rows on the conventional schema).

But such is the sacrifice for reliability. When NASA hit a spat of failed missions to Mars (explosions, collisions, miscalculations etc...) people were joking about the creed at the time, "Faster, Better, Cheaper; Pick 2".

This kind of engineering is a turn off for a lot of folks, so I can certainly see the appeal of a document DB. I've also seen people fall into the trap of "I don't know what kind of data I'll be getting so I'll go with a doc store". Well, there has to be _some_ hint of what kind of data could be arriving so engineering for that with some wiggle room early is a good idea.

You shouldn't take my word or the slides at face value, though. Really think through, experiment, throw stones at it etc... and see if it works for you.

<comment><USER12> Reliability is gold.

Thanks for the pointers. I will read up on GIN and hstore.

If nothing else, duplicating an element from a doc column to a first class field seems reasonable. Also, even in mongo one must define indexes which is a schema of sorts and can be non trivial to change.

</comment></comment></comment></comment><comment><USER13> Really great presentation/slidedeck.

For me I still feel MongoDB is a semi niche solution. I rather go with a standard SQL database unless there is a specific reason why you need NoSQL.

<comment><USER14> I've tried both MongoDB+Mongoid and Postgresql+hstore as a way to do better inheritance than STI in ActiveRecord.

Both have deficiencies: Mongo is not mature enough for an old lag like me to trust (and is incompatible with some Rails extensions), and hstore had problems with key deletion and cannot do object references or indeed types other than strings.

<comment> shawn-butler Can you elaborate on what you mean by key deletion problems?

</comment></comment></comment><comment><USER15> First thing I remembered after reading the title is [http://pgsnake.blogspot.in/2010/04/postgres-91-release- theme...](http://pgsnake.blogspot.in/2010/04/postgres-91-release-theme.html)

<comment> mh- haha, thanks for posting that. i enjoyed:

 _"It's time for us to switch to something fresher. I personally would have preferred XSLT, but QUEL is almost as good."_

</comment></comment><comment><USER16> The new JSON functions have landed in the development version, for anyone interested in seeing the future syntax: [http://www.postgresql.org/docs/devel/static/functions- json.h...](http://www.postgresql.org/docs/devel/static/functions-json.html)

</comment><comment><USER17> Damn. The "XML. It seemed a good idea at the time" slide caused me to spray coffee on my laptop.

<comment><USER18> See also "XML literals as a native syntax in Scala."

</comment></comment><comment><AUTHOR> I want to admit that conclusion in presentation can be interpreted vise versa: MongoDB almost is fast as PostgreSQL. Also, notice that comparsion was done with Mongo 2.2, in 2.4 JS engine was changed to V8, so there is a lot of speed improvements. And compare PG have 18 years of development vs 5 of MongoDB. So what I want to say? Difference is not so big after all.

<comment><USER19> PostgreSQL hasn't spent 18 years pitching itself as a pure performance play, though. It started as a research project into the object-relational model and was adapted to be a safe and featuresome database first and performant second.

Mongo's performance will converge downwards as people demand more features and safety guarantees.

If however you want the original premise -- memory-only, schemaless, auto- sharded document database -- Mongo is still the easier pick.

It's a cliche, but it's easy to:

1\. overestimate the size of your data, (especially since relational models _by design_ squeeze out all duplicated statements of data)

2\. overestimate how much traffic they need to serve

3\. underestimate what can be achieved by stock RDBMSes with modern hardware

4\. underestimate the deferred development costs of schemaless data and

5\. underestimate the costs of avoiding durable storage.

It's a really hard field to make generalisations about right at the moment. The algo-economics of hardware (especially SSDs) and software have been changing so fast that any generalisation formed even 2 years ago is unreliable.

Speaking only for myself, I need a lot of talking down from my data- safety/data-integrity tree. _I don't trust myself_. That's why I use tools that slow me down.

<comment><USER20> "1. overestimate the size of your data, (especially since relational models _by design_ squeeze out all duplicated statements of data)"

Of course, this is opt-in in a SQL world, more's the pity. I've seen some nightmarish ActiveRecord spawned Postgres monstrosities.

I wonder how much of the attraction of schema-less DBs comes from the lack of friction between the largely guarantee-free idea of data storage precipitated by Rails misfeatures -- there's much less distance between a MySQL or Postgres database as "designed" by ActiveRecord and a NoSQL instance. Having already opted out of your database's particular guarantees, there's very little keeping you there.

<comment><USER21> Here's a question for people who prefer to put their guarantees (i.e. unique/check constraints, foreign keys etc) in their database. How do you present these constraints to the user? How do you show when they've been violated? Just present them with the PG error message? Maintain a mapping of PG errors to more domain specific messages? Bite the bullet and just code them twice (once for the DB, and once for the user)?

<comment><USER20> One could build out from the database schema -- Postgres certainly provides sufficient type information to build consistency double-checks in other layers of the application. As an added bonus, other consumers of the data won't be flying blind.

<comment><USER21> Yeah, I'm trying to build something like that right now. Nice to know others have similar ideas.

</comment></comment></comment></comment></comment></comment><comment><USER12> It's only one of several interesting comparisons in the slides, but I think we need more data points on the disk space to make use of the data. Does it grow linearly? probably, but is there a +x in there? Mongo has some big upfront disk allocations. Were those considered somehow?

</comment><comment><USER22> When the benchmark shows postgres can do the same as mongo, only faster, then why aren't we seeing more hybrid solutions? Online retailers would be better off using relational storage for monetary transactions and time series, and json storage for document like data.

<comment><USER23> The great thing about PostgreSQL is that it _is_ hybrid.

With <http://www.postgresql.org/docs/9.1/static/hstore.html> you can put schemaless documents into the same database, even into the same table row as your usual relational data.

You could start with a schema that's only a primary key and hstore for all other data (like a fully schemaless database), but over time extract important fields from the hstore into columns to add indexes, constraints or joins.

<comment><USER24> That is... I am not sure, either madness or genius.

But an intriguing idea either way.

</comment></comment></comment><comment><USER25> I am always wondering why no-sql has better performance than SQL if we have same simple use case (no join etc.).

If the data is well structured and no join is needed, can I assume that relational database should have better performance than no-SQL?

<comment><USER26> Depends what you mean by "NoSQL". MongoDB is pretty terrible overall, so don't even bother.

Systems like Dynamo/Riak/Cassandra sacrifice consistency (most of the time) for performance, largely because a distributed setup (sharding in particular) is easier when you don't guarantee consistency. Note that none of them sacrifice durability.

It is possible to have a distributed _and_ consistent database, but you either have a bottleneck (master nodes in BigTable/HBase), require significant infrastructure (Google Spanner) or make schema changes very hard (Hyperdex). It's a hard problem.

<comment><USER25> Thanks for your reply.

But I have one more question, if I need to manage bank account, then the database must be consistent. Lets say that I just need to keep balance and social security number for the account, and only need to query the balance. Can No-SQL like redis have better performance than relational database? (assume same on logs and transactions).

<comment><USER26> Redis is consistent, it's just less durable than Postgres. A query cache is a very good usage for it. It's also likely to be faster than Postgres for the simple operations it is capable of.

The only downside is that (for now at least) it is single node, with optional replica slave(s).

</comment></comment></comment></comment><comment><USER27> There is still some way to go to support better indexing of json fields, but I'm pretty excited about these developments. In the future you might see schemaless and schema driven data living side by side on postgres.

</comment><comment><USER28> Would be interesting to know which version of MongoDB was used (2.4 also uses V8).

The original slides are at: [https://wiki.postgresql.org/images/b/b4/Pg-as- nosql-pgday-fo...](https://wiki.postgresql.org/images/b/b4/Pg-as-nosql-pgday- fosdem-2013.pdf)

There is also another presentation on the use of PL/V8 (on Heroku) at the PostgreSQL wiki - from last week's NYC talks: <http://plv8-talk.herokuapp.com/#1>

<comment><USER29> In one slide it says he used MongoDB 2.2 on his mac.

</comment></comment><comment><USER26> Sadly it's not a fair fight, of course it'll beat Mongo (even though Postgres is ACID).

I would be more interested in comparing the biggest machine you can get on AWS running Postgres with a cluster of smaller machines running Cassandra. I expect Cassandra to win, but of course it's not consistent.

</comment><comment><USER30> Well that's Monday's afternoon decided !

</comment><comment><USER31> Almost 6,000 synchronous commits per second to a 7,200 rpm disk seems high to me.

Anyone seeing those numbers for any database that synchronously writes the log to disk?

<comment><USER32> [http://pgeoghegan.blogspot.com/2012/06/towards-14000-write-t...](http://pgeoghegan.blogspot.com/2012/06/towards-14000-write- transactions-on-my.html) may interest you

<comment><USER31> While very impressive, I don't see how your post is applicable in this case. The OP at the very least implies that the transaction are done end to end and not batched. I find 6,000 distinct disk I/O operations per second, to append the log, very very high for a 7.2k disk.

<comment><USER32> that post, also discusses using a 7200rpm disk.

I think where you may be confused is considering them as distinct IO operations, vs seeing it as a point which must have been flushed past ( sequentially )

<comment><USER31> No, I don't think I'm confused. The OP says postgresql is used in its default configuration:

 _Stock PostgreSQL 9.2.2, from source. No changes to postgresql.conf_.

Given that statement then either the writes are serialized or postgresql is not ACID compliant in its default configuration. I'm not an expert on postgresql but I assume that it is ACID compliant in its default configuration. Therefore my skepticism on the 6K writes per second.

<comment><USER32> from the link: "In Postgres 9.2, this improvement automatically becomes available without any further configuration."

"Essentially, it accomplishes this by reducing the lock contention surrounding an internal lock called WALWriteLock. When an individual backend/connection holds this lock, it is empowered to write WAL into wal_buffers, an area of shared memory that temporarily holds WAL until it is written, and ultimately flushed to persistent storage."

"With this patch, we don't have the backends queue up for the WALWriteLock to write their WAL as before. Rather, they either immediately obtain the WALWriteLock, or else queue up for it. However, when the lock becomes available, no waiting backend actually immediately acquires the lock. Rather, each backend once again checks if WAL has been flushed up to the LSN that the transaction being committed needs to be flushed up to. Oftentimes, they will find that this has happened, and will be able to simply fastpath out of the function that ensures that WAL is flushed (a call to that function is required to honour transactional semantics). In fact, it is expected that only a small minority of backends (one at a time, dubbed "the leader") will actually ever go through with flushing WAL. In this manner, we batch commits, resulting in a really large increase in throughput..."

I am sorry to cut and paste so much of the article, but I hope this is helpful?

<comment><USER31> Looking at your linked posting more closely I do not believe it is applicable to this situation for two reasons. Again I'm not an expert on postgresql but:

1) It appears to me that the technique discussed in your linked post is about ganging unrelated transactions together into a single flush to disk. I do not see that to be the case in the OP. Since all of the writes are going to the same table they are related.

2) Looking at the second graph in the linked post _pgbench transactions/sec insert.sql_. The number of clients is high. I got the impression from the OP that there was only a single client. Indeed if there were more than a few clients the benchmark would have been subject to the deficiencies of the client libraries used.

<comment><USER32> you are good to be suspicious into any benchmark(as even the original slides note). I dont think we really have any information as it relates to transaction boundaries present, nor clients used:

From the slides: "Scripts read a CSV file, parse it into the appropriate format, INSERT it into the database. • We measure total load time, including parsing time. • (COPY will be much much much faster.) • mongoimport too, most likely."

Postgres does its best to use intelligent defaults, but it is only a part of the system, it is generally up to practioners to be wary. Tools like: [http://www.postgresql.org/docs/current/static/pgtestfsync.ht...](http://www.postgresql.org/docs/current/static/pgtestfsync.html) Assist in this goal, but generally have to watch out for things like raid controllers that are not battery backed etc ( depending upon your environment).

There is no mention of the number of clients used directly.

I was simply trying to highlight that with the (little) information available, it is very possible.

It is also possible, that the session(s) should themselves choose to pursue an asynchronous commit strategy ( on a session level see:[http://www.postgresql.org/docs/9.2/static/wal-async- commit.h...](http://www.postgresql.org/docs/9.2/static/wal-async-commit.html) ) which would also not require modifying the configuration, I do not know as I have not seen the scripts, but it is similar to how a library could interact with:

[http://docs.mongodb.org/manual/reference/command/getLastErro...](http://docs.mongodb.org/manual/reference/command/getLastError/)

Thanks.

<comment><USER31> Nice discussion, thank you.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER33> The problem with this scheme is that Postgres does not support JSON indexing and partial updates which are important for high transaction databases.

</comment><comment><USER34> When I checked hstore's functionality the last time (~6 months ago) it was rather limited compared to Mongo's full-blown API. Any new observations?

</comment><comment><USER35> MongoDB's native storage type is not JSON, but BSON. One should know this if they're about to wade in a deep river, trying to argue against it.

</comment><comment><USER36> would it be possible to build an extension to expose a mongo compatible interface in postgres - essentially being able to use all the ORM/code built around mongo for postgres, seamlessly.

<comment><USER26> Yes, but why bother? There are already so many good SQL clients out there, like SQLAlchemy.

<comment><USER36> there are many apps built around mongodb that it would be interesting to do a drop-in swap with postgres (assuming protocol compatibility).

hell, ill personally pay a lot to have a mysql compliant interface just for WordPress!

<comment><USER26> I see, that motivation makes more sense to me :-)

I guess it should be possible to write a wrapper that executed at isolation level "single".

</comment></comment></comment></comment><comment><USER37> Oh. This is legit?

I skipped this story because I assumed it was an April Fools' prank.

</comment><comment><USER38> What I get from these slides is that MongoDB has better average performance (when both using indexes or doing full scans) compared to all schemaless PostgreSQL options.

</comment><comment><USER39> sorry but the first thing I readt was "PostgreSQL the shemales database". Confusing ..

<comment><USER40> Same here, don't feel bad.

And to the downvoters, the correct word is aschematic or schema-less with a dash.

If a word ends in a vowel, you must use a dash.

Hatless is fine, but scrupleless should be scruple-less. The more you know, you scruple-less bastards!

<comment><USER41> > If a word ends in a vowel, you must use a dash.

Feeling prescriptive these days?</comment></comment></comment>


Scientists Predict Star Collision Visible to the Naked Eye in 2022 - happy-go-lucky http://www.npr.org/sections/thetwo-way/2017/01/09/509010493/scientists-predict-star-collision-visible-to-the-naked-eye-in-2022 <comment><AUTHOR> Previous discussion: [https://news.ycombinator.com/item?id=13348829](https://news.ycombinator.com/item?id=13348829)

</comment><comment><USER2> Maybe I'm up too early, but I have an idea: Where will you be in your life when those stars collide?

Write down (briefly) where you are in your life right now, on a blog, Facebook, or even some tweets.

Then write down where you think you'll be in your life in the five or so years when these stars collides. Hashtag it #whenstarscollide or give it a similar title so people can find it when the times comes.

<comment><USER3> In the year 222, when those stars collided, i wasn't anywhere.

At best i could note where i am when the light of the collision arrives here.

</comment></comment><comment><USER4> Good news is that the James Webb telescope should be up and running by then.

</comment><comment><USER5> I've told that to my mum. She basically freaked out when I mentioned that the new star will be red. She has very little if any scientific background and she struggled to find words to describe what she felt but I could feel she saw this upcoming event as a esoteric sign or something.

People who are even more into astrology and stuff are probably losing their s.it.

</comment><comment><USER6> The prediction is really that these stars collided about 1795 years ago, and we may find out about it soon.

<comment><USER7> Because simultaneity itself is relative, in our frame it actually does happen in 2022. It doesn't make sense to say the stars already collided if it's not in our past light cone.

<comment><USER8> Simultaneity is only defined when you talk about two (or more) events. The discussion was about an event: the collision of two stars. What is the other event?

<comment><USER9> The generous interpretation of what he means is that considering that this event is outside of our past light cone, there exists an inertial reference frame such that the collision happens "after" our "present".

Of course, relative to our own inertial reference frames as we experience them, i.e. where we are mostly not moving, the collision would be thousands of years in the "past".

When he says "in our frame" he is clearly at least slightly confused.

<comment><USER10> It's like saying lighting struck the moment you heard the sound.

</comment></comment></comment></comment></comment><comment><USER11> So will it be visible in both hemispheres?

How will we know when and where to look?

<comment><USER12> In the northern constellation Cygnus (the Swan) As far as places in the sky to notice a new star showing up by eye it is suboptimal since it is right in the middle of the Milky Way.

<comment><USER7> Plus Cygnus and nearby constellations already have some of the brightest stars in the sky.

<comment><USER13> Yes Cygnus has bright stars. But this star is predicted to change apparent magnitude from 12 to 2. If it becomes apparent magnitude 2, it will be the second brightest star in Cygnus, just past Deneb.

</comment></comment></comment></comment><comment><USER14> Any one know for how long we'll continue to observe the aftermath of the collision?

</comment><comment><USER15> i love how you posted this ironically.

"we might be able to see this with the naked eye! why bother when you can see it through JWST?"

<comment><USER16> We've banned this account for repeatedly posting unsubstantive and/or uncivil comments and ignoring our requests to stop.

We detached this comment from [https://news.ycombinator.com/item?id=13363274](https://news.ycombinator.com/item?id=13363274) and marked it off-topic.</comment></comment>


Ecuador Turning to Virtual Currency - <AUTHOR> http://www.bloomberg.com/news/2014-08-11/ecuador-turning-to-virtual-currency-after-oil-loans-correct-.html <comment><USER2> What's virtual about it? I can't tell from reading the article. Is it virtual merely because none of it will be coins or bills?

Do other such "virtual" currencies exist, i.e. virtual merely because they only exist in ledgers and bank balances? I mean, other than the vast majority of US dollars and other traditional currencies.

All this talk of virtual currencies reminds me of how Europeans viewed Chinese paper money with suspicion when it was first described in The Travels.

<comment><USER3> Yes, most US dollars are "virtual" by their definition.

[http://upload.wikimedia.org/wikipedia/en/5/58/MB%2C_M1_and_M...](http://upload.wikimedia.org/wikipedia/en/5/58/MB%2C_M1_and_M2_aggregates_from_1981_to_2012.png)

[http://en.wikipedia.org/wiki/Money_supply](http://en.wikipedia.org/wiki/Money_supply)

</comment></comment><comment><USER4> Given Ecuador's financial situation, this seems like a plan to print money to pay bills, and make that "virtual" money to saving on printing :-)

<comment><USER5> That would be incredibly shortsighted. Assuming it's not the case, it's still a reasonable move: they need to pay their national debts in American dollars. There's no need for countless dollar notes to be circulating throughout the country every day for intra-national commerce, anything else that would be inflated at most below US QE levels would do the same trick and release those USD notes from that task, hopefully sooner or later trickle them back to the national treasury in order to send them back to where they originated. Plans of mice and men, but they gotta give it a try. No country can afford to depend on the US national currency for too much in the long run.

<comment><USER6> Presume someone resorting to gimmicks to manage spending in excess of revenue is shortsighted (intentionally or otherwise.) Borrowing money in a foreign currency has pros and cons but I suspect most citizens don't realize the long term consequences.

</comment></comment></comment><comment><USER7> My mom is Ecuadorian and I have lots of family there and happen to own property there.

Correa is a gr8 president, but coming from the perspective of someone like myself who is well read on the intricacies of modern monetary systems, sticking with the dollar for this long has been one of the few very bad policies. From a political perspective its understandable given Ecuador's history of currency devaluations. This move will very likely permit Ecuador to continue this economic miracle and transformation that has taken place(lifting many out of poverty, etc) . These changes should enable Ecuador to increase GDP , reduce unemployment further, continue to improve public infrastructure, increase standards of living, and become less dependent on exports to drive the economy. Hopefully, this will become a lesson to the entire world wherein electing leaders that are literate about economics and the functioning of the modern monetary system -Correa has U.S. PHD in Economics- , who don't use inapplicable house-hold analogies ( i.e. gov should run like house-hold) to pander to voters ( are you listening EU ? ) can make the kind of strides that policy makers talk about but never achieve.

Read 7 Deadly Innocent Frauds of economic policy[1] : 1) Government spending is limited by taxation and borrowing 2) With Government Debts we are leaving our debt burden to our children 3) Government budget deficits take away savings 4) Social Security is broken. 5) The trade deficit is an unsustainable imbalance that takes away jobs and output. 6)We need savings to provide the funds for investment. 7)It's a bad thing that higher deficits today mean higher taxes tomorrow.

[1] From Mosler's book [http://moslereconomics.com/wp- content/powerpoints/7DIF.pdf](http://moslereconomics.com/wp- content/powerpoints/7DIF.pdf)

</comment><comment><USER8> Sounds like they're doing something closer to URVs [1] than Bitcoin.

[1]: [http://en.wikipedia.org/wiki/Unidade_real_de_valor](http://en.wikipedia.org/wiki/Unidade_real_de_valor)

<comment><USER9> At the same time, here in Argentina we invented the Convertibility: "One Peso equal to One Dollar" [http://en.wikipedia.org/wiki/Argentine_Currency_Board](http://en.wikipedia.org/wiki/Argentine_Currency_Board) The main difference is that here the pesos were always "actual" bills, but the real started as "imaginary" bills.

It worked initially (better than I expected), but we have an inflation that was 1% or 2% over the dollar inflation. So after 10 years we have a 20% difference in price, but we maintained the "One Peso equal to One Dollar" until it exploded. IIRC in Brazil the transition from 1-1 to 1-X was softer.

</comment></comment><comment><USER10> I've only been to Ecuador once, but it seemed like people in general didn't trust the government and thought it was corrupt. If that is the case, Ecuador's problems wouldn't be monetary or fiscal at all, it'd be that they have a bunch of crooks robbing them blind.

It always seems like it's my lay-about, heavy-drinking, multiple-DUI- offending, chain-smoking, new-car-on-lease-every-two-years cousins who have the most trouble with money. I once saw one of them chug the leftover half of a bottle of whiskey, smash it over the back of his truck (oblivious to the damage he did to the tail light) and scream, "That's how we DO!" Yeah, you have no idea how truthfully you speak, you idiot. And those three kids by two women you had out of wedlock are the ones who suffer the most for it. But "live fast, die young, and leave a beautiful corpse," amiright!?

If you don't get the analogy, my cousin is corrupt government, the kids are the people they govern. I'm not in general calling Ecuadorians drunken layabouts. And I'm not saying I specifically know this is the problem. I'm just saying, money is a communication of value, so if you have money problems, you most likely have value problems.

</comment><comment><USER11> Perhaps a smarter idea for Ecuador is to take all of the USD they're circulating and convert it into an existing and proven cryptocurrency. While becoming the best place in the world for a unhindered exchange of USD --> BTC, they'd also not have to worry about all of the problems of building and administering a centralized virtual currency from scratch.

<comment><USER12> Imagine you buy a house worth 100k one day in bitcoin. The next day, bitcoin increases its value 25%. That same house is now worth 75k, but you still owe 100k plus interest.

This is one of the biggest flaws in bitcoin that people just can't seem to understand (or just don't _want_ to understand). You can't have a good loaning system with an unstable currency.

Edit: I posted a better explanation below:

"You take out a 100k mortgage in USD. Over the next year, the USD suffers suffers from extreme deflation (increases in value). It is now worth twice what is was worth last year. As a consequence, wages begin to go down. Now let's say you were earning 50k a year while paying down a 100k mortgage (not assuming a downpayment, this is for simplification). Next year, you are now being payed 25k but still owe that 100k mortgage. And the house is now worth 50k."

<comment><USER13> I don't understand. Did you buy a house in BTC or in USD? If you bought it in USD, then the value of it in USD hasn't changed. If you bought it in BTC, then your debt is in BTC. What am I missing?

<comment><USER12> If you bought it in BTC, the house is now much more expensive then when you originally bought it.

<comment><USER13> It's more expensive in USD, yes. Which is a problem if you're being paid in USD, instead of BTC. In other words: if your debt is not in the same currency as your income, that might be a problem. Is that what you're saying?

</comment></comment></comment></comment></comment><comment><USER14> > A monetary authority will be established to regulate the money

I suppose I should not have been so excited. The title really does read a "virtual" currency and not a cryptography-backed cryptocurrency.

This isn't what it sounded like, it's just like Paypal except the government is in charge.

<comment><USER15> You can have a cypto currency controlled by a central bank. In fact, we'll probably have them eventually. It would have a lot of the benefits of a cypto, like instantaneous transfer, without the drawbacks like major volatility.

</comment></comment><comment><USER16> Oh, boy.

I have a house in Ecuador. I am a fan of Bitcoin. And I am, on the whole, probably still a fan of Correa.

But ... yikes.

</comment><comment><USER17> 51% attack, anyone?

<comment><USER2> The article is scant on details the actual currency, but it doesn't sound like they are intending to make it decentralised nor a cryptocurrency.</comment></comment>


Designing with White Space: Why 1+1=3 - <AUTHOR> http://www.kadavy.net/blog/posts/whitespace-113/

<comment><USER2> A decent, introductory write-up about white space. It's a bit difficult to take seriously since the design of the blog doesn't use whitespace effectively. Of course, "The shoemaker's children are often shoeless", and all that.

Irrelevant aside: 'dogsfurry' can work. As someone who spends a lot of time with marketing firms and branding agencies, dogsfurry is not an inconceivable label for a high-end pet supplies company.

<comment><AUTHOR> > the design of the blog doesn't use whitespace effectively.

Could you be more specific? My blog definitely needs a design refresh (your quote is right – I've been writing a book the past year so haven't had the time to design), but I'm sure you don't mean that the entire blog design fails to use white space effectively :)

<comment><USER3> First off, thanks for writing Design for Hackers.

What distracted me when I was reading this post is that many parts of the page appear to be improperly aligned: <http://i.imgur.com/77bVj.png>

</comment></comment></comment><comment><USER4> I could hardly believe that the author of the book designed that blog.. I mean, who would like to buy a book _about_ design with such an ugly look and feel and 1990 taste. Ironically, even the white spaces are wrong :-/ IMO, it's way better to keep things simple and avoid ugly graphics and colors if you're not good enough to use them correctly.

</comment><comment><USER5> Weirdly, with the table designs, he switches to using italics for the %s without discussion.

I assume that's because, in the absence of a dividing line, it provides an additional hint that the two blocks are separate? That seems to weaken his point about the importance white space for this example, though...

<comment><AUTHOR> This post is based upon a part of my book, in which I do mention the italics, but I left it out on the first run. I've now edited to post so I do mention it. Thanks for noticing / commenting!

</comment></comment><comment><USER6> [http://illusion.scene360.com/wp- content/themes/sahara-10/sub...](http://illusion.scene360.com/wp- content/themes/sahara-10/submissions/american-architects-logo.jpg) this is why negative space is important. It can create unintended things if you are not aware that it exists.

</comment><comment><USER7> I don't really get how "1+1=3" is related here. With a proper grammar, such as the one used in infix arithmetic, whitespace is unnecessary. (and missing deliminators in postfix or prefix for 1+1 wouldn't yield three, just an error, or 11 if there's a unary + operator defined.

</comment><comment><USER8> For a website that is concerned with design matters, it is odd that it uses @font-face typeface that makes it virtually unreadable on Windows.

<http://i51.tinypic.com/1jusfs.png>

<comment><USER9> I'm not sure what part of that screenshot is supposed to be unreadable?</comment></comment>


Show HN: My blooper reel tribute to other YC S10 applicants - <AUTHOR> http://youtube.com/watch?v=m0ouOkNeBS4

<comment><USER2> Cool, thanks for sharing.</comment>


The golden age of computer user groups - <AUTHOR> https://arstechnica.com/information-technology/2020/08/the-golden-age-of-computer-user-groups? <comment><USER2> I was a kid in the 80s and 90s, and computer user groups gave me a unique and somewhat unusual community where I got to interact with people much older than I was, and who worked in varied professions (there were lawyers, engineers, consultants, etc.). We all bonded over the fact that we were all computer nuts, which was a fairly rare demographic back then.

I think having that community helped to grow up quickly and gave me a view of world that few of my peers had. I had many interesting peer relationships (through email conversations) with adults who weren't my parents. This sounds like a red flag these days, but those days it wasn't because it was a closed- knit moderated community: we would chat about DOS, OS/2, Gravis Ultrasounds, Macintoshes, etc. I was only a kid but people would treat me like an adult.

I know of no other environment where kids and adults could interact as equals (more or less). I say that because in the computer world, competence is not always correlated with age. The sysop of one of the BBS's I hung out at was a brilliant computer guy and it was only when we met in real-life I found out he was only 15.

p.s. I don't know if anyone remembers dialing into a BBS using software like QmodemPro, downloading a mail packet (usually QWK -- a compressed file with unread emails), replying to emails offline using a reader like Bluewave or Offline Express, and then logging in to upload replies. Those were the days.

<comment><USER3> As a 1980s kid, I had similar experiences. Some bright spark in my northern England smalltown had the idea of bringing together all the families and professionals and their 8-bit machines and having them all set up on a bunch of trestle tables in some old church hall or scout hut. It was wild. I remember seeing a Macintosh for the first time and freaking out with excitement.

There were also genuine red flags though in a less well supervised setting. Some dodgy bloke with a little shop in the back of a rundown mall created a "community" of computer enthusiasts which turned out to be a front for some extremely unsavoury "grooming" type activities.

</comment></comment><comment><USER4> Fun read. Some of the local tech Meetups I've been to, particularly ones discussing open source, capture the spirit of user groups.

I don't think the internet can completely replace the tight knit communities that are created from in person gatherings.

</comment><comment><USER5> There are still user groups around, plenty in fact.

[https://lug.org.uk/](https://lug.org.uk/)

</comment><comment><USER6> Our Chicago area group (APCU) is down to about 50 members, and we meet virtually thanks to the mismanagement of the plague here in the US, last time was a Zoom call.

The hardware has gotten reliable, and even the software isn't so cringy as it used to be. Gone are the days when we started each meeting with "Microsoft Bashing" just to get it out of the way.

</comment><comment><USER7> Surprised that A.P.P.L.E. didn't get a mention. They were influential enough for me to be aware of them a a high school student halfway across the country in Chicago.

[https://en.wikipedia.org/wiki/Apple_Pugetsound_Program_Libra...](https://en.wikipedia.org/wiki/Apple_Pugetsound_Program_Library_Exchange)

<comment><USER8> The most common response (or at least gentle complaint) I've gotten was, "Why did you leave out...?" From DECUS (one of the oldest groups) to Melbourne PC User Group (one of the largest), people have been sad that their favorite wasn't included.

But the story isn't about the groups' size or age. It was about the experience and what they meant. The whole point is that these were worldwide, in tiny towns (like my own Deer Isle Maine) and in huge cities.

</comment></comment><comment><USER9> Fond memories of the old Commodore 64 group that I attended back in the '80s

</comment><comment><USER10> I learned a lot, thanks for posting!

</comment><comment><USER11> This reminds me of MUPTs</comment>


Unroll.me is sorry-not-sorry it sold email data to Uber - <AUTHOR> https://www.engadget.com/2017/04/24/unroll-me-pseudo-apology-for-uber-deal/ <comment><USER2> Discussed heavily yesterday:

[https://news.ycombinator.com/item?id=14181152](https://news.ycombinator.com/item?id=14181152)

</comment><comment><USER3> "Our users are the heart of our company and service." What did you think they were doing? Kindly rolling your emails up for you every day for free?

<comment><USER4> Perhaps someone kindly asked them and they couldn't say no?</comment></comment>


Birth of A Botfly Maggot: A Curious Adventure with "Doctor Bugs" - <AUTHOR> http://www.youtube.com/watch?v=h_JarbNaQ10

<comment><AUTHOR> Also a botfly in a woman's head: <http://www.youtube.com/watch?v=KNDG7WPtVO4>

<comment><USER2> This comment from Youtube is spot on... "WHY CAN'T I STOP WATCHING BOT FLY VIDEOS!?!?!"</comment></comment>


Game startup to feature figurines (Will create your WoW Avatar) - <AUTHOR> http://seattletimes.nwsource.com/html/brierdudley/2004055519_brier06.html <comment><USER2> They're coming out with avatar figurines for Rock Band (the new sequel to Guitar Hero). Saw one the other day, and it looked just great -- bigger than the WoW ones, and cheaper, too.

3D printing has come a long way, and this is just the beginning!</comment>


McNugget Calculator - <AUTHOR> https://heavenfox.github.io/mcnugget/ <comment><USER4> Fun fact: each nugget costs $0.06 to make (excluding overhead). [1] [2]

So even though it's $5 for 20 which seems cheap... it's still only $1.20 in ingredients.

[1] [https://www.quora.com/What-is-the-profit-margin-on- McDonalds...](https://www.quora.com/What-is-the-profit-margin-on-McDonalds- Chicken-Nuggets)

[2] [https://imgur.com/gallery/CvHqp6V](https://imgur.com/gallery/CvHqp6V)

<comment><USER5> I recall famous chef / restauranteur, possibly Gordon Ramsay, giving a bare minimum sale price of 4x the raw ingredient price for profitability. Assuming the $0.06 estimate, $1.20 would imply a price of at least $4.80, pretty close to the $5.00 price

<comment><USER6> But McDonalds has much smaller overheads than other restaurants because the food is largely prepared and the prep work that does need to be done has been engineered to be easy

<comment><USER7> Uh no. The average franchise takes home 10% or so of sales before tax.

<comment><USER6> 3-5% is the average profit margin for a restaurant. Also, the profit for the franchise owner is not the same as the total profit. McDonalds as a corporation is charging the franchisee licensing fees (and often rent as well), which should also be considered part of the profit.

Source: [https://pos.toasttab.com/blog/average-restaurant-profit- marg...](https://pos.toasttab.com/blog/average-restaurant-profit-margin)

<comment><USER7> But that profit doesn't accrue to the franchisee, who is the one paying for equipment and labor. Franchise fees are a cost to him/her.

</comment></comment></comment></comment></comment></comment><comment><USER8> For me the rule of thumb here in SF is: even though I may only want 10 McNuggets, if there's anyone hanging around outside who looks like they'd appreciate some, get the extra 10 (for 2¢/each!) to give away. (It comes in 2 boxes.)

<comment><USER9> And that is exactly what they would like you to do. If you buy meat (or anything for that matter), mind your footprint, only buy what you actually need.

<comment><USER8> Is 'they' McDonalds, or the people hoping for a handout outside?

Why can't I buy what somebody else needs? (The recipients seem happy to receive the food, and though they'd probably prefer cash, I doubt they'd prefer just 20¢.)

<comment><USER10> Think of the ecological footprint! Let them starve.

<comment><USER11> There are other and better ways to stop them from starving. But muh freedom.

<comment><USER12> Taking "giving immediate food to someone hungry and asking for food" as the baseline, what process improvement would you suggest over the $0.21-for-10-nuggets expenditure?

</comment></comment></comment></comment></comment></comment><comment><USER13> I thought this would be about what quantities of nuggets you need to buy to sum to the target number.

[http://mathworld.wolfram.com/McNuggetNumber.html](http://mathworld.wolfram.com/McNuggetNumber.html)

<comment><USER14> Or what the Frobeniusnumber is for given box sizes (ie, the largest number of McNuggets that you cannot buy (exactly). For the classical sizes of {6,9,20} it's 43).

<comment><USER15> Here's a Numberphile video on it. [https://www.youtube.com/watch?v=vNTSugyS038](https://www.youtube.com/watch?v=vNTSugyS038)

</comment></comment></comment><comment><USER16> The calculator isn't global friendly :)

Here McNuggets arrives in the following packaging: 4, 5, 9, 12, 24

<comment> bash-j In my part of Australia it's: 3: $3 553kJ 6: $5.95 1110kJ 10: $8.30 1840kJ 20: $13.35 3690kJ 24: $9.95 3690kJ 10:30AM - <USER3>

Chicken bites (similar to popcorn chicken): 10 for $2 734kJ 10AM - <USER3>

<comment><USER17> Are your nuggets rated in kilojoules?

<comment><USER18> Most of the world uses kilojoules instead of calories.

<comment><USER17> Yea, which is weird because calorie is already metric.

</comment></comment></comment></comment></comment><comment><USER19> This makes me happy. I don't even eat meat but it's a charming project.

<comment><USER20> You're not eating meat even if you eat McNuggets /s.

<comment><USER21> You kid, but I recently found out that the Jack in the Box "2 tacos" are actually meatless. (I think... it was hard to get a straight answer out of the person I spoke to, but so far as I can tell they both "do not contain meat" and "are not vegetarian")

<comment><USER22> I think they do contain meat, just really processed [1]:

>Filling Ingredients: Beef, Chicken, Water, Textured Vegetable Protein

[1] [http://assets.jackinthebox.com/pdf_attachment_settings/108/v...](http://assets.jackinthebox.com/pdf_attachment_settings/108/value/Ingredients_and_Allergens.pdf)

<comment><USER23> Jack in the box is actually a subversive vegan activist organization.

<comment><USER24> I don't actually mind vegan food (a lot of Thai/Chinese dishes with tofu are very good), but meat substitutes are weak sauce IMHO. Don't try to make a better walkman - invent the ipod. Which in this analogy is Mapo tofu:

[https://en.wikipedia.org/wiki/Mapo_tofu](https://en.wikipedia.org/wiki/Mapo_tofu)

</comment></comment></comment></comment></comment></comment><comment><USER25> I do a similar cost comparison when eating pizza by determining how much bigger (more square area) the next size up is. If all you have is a non- scientific calculator, the trick was to divide the square of the radius of each pizza (since you are looking for the ratio of area, pi*r^2, and pi drops out). For example, if comparing 16" and 12", you just evaluate 8×8 / 6×6 = 1.7x bigger. Usually cost of the 16" is much lower than 1.7 the 12" so might as well get the larger size (and leftovers are fine).

<comment><USER26> It can be false economy of course because, leftovers or no leftovers, there's a strong possibility that'll you'll just eat more food than you would have otherwise. Food economy isn't a zero sum game.

<comment><USER25> I think in an economic sense I estimate the amount of utilization more pizza will give me, and once I know how much costlier the next size will be per square inch, only then I conclude if it's worth it or if "I'm not really hungry enough to pay _that_ much!"

</comment></comment></comment><comment><USER27> This is basically the knapsack problem right?

[https://en.m.wikipedia.org/wiki/Knapsack_problem](https://en.m.wikipedia.org/wiki/Knapsack_problem)

<comment><USER28> No. Linear optimization.

Edit: Sorry. It's linear optimization with integers and therefore in the same complexity class as any other np complete problem.

<comment><USER28> For anybody reading this: IN THEORY

In practice the nugget per package gets cheaper as the package size increases and therefore a greedy algorithm is very useful and probably linear in the number of package types.

</comment></comment></comment><comment><USER29> Wow, I wish 40 nuggets cost 10 bucks here. It's $25 CAD ($18.56 USD)

<comment><USER30> In Canada that's only possible at Burger King.

</comment></comment><comment><USER31> Im upset. Can't use this in Germany. We don't have 4pcs. And 10pcs is a 9pcs over here.

</comment><comment><USER32> Will use this calculator should McDonald's ever switch back to the original McNugget recipe.

<comment><USER33> What's the original recipe and when did it change?

<comment><USER34> They changed from the assorted chicken parts (aka pink slime) to an all white meat recipe (aka tastes like nothing) in the early 2000's. They've refined the white meat recipe since then to remove chicken skin and artificial preservatives.

I had occasion to eat at a McDonalds in India in 2010 or so, and they still had the old recipe McNuggets and it was so much tastier.

</comment></comment></comment><comment><USER35> "nuggests"

<comment><USER36> That's the first thing I saw on the site too haha.

</comment></comment><comment><USER37> It doesn't break much when you use the McDelivery prices in Korea: 4: 2100 6: 3600 10: 5100 20: 8200

<comment><USER38> It used to be pink slime. I guess each to their own.

</comment></comment><comment><USER39> This would be a nice test assignment for recruiting

</comment><comment><USER40> It's sad that a brain this bright is being wasted on a McNugget Calculator

</comment><comment><USER41> aka a calculator

</comment><comment><USER42> The real question: Why are you eating at McDonalds?</comment>


Show HN: Materia – Backend builder to create web and mobile application faster - <AUTHOR> https://getmateria.com/?hn <comment><AUTHOR> Hello, I'm Thibaud, Founder & CEO of Webshell.

We've made Materia to help developers to build apps a lot faster. We think most backends can be a set of bricks that could be arranged to speed up the development process of your apps and let you focus on the core functionalities of your company. Materia is client agnostic which mean you can build of it for any client: web, mobile or desktop application (or anything that can make HTTP calls to communicate with your server).

I would love to help and answer any questions you may have. Anyway, all feedbacks are highly appreciated!</comment>


Tesla falls on Model 3 production delay, but analysts upbeat - <AUTHOR> https://www.reuters.com/article/us-tesla-production-stocks/tesla-falls-on-model-3-production-delay-but-analysts-upbeat-idUSKBN1ET1FW <comment><USER2> Reading this, I had a thought come to mind that Tesla's brand might not be very valuable. I think everyone wants a Tesla because not many people have a Tesla. It's exclusive. But if I start seeing Teslas all over the place, I'm not going to be very interested in one anymore. It loses the mystique. And it used to be that electric cars weren't good looking, but I think that will improve as well. A Tesla will only have an advantage to the extent it has better features, but I if GM, Volvo, etc. do a decent job with self-driving cars, I don't see how Tesla can justify it's current valuation.

<comment><USER3> Like the iPhone?</comment></comment>


Thought-powered helicopter takes off - <AUTHOR> http://www.bbc.co.uk/news/science-environment-22774499

<comment><USER2> And here is how to do it yourself: [http://www.instructables.com/id/Brain- Controlled-RC-Helicopt...](http://www.instructables.com/id/Brain-Controlled- RC-Helicopter/)</comment>


Dan Grover – Bots won't replace apps. Better apps will replace apps - <AUTHOR> http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps.html <comment><USER2> Recent: [https://news.ycombinator.com/item?id=11538355](https://news.ycombinator.com/item?id=11538355)</comment>


Replacing Celery with Elixir in a Python Project - <AUTHOR> https://klibert.pl/statics/python-and-elixir/ <comment><USER3> Author of Celery here. This is an interesting presentation!

I want to clarify something about Celery and RAM usage.

When writing web crawlers, and other (mostly) I/O-bound tasks, you should be using the eventlet/gevent execution pools instead of the multiprocessing one. This will _drastically_ reduce memory use, and perform better.

If you have four CPU cores you can start four worker instances with 1000 threads each (for a total of 4k threads): `celery multi start 4 -A proj -P gevent -c 1000`

This will utilize all the CPU/cores in your system, working around the GIL.

One of the new features coming in Celery 4 is a message protocol with support for multiple languages, maybe we could have an Elixir worker soon.

<comment><AUTHOR> Hi, thanks for reading. I need to stress that this is just an example project I came up with _after_ I decided to play with Elixir and Python integration. The project itself doesn't even make much sense - as you and others point out, there are many other, more pythonic ways of handling this task. I chose web scraping because it was easy to split into two subtasks: one IO and one CPU bound.

> The slides mention problems with Celery and RAM usage when writing crawlers, > but since this is a mostly I/O-bound task you should be using the > eventlet/gevent execution pools instead of the multiprocessing one.

Here: [https://klibert.pl/statics/python-and- elixir/#/5/6](https://klibert.pl/statics/python-and-elixir/#/5/6) in line 18 you can see a `_do_some_real_processing_function`. The whole premise of the project is that this function is CPU-bound. My processor has four cores, so I create a pool of 4 Python processes ([https://klibert.pl/statics/python-and- elixir/#/5/2](https://klibert.pl/statics/python-and-elixir/#/5/2) line 10).

Fetching pages is an IO-bound task, so it's done by Elixir. There we have a pool (for rate limiting) of 10 processes (the Erlang ones - <USER2> distinction) that do the downloading.

I think the closest analogy to what happens in this project is a Twisted (EDIT: or any other concurrent, but not parallel framework) app which uses a pool of processes for CPU-bound tasks. Here the Twisted part is replaced with Elixir.

EDIT: Also, we use Celery at work extensively and it works great and there is no real need to replace it with anything! Again, this project is just a tech demo, it doesn't make (much) sense on it's own. But there are other possible integration patterns where Elixir and Python have different roles, which actually do make sense. I think.

<comment><USER3> I found your project very interesting! I'm very familiar with Erlang/OTP, and have been meaning to play with Elixir for some time. Since you're separating downloading from processing, maybe IPC is a better term for what you're doing, as you don't want these two steps to be separated over the network for data locality.

I only wanted to clarify the RAM situation described in your slides, as it's not widely known that you can use eventlet/gevent with Celery.

</comment></comment></comment><comment><USER4> Am I the only one not liking the slides having both vertical and horizonzal navigation? Going through the whole presentation requires going in the right direction (down unless last slide in section, in which case it is right...)

<comment><AUTHOR> You can use Space, which should take you to the correct next slide, without thinking about directions. I found it a neat format to structure a talk into segments. Also, you can press Esc and get a "map" of the slides.

<comment><USER5> Nope. I used space. After the Erlang slide it moves to the right "tools used", which is the slide on the same level in the next column.

This along with back button hijacking makes for the single worst presentation format I've encountered so far.

EDIT: it appears that only happens when you have been in that column before. This is still pretty surprising behavior.

<comment><AUTHOR> Sorry about that. Maybe some of this behavior is configurable in Reveal.js, I'd need to check.

It was fine for me because these are the slides for a talk and I was the one showing them to people.

I'd be happy to post the video of the talk ([https://www.facebook.com/events/211449562541292/](https://www.facebook.com/events/211449562541292/)), but after three months I still haven't heard anything from the people behind the event...

<comment><USER5> No reason to apologize. The content was definitely still viewable.

</comment></comment></comment></comment></comment><comment><USER6> Interesting article!

I would just like to point out that concurrent downloads can be handled much more efficiently in Python > 3.4 thanks to the asyncio library. For an example, look at Guido van Rossum's crawler [0].

[0]: [https://github.com/aosabook/500lines/tree/master/crawler](https://github.com/aosabook/500lines/tree/master/crawler)

<comment><USER7> Asyncio doesn't support http yet but that could be fixed with a library like aiohttp

<comment><USER8> The linked crawler uses aiohttp.

</comment></comment></comment><comment><USER9> The multiprocessing slide mentions the RAM usage issue with things like Celery (because you start many instances of Python and load in dependencies). Does this solve them?

If so, how does it get around the whole GIL thing and whatnot? Or maybe I'm misunderstanding at what level things are happening?

Is it that you still have one python process but the bottleneck/URL fetching is happening inside your elixir stuff?

Super interested in this, we have this problem with Celery workers and would love to not be bound by RAM for worker count

<comment><AUTHOR> Unfortunately, no, this doesn't solve the problem of RAM usage by worker Python processes: they are still separate OS processes, although they look like "normal"[1] Erlang/Elixir processes from that side.

By "replacing" I mean pretty literally getting the same effect as with Celery. [EDIT: BTW, in the recent post about Celery it was praised because it lets you compose background/async tasks. Of course, Erlang has something like that, too: [https://chrisb.host.cs.st-andrews.ac.uk/skel-test- master/tut...](https://chrisb.host.cs.st-andrews.ac.uk/skel-test- master/tutorial/bin/tutorial.html)] There really is no other reliable way around the GIL in Python other than multiprocessing.

The main idea here is that we should keep IO-bound code on the Elixir side as it's simply more efficient and easier to write there, but to delegate CPU- bound code to a pool of external processes. But this is only one of many possible patterns of integration: I can imagine a Django project where most of the logic is on the Python side and Elixir only handles WebSockets/long polling connections. Thanks to ErlPort passing data and calling functions between Elixir and Python is effortless, you can call an Elixir function just as easily as you can make Elixir call a Python function in a different process.

Moreover, ErlPort supports Ruby, so you can have workers written in it, too. There's no problem with workers in other languages, too, as long as you write the glue code yourself (or generate using Elixir macros).

[1] It bears repeating: Erlang processes _are not_ OS-level processes. Erlang Virtual Machine, BEAM, runs in a single OS-level process. Erlang processes are closer to green-threads or Tasklets as known in Stackless Python. They are extremely lightweight, implicitly scheduled user-space tasks, which share no memory. Erlang schedules its processes on a pool of OS-level threads for optimal utilization of CPU cores, but this is an implementation detail. What's important is that Erlang processes are providing isolation in terms of memory used and error handling, just like OS-level processes. Conceptually both kinds of processes are very similar, but their implementations are nothing alike.

<comment><USER10> If you're having RAM problems, you don't want to start up each processor independently as a "port". You'd want to write a Python server that offers a socket, start one instance of that that listens to the socket and then forks (since you control the system fully, probably let it fork once per connection rather than worrying about "preforking" or other complicated scenarios for when you don't control the concurrency), and then write the glue code to talk across that socket. [1]

That way, your Python processes load all their modules and do all their initialization, and the forking automatically takes care of Copy-On-Write semantics in the RAM and you end up with only more-or-less one copy of them in RAM.

[1]: I don't know if there's a "perfect" off-the-shelf solution for you, but there's enough existing code in the world with all the pieces that it's pretty easy to do that nowadays. For instance, one very easy protocol that leaps to mind is:

        4 bytes to say how much JSON there is
        a JSON object containing the metadata for your connection
        4 or 8 bytes to say how long the incoming webpage is
        the contents of the webpage you're telling Python to process

It's not perfectly efficient, but it has great bang-for-the-buck, and will carry you a long way before you start needing to do anything else fancier.

You might also be able to bash ErlPort into compliance and get the client and server side going, but, well, this isn't very difficult code to write and it doesn't take much "screwing around with opaque library code that doesn't really want to be used separately" before you could have just written this.

</comment></comment></comment><comment><USER11> For something as straightforward as downloading files in parallel, why not just use Python threads? Since most of the IO will occur in C without holding the GIL, it seems silly to be forking off entire new processes for this sole purpose.

The processing of the file in Python would hold the GIL, but this could still be resolved with a multiprocessing pool of workers.

In other words, while academically interesting, mixing Elixir and Python for web crawling doesn't make much actual sense.

<comment><AUTHOR> > In other words, while academically interesting, mixing Elixir and Python for > web crawling doesn't make much actual sense.

That's right. It wasn't supposed to be a real project, it's a just a tech demo of sorts. I will have some more real-world uses for the integration in my Raspberry project, I think, but I'm not there yet.

</comment></comment><comment><USER12> I personally like to bridge Elixir and Python via nanomsg with MessagePack serialization.

Here are some useful libraries:

[http://nanomsg.org/](http://nanomsg.org/)

[https://hex.pm/packages/exns](https://hex.pm/packages/exns)

[https://github.com/walkr/nanoservice](https://github.com/walkr/nanoservice)

</comment><comment><USER13> Can we avoid the overhead of starting and shutting down processes by running a single Python process and communicating using something like grpc [0] (or even JSON-RPC for maximum simplicity)?

How do web frameworks like Flask handle multiple concurrent requests? Would performance increase if we started multiple instances of this Python web server on the same machine and load balanced them? The code would be much simpler if there was no need to handle process management.

[0]: [http://www.grpc.io/](http://www.grpc.io/)

<comment><USER14> The whole point here is we need parallel processing, and Python cannot provide this in a single process due to the GIL. Flask applications depend on the workload being I/O bound, so it can achieve concurrency where parallelism is not required. If you built a process that is CPU bound in Python code in Flask, you'd find it could not achieve much concurrency at all.

<comment><USER15> There are plenty of multiprocessing wsgi servers, flask builtin included

</comment></comment></comment><comment><USER16> I think I misunderstood your presentation at first glance. Elixir "processes" are actually green threads. Thus, you actually have Python interpreters in separate OS processes, right?

</comment><comment><USER17> While BEAM is indeed great I'm wondering why you didn't use Scrapy? It handles concurrency well and is a battle tested production scraper.

<comment><AUTHOR> The same question was asked when I gave the talk. I'm afraid the only answer is: because I could :-) I thought up a project where it would make some sense to use Elixir and went with it. It's not practical or "real world" project at all!

The point is that the integration Python<->Elixir can be so tight and that there's little overhead when using each language for things it's good at.

<comment><USER18> > because I could

Love that!

I will look it over, and because I favor LFE (Lisp Flavored Erlang) over Elixir, try to do something neat like this. Although, Elixir is growing on me from my initial take on it a year ago.

</comment></comment></comment><comment><USER19> Thanks for sharing.

First my complaint: the slides are really annoying :) A traditional left-right stack would've been nicer.

That said, this is something I've been looking at lately as I've got a bunch of python code that I want to parallelize, and a strong interest in Elixir. I found your code samples very helpful.

Edit: I see the other comments mention the slides, and how to navigate them w/ space. Disregard my complaint.

</comment><comment><USER20> OP mentioned that Elixir is better at concurrency and parallelism. Python is better at processing (more libraries/toolsets available).

As someone who wants to use Elixir more and see its community flourish, what libraries does Elixir need so this project could be done without Python?

I'm guessing some sort of Beautiful Soup or Nokogiri equivalent?

<comment><AUTHOR> No, actually, there is _nothing_ lacking in Elixir ecosystem if you want to write a web scraper! That's the point. I used HTTPoison for sane (and efficient) HTTP requests and I could have used (and used in some other projects) Floki ([https://github.com/philss/floki](https://github.com/philss/floki)) for HTML parsing and querying.

However, there are things like generating PDFs with graphs based on the tabular data on the pages or running some more involved Pandas TimeSeries transformations which are simply not available in Elixir. Nor they should be, I think: reportlab or Pandas are already written and do a good job at what they are meant to do. This is the idea: we write a crawler in Elixir and delegate processing to something else. Anything else, in fact: Python was chosen because of ErlPort and how easy it is to integrate, but your workers can in practice be written in anything that understands JSON.

</comment></comment><comment><USER21> Interesting use case but is it actually nice to mix up the technologies on your stack just for this? Also why re-invent a solution for something that already works? Just my 2 concerns :)

</comment><comment><USER22> Fantastic. I recently used Elixir for scraping courses off Udemy. I've put the results of this into a site, [http://www.coursenut.com](http://www.coursenut.com)

Also I've added an Elixir course promotion to:

[http://www.coursenut.com/courses/3692](http://www.coursenut.com/courses/3692)

<comment><USER23> What is the purpose of the scrape?

</comment></comment><comment><USER24> Does this solve the state-sharing difficulties of the python solution?

</comment><comment><USER25> This is a cool way to use both tools together.

</comment><comment><USER26> I have been looking at exactly this type of solution, as I want to use Elixir for distribution of python(numpy)-based computations to clients. Just a quick question.... is ErlPort maintained? It looks like it's been in alpha for 3 years...

<comment><AUTHOR> There's little development, it seems, but it's not dead. There are discussions below issues opened, PRs are accepted, occasional commit sometimes lands in master.

It was stable when I used it. But it's also rather simple and short (less than ~1k loc on Python side, for example), so I think it wouldn't be a heavy burden to maintain, even if the current maintainers and users all dropped dead tomorrow :-)

<comment><USER26> enough for me. I'm going to use this because the combo of Python (more accuratey, _numpy and pandas_ ), plus Elixir, is pure goodness.</comment></comment></comment>


A link back to the HN front page on the dead link page? - <AUTHOR>

The "Unknown or expired link" page should redirect or link back to the HN front page. Adding this would make the site navigation experience more fluid.
Right now, this happens pretty often: 1 Click the "Next" button 2 Uh-oh, expired link 3 Hit back 4 Refresh 5 Click "Next" again
If we added a link or redirect, the process becomes a lot simpler: 1 Click the "Next" button 2 Back to main page 3 Click "Next" again
Furthermore, people who've been linked to expired URLS get a handy way back to the main page. Best of all, there's no need to rework any of the existing framework -- all you have to do is change deadlink! <comment><USER2> I submitted a patch for that about 16 months ago, I assume the behavior is intentional or desirable.

Depending on the day, I can get through 2-3 pages of /newest before getting that. I probably miss quite a few decent posts that were entered overnight.

More annoying is typing a response to a comment, only to have it lose the session, you hit back and sometimes the browser tells you to refresh, losing your post. I've gotten burned on that on a few longer replies so, I figure the site prefers more concise answers.

</comment><comment><USER3> OR, fix pagination so it works when you click on the link at the bottom, no matter how slowly and thoughtfully you go through the articles.</comment>


50 Data Science Applications Area and Uses - <AUTHOR> http://www.dataanalyticsnews.com/50-data-scie-applications/ <comment><AUTHOR> Top 50 industry area where data science is used.</comment>


Skylake now GA on GCE - <AUTHOR> https://cloudplatform.googleblog.com/2017/05/Compute-Engine-updates-bring-Skylake-GA-Extended-Memory-and-more-VM-flexibility.html <comment><USER2> The Google Compute Engine is really easy to use, and the browser console makes it easy to treat it like a Linode or Digital Ocean type host, if that's all you want.

I recently transitioned off of Linode (after about a decade?!) to Google. Largely, this was because I needed a lot more disk space to mirror some large public databases, and it was significantly cheaper to do this with Google than it would have been with Linode. This is because Linode couples the disk space with both the RAM and the number of processors. I needed more disk, and would have liked more RAM, but had no use for more processors.

With this announcement, Google has now decoupled the RAM from the number of processors, which is a development I'll be taking advantage of.

<comment><USER3> >"The Google Compute Engine is really easy to use, and the browser console makes it easy to treat it like a Linode or Digital Ocean type host, if that's all you want."

I agree, the browser console is really fantastic. Does anyone know how this is implemented?

<comment><USER4> [https://github.com/yudai/gotty](https://github.com/yudai/gotty) is a good place to start figuring out how they're implemented (tty streaming server + termjs type thing).

</comment></comment></comment><comment><AUTHOR> I want to point out that a bunch of things are rolled into this announcement:

\- Skylake is now generally available

\- Broadwell is now globally visible

\- "Extended Memory" custom machine types are now in Beta

I decided to focus on Skylake as GA for the title, but depending on your interest these other announcements may be good to know as well. In particular, this changes the historical "us-central1-a is the oldest zone" into "us- central1-a is a better choice than 1-f".

Disclaimer: I work on Google Cloud (and want to sell you cores)

[Edited: I always forget newlines]

<comment><USER5> Whoever is setting your bandwidth pricing is preventing me from buying your cores.

N.B.: My use case is batch jobs that have a significant amount of output bytes, so I don't care about the "quality" of the bandwidth.

And for those who aren't aware, Google is charging about two _orders of magnitude_ more than market rates for Internet egress.

<comment><USER6> For comparison, bandwidth that costs me if I directly peer or rent from a European hoster 20$ a month costs at GCP over 900$ a month. With Firebase, it'd be a major 5-digit sum even.

<comment><USER7> So it would be a huge no no if I have part of my data pipeline in GCE with rest in AWS.

</comment></comment></comment></comment><comment><USER8> I love a couple of inexplicable acronyms in the morning.

EDIT - mods - any chance of a title edit? Search engines would appreciate that as much as humans.

<comment><USER9> Speaking of search engines: why don't you just look up acronyms you don't know? The one-time inconvenience would seem to be much better than denying everyone the benefit of using acronyms (as demonstrated by their use). And what's the standard for acceptable acronyms? Is it "GA" or "GCE" you don't know? What about GPU, CPU, FLOP, CIA? And why only acronyms? Surely, some people will have to figure out what "Skylake" is, as well. And considering nobody is born speaking English, should the title include some sort of explanation of "now" that starts from first principle?

<comment><USER8> It's about good writing and judging your audience.

You can safely bet anyone here will know what CPU stands for. But this title is just badly written and lazy. I actually managed to figure it out after a couple of seconds of staring but the point is - _I shouldn 't have to_.

</comment></comment></comment><comment><USER10> I really love Google's UI, it's miles ahead of AWS (though kudos on Lightsail, they nailed that one), but last time I tried porting my toy projects, I was stopped by Google's requirement to be a registered business. Is that still a limitation, or can anyone sign up these days?

<comment><USER11> That unfortunate limitation applies only to European customers. The reason for it is that they don't charge VAT and customers are responsible for their own tax liability, which wouldn't fly if they were selling to consumers instead of to businesses.

</comment></comment><comment><USER12> Beaten Amazon to it, then. c5s are still mia.

<comment><AUTHOR> Yeah, it's one of the benefits of being attached to "the rest of Google". As with Broadwell, Google was front and center in helping Intel qualify this part. The distinction is that with Skylake the lead time over general availability was even longer (see our announcement of this at Supercomputing in November).

We still have some things on Skylake we're going to do beyond today's announcement, so today is really "Welcome!".

Disclaimer: I work on Google Cloud.

</comment></comment><comment><USER13> Any idea when these will be available for GKE users? I would love to spin up a node pool to compare but it seems unavailable:

      $ gcloud container node-pools create skylake --cluster montana --image-type gci --machine-type n1-standard-4 --zone us-central1-b --min-cpu-platform "Intel Skylake"
      ERROR: (gcloud.container.node-pools.create) unrecognized
      arguments:
        --min-cpu-platform
        Intel Skylake

<comment><USER14> I cant use skylake using the console.cloud.google.com website either.

<comment><AUTHOR> See my reply (I was at dinner).

<comment><USER14> Thanks for the information and for being so responsive!

</comment></comment></comment></comment><comment><USER15> With all the craziness of different CPUs and server configs... just makes me want to hurry up and go serverless!

<comment><AUTHOR> That's why we didn't do this with machine types. If you don't care, you don't have to. If you care, ask for a particular CPU platform (or newer). Most people won't and probably shouldn't care, but Skylake is enough of a jump that we finally are giving people the explicit ability to choose.

Disclosure: I work on Google Cloud.

<comment><USER16> > Skylake is enough of a jump

Can you give more details about this?

<comment><AUTHOR> Both clock-for-clock and even more strongly for applications that can take advantage of AVX-512, Skylake can be a lot faster than either Haswell/Broadwell. Like the jump from Sandybridge/Ivybridge to Haswell/Broadwell, the jump to Skylake is a big one. For now the best comparisons are going to come from independent sources like Anandtech and hopefully from customers running their actual code on Skylake and publishing the results.

<comment><USER17> I mean it is no Kaby Lake ;)

</comment></comment></comment></comment></comment><comment><USER18> The pricing of the new "extended memory" stopped me from using it.(For a committed use case)

Still waiting for the AWS c5 instance: [https://aws.amazon.com/about- aws/whats-new/2016/11/coming-so...](https://aws.amazon.com/about-aws/whats- new/2016/11/coming-soon-amazon-ec2-c5-instances-the-next-generation-of- compute-optimized-instances/)

</comment><comment><USER19> Dear Google,

I know this may be a rare use-case, but is it possible to allow creating VM instances with more than 13GB of memory and only two cores?

I'd like to do 2 cores and 24GB for example.

Also, my project's are not seeing the new UI changes (cpu platform and extend memory), is this still rolling out?

<comment><AUTHOR> Yeah, still rolling out.

As for extended memory, as I clarified elsewhere, while you _can_ make a 2x24 it won't be saving you money compared to having the cores. I forget if this is made clear in the docs, but via the API this is laundered through machine types as custom-2-24000-ext.

</comment></comment><comment><USER14> I dont have the ablity to select cpu type from the GUI. Is that being rolled out gradually?

<comment><AUTHOR> Yeah, like all other rollouts the console rollout is gradual. Sorry about the confusion.

</comment></comment><comment><USER20> Can I modify an existing instance (and reboot to pick up the new cpus), or do I have to build a new instance to get these?

<comment><USER21> Yes, you can do that from the gcloud command line tool. Instructions here [https://cloud.google.com/compute/docs/instances/specify- min-...](https://cloud.google.com/compute/docs/instances/specify-min-cpu- platform)

</comment></comment><comment><USER22> > Skylake now GA on GCE

What now what on what?

<comment><USER23> Intel Skylake Processors now General Availability on Google Compute Engine.

<comment><USER22> > Intel Skylake Processors now General Availability on Google Compute Engine.

That's a great idea for a title of this item :)

</comment></comment></comment><comment><USER24> RAM limit is still too low for big RDBMSes. I guess this is host machine limitsd.

<comment><USER25> 455GB is too low?

</comment></comment><comment><USER26> Does this have AVX-512 available?

<comment><AUTHOR> Yep! Explicitly called out in the blogpost.</comment></comment>