How We Built r/Place - <AUTHOR> https://redditblog.com/2017/04/13/how-we-built-rplace/ <comment><USER3> I love write-ups like this because they are such a nice contrast to the too- common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. Something as superficially trivial as the r/Place requires a tremendous amount of effort to run smoothly and there are countless gotchas and issues that you'd never even begin to consider unless you really tried to implement it yourself. Big thanks to Reddit for the fun experiment and for sharing with us some of the engineering lessons behind it.

<comment><USER4> > the too-common comments on Reddit and HN where people claim that they could > rebuild FB or Uber as a side project.

I do and don't agree with you. Whats really going on here is that development time scales linearly with the number of decisions you need to make. Decisions can take the form of a product questions - "what are we making?" and development questions - "how should we implement that?".

There are three reasons why people feel confident saying "I could copy that in a weekend":

\- When looking at an existing thing (like r/place), most of the product decisions have already been made. You don't need to re-make them.

\- If you have a lot of development experience in a given domain, you don't need to make as many decisions to come up with a good structure - <USER2> intuition (tuned by experience) will do the hard work for you.

\- For most products, most of the actual work is in little features and polish thats 'below the water', and not at all obvious to an outsider. Check out this 'should we send a notification in slack' flowchart: [https://twitter.com/mathowie/status/837807513982525440?lang=...](https://twitter.com/mathowie/status/837807513982525440?lang=en) . I'm sure Uber and Facebook have hundreds of little features like that. When people say "I could copy that in a weekend", they're usually only thinking about the core functionality. Not all the polish and testing you'd actually need to launch a real product.

With all that said, I bet I _could_ implement r/place in less than a weekend with the requirements stated at the top of that blog post, so long as I don't need to think about mobile support and notifications. Thats possible not because I'm special, and not because I'm full of it but because most of the hard decisions have already been made. The product decisions are specified in the post (image size, performance requirements) and for the technical decisions I can rely on experience. (I'd do it on top of kafka. Use CQRS, then have caching nodes I could scale out and strong versions using the event stream numbers. Tie the versions into a rendered image URL and use nginx to cache ... Etc.)

But I couldn't implement r/place that quickly if reddit didn't already do all the work _deciding_ on the scope of the problem, and what the end result should look like.

In a sense, I think programming is slow and unpredictable because of the learning we need to constantly do. Programming without needing to learn is just typing; and typing is fast. I rule out doing the mobile notifications not because I think it would be hard, but because I haven't done it before. So I'd need to learn the right way to implement it. - And hence, thats where the scope might blowout for me. Thats where I would have to make decisions, and deciding correctly takes time.

<comment><USER3> First off, it's a bit unfair to me that you were the one to make this comment, since your very extensive experience and deep knowledge of real-time systems makes you uniquely qualified to disprove my point!! hehe

But in all seriousness, I agree with most of what you said - I think I'm just more bearish on people's ability to infer those many decision points without being given the blueprint like we were in this article.

If you are a senior engineer at FB and you decide to make Twitter in your spare time, I buy that lots of experience and knowledge gained at your job can probably get you going fairly quickly. But I have never seen an example of engineers discussing sophisticated systems like these where crucial aspects of its success in terms of implementation didn't rely on some very specific knowledge/study of the particular problem being solved that could only be gleaned after trying things out or studying it very carefully. The representation of the pixels is a great example in this case -- they go into wonderful detail about why they decided to represent it the way they did, which in turn informs and impacts how the rest of the stack looks like.

I think at one point Firebase had it as one of their example apps something which very closely mirrored what they did with r/Place, so I agree that one could probably build some "roughly" like it somewhat quickly. I agree that in general knowledgeable individuals could probably grok things which would in some form resemble popular services we know today. The devil is in the "roughly," though. I think that often what makes them be THE giant services we know are things that let them have the scale which very few of us have ever needed or know how to deal with, or because they have combined tons of "below the water" features and polish like you mentioned. When really basically most web apps that we use are CRUD apps which we all know how to build, I think maybe we need to give more weight to these "below the water" features in terms of how much they actually contribute to the success of the applications we use.

<comment><USER5> > The representation of the pixels is a great example in this case

That's funny because I thought (assuming you're referring to the bit-packing of the pixels) that seemed to me one of the more obvious choices (to do something _else_ would have been more remarkable to me).

Beyond that though, I have zero experience with real-time networked systems. Especially with a gazillion users and that everybody gets to see the same image, that seems _hard_.

The _cleverest_ solution that I read in the article, that I really liked but probably would never have thought of myself (kinda wish I would, though) was the part where the app starts receiving and buffering tile-changes right away, and only _then_ requests the full image state, which will always be somewhat stale because of its size, but it has a timestamp so they can use the buffered tile-changes to quickly bring its state up to time=now. Maybe this is a common technique in real-time networked systems, I don't know, but it was cool to learn about it.

</comment></comment></comment></comment><comment><USER6> What a fantastic writeup. I had some vague ideas regarding the challenges involved to build an application of such scale, but the article really makes it clear for everyone the amount of decision points encountered as well as why certain solutions were selected.

I also like the way the article is broken down into the backend, API, frontend and mobile. This isolated approach really highlights the different struggles each aspects of the product has, while dealing with what is essentially a shared concern: performance.

What I also found interesting is the fact that they were able to come up with a pretty accurate guess in terms of the expected traffic.

> "We experienced a maximum tile placement rate of almost 200/s. This was > below our calculated maximum rate of 333/s (average of 100,000 users placing > a tile every 5 minutes)."

Their guess ended up being a good amount above the actual maximum usage, but it was probably padded against the worst case scenario. The company that I work for consistently fails to come up with accurate guesses even with our very rigid user base, so it's pretty impressive that Reddit could accommodate the unpredictable user base that is the entire Reddit community.

<comment><USER7> Agreed, great write-up. Anyone have other recommended links to write-ups about specific challenges and how to make them scale?

<comment><USER8> Here are some recent ones that have been popular:

[http://highscalability.com/blog/2016/4/20/how-twitter- handle...](http://highscalability.com/blog/2016/4/20/how-twitter- handles-3000-images-per-second.html)

[https://redditblog.com/2017/1/17/caching-at- reddit/](https://redditblog.com/2017/1/17/caching-at-reddit/)

[https://blog.twitter.com/2017/the-infrastructure-behind- twit...](https://blog.twitter.com/2017/the-infrastructure-behind-twitter- scale)

[https://www.netlify.com/blog/2017/03/16/smashing-magazine- ju...](https://www.netlify.com/blog/2017/03/16/smashing-magazine-just- got-10x-faster/)

[https://nickcraver.com/blog/2016/02/03/stack-overflow-a- tech...](https://nickcraver.com/blog/2016/02/03/stack-overflow-a-technical- deconstruction/)

[https://engineering.linkedin.com/blog/2016/10/instant- messag...](https://engineering.linkedin.com/blog/2016/10/instant-messaging-at- linkedin--scaling-to-hundreds-of-thousands-)

[http://techblog.netflix.com/2016/08/building- fastcom.html](http://techblog.netflix.com/2016/08/building-fastcom.html)

[http://highscalability.com/blog/2016/9/28/how-uber- manages-a...](http://highscalability.com/blog/2016/9/28/how-uber-manages-a- million-writes-per-second-using-mesos-and.html)

Or a recent writeup on a challenge building IMDB's forums in 2001(!) [https://www.beatworm.co.uk/blog/internet/imdb-boards-no- more](https://www.beatworm.co.uk/blog/internet/imdb-boards-no-more) .. fun and scary in equal measures.

(I'm plucking these from things were popular recently in our [http://webopsweekly.com/](http://webopsweekly.com/) :-))

</comment></comment></comment><comment><USER9> > We used our websocket service to publish updates to all the clients.

I used /r/place from a few different browsers with a few different accounts, and they all seemed to have slightly different view of the same pixels. Was I the only one who experienced this problem?

When /r/place experiment was still going, I assumed that they grouped updates in some sort of batches, but now it seems like they intended all users to receive all updates more or less immediately.

<comment><USER10> Yeah, we went into it a bit in the "What We Learned" section, but that was most likely during the time we were having issues with RabbitMQ. I believe it was mostly fixed later on, but either way, we found a new pain point in our system we can now work on.

<comment><USER11> Surprised you're using RabbitMQ. It's one of those things which work great until they don't (clustering is particularly bad), and then you have almost zero insight into the issue, and have to resort to the Pivotal mailing list.

Have you looked at NATS at all? We're using it as a message bus for one app and it's been fantastic. It is, however, an in-memory queue, and the current version cannot replace Rabbit for queues that require durability.

<comment><USER12> i've been using rabbitmq heavily (as in, the whole infrastructure is based on two rabbitmq servers) for a long time and i've never seen it fail.

tbh, i never used clustering (because it's one of the shittiest clustering implementations i've ever seen) but we do use two servers (publishers connect to one randomly and consumers connect to both) and it seems to handle millions of messages without any issues.

of all servers i've ever used, rabbitmq is by far the most stable (together with ejabberd).

<comment><USER13> RabbitMQ is decent if you don't use clustering (which, I agree, is shitty). I have some quibbles with the non-clustered parts, but nothing big.

Right now, the main annoyance is that it's impossible, as far as I understand, to limit its memory usage. You can set a "VM high watermark" and some other things, but beyond that, it will — much like, say, Elasticsearch — use a large amount of mysterious memory that you have no control over. You can't just say "use 1GB and nothing more", which is problematic on Kubernetes where you want to pack things a bit tightly. This happens even if all the queues are marked as durable.

<comment><USER12> yeah we have dedicated machines to rabbitmq because it's basically memory hungry. but i like it that way because it's only going to crash if the machine crashes.

</comment></comment></comment></comment></comment></comment><comment><USER14> I thought it was interesting that one of their requirements was to provide an API that was easy to use for both bots and visualization tools. I remember reading some speculation when this was running that r/place was intentionally easy to interface with bots, while there were also complaints that the whole thing had been taken over by bots near the end.

<comment><USER15> Without bots, I doubt that /r/place would have been very interesting. It's a nice thought that a million random strangers can be cohesive without automation, but for some reason I don't find that to be particularly realistic..

<comment><USER16> As a concrete example, as far as I can tell the entire Puella Magi Madoka Magica section, starting from Homura Did Nothing Wrong next to Darth Plagueis The Wise, was hand-crafted and hand-maintained. On their discord they were actively discouraging community members that wanted to use bots.

</comment></comment></comment><comment><USER17> Now I'm curious, Are there any websites that do something similar to /r/place? (hackathon idea?)

Also, reminds be of the million dollar front page [1].

[1] [https://en.wikipedia.org/wiki/The_Million_Dollar_Homepage](https://en.wikipedia.org/wiki/The_Million_Dollar_Homepage)

<comment><USER18> A long time ago, I built [http://www.ipaidthemost.com/](http://www.ipaidthemost.com/), which is kinda related, at least to TMDH anyhow. Far, far less collaborative than /r/place, but similar in terms of staking out ownership.

<comment><USER19> Hilarious. That's got to be the most direct monetization strategy since tmdhp.

</comment></comment></comment><comment><USER20> Why use Redis and multiple machines instead of keeping it in RAM on a single machine? I'm not claiming the Reddit people did anything wrong; they have a lot more experience than me here obviously. I'm just trying to figure out why they couldn't do something simpler. 333 updates/sec to a 500kB packed array, coupled with cooldown logic, should have a negligible performance cost and can easily be done on a single thread. That thread could interact via atomic channels with the CDN (make a copy of the array 33 times a second and send it away, no problem) and websockets (send updates out via a broadcast channel, other cores re-broadcast to the 100K websockets). Again, I'm not saying this is actually a better idea, this is just what I would do naively and I'm curious where it would fall apart.

<comment><USER21> > keeping it in RAM on a single machine

> 500kB packed array, coupled with cooldown logic, should have a negligible > performance cost and can easily be done on a single thread. That thread > could interact via atomic channels with the CDN

That's not simpler.

We used tools that we're already using heavily in production and are comfortable with.

<comment><USER20> > That's not simpler.

With respect to your experience in the matter, I strongly disagree. What I described is complicated to say, easy to implement. What the OP describes ("use redis") is easy to say, complicated to implement. Not just in terms of human work time (setting up the redis machine and instance, connecting everything together), but also in terms of number of moving parts (more machines, more programs, etc.).

> We used tools that we're already using heavily in production and are > comfortable with.

That's entirely fair, and what I figured was the most likely explanation.

</comment></comment></comment><comment><USER22> > Users can place one tile every 5 minutes, so we must support an average > update rate of 100,000 tiles per 5 minutes (333 updates/s).

It only takes a couple of outliers to bring everything down. I'm not exactly well-versed in defining specs for large scale backend apps (not a back-end engineer) but it seems to me that preparing for the average would not be a wise decision?

For example, designing with an average of a million requests per day in mind would probably fail, since you get most of that traffic during daytime and far more less at the nightly hours.

Could anyone more experienced shed some light?

<comment><USER23> The entire reddit website goes down every night, especially during weekends, sport matches, etc, so there you have your answer.

<comment><USER24> Is that hyperbole or are you really experiencing that much downtime of Reddit? I see it occasionally but it's never down for long, the odd "servers are busy" message usually disappears after a single refresh.

<comment><USER25> The search is consistently broken, but the rest of the site seems to have good uptime now, much better than it once was.

<comment><USER24> Search has never worked. I always use Google site: modifier and have never had an issue.

<comment><USER26> Search worked pretty reliably up until 7-8 months ago for me

</comment></comment></comment></comment></comment></comment><comment><USER27> So I got hit by an unfortunate bug on the first day of /r/place.

I was trying to draw something, one pixel at a time, and all of a sudden, after a bunch of pixels, it stopped rate-limiting me! I could place as many as I wanted! So I just figured that they periodically gave people short bursts where they can do anything. This was backed up by my boss, who was also playing with /r/place, saying that the same thing happened to him not long before that (yes, my whole team at work was preoccupied with /r/place that Friday). So I quickly rushed to finish my drawing.

And then I reloaded my browser... and it wasn't there. Turns out that what I thought was a short burst of no rate limiting was just my client totally desyncing from Reddit's servers. Nothing was submitted at all.

Not too long after that, another guy on my team got hit by the same bug. But I told him what happened with me, so he didn't get his hopes up.

<comment><USER28> It happened to me as well. I did verify that my changes actually made change (from the same IP, but in incognito mode). Didn't bother to check if the changes stayed.

</comment></comment><comment><USER29> This is fantastic. I learned a lot, and it seems like they nailed everything.

I really enjoyed the part about TypedArray and ArrayBuffer. And this might be a common thing to do, but I've never thought about using a CDN with an expiry time of 1 second, just to buffer lots of requests while still being close to real-time. That's brilliant.

</comment><comment><USER30> Startup tech writers, take note. This write up has been more helpful than many in the past. Thank you very much Reddit developers

</comment><comment><USER31> Given the scale described, it sounds like they could have had a single machine that held the data in memory and periodically flushed to disk/DB to support failing over to a standby.

<comment><USER21> You're basically describing how we used redis for this project.

<comment><USER31> I suppose so, but then what did you gain from the extra hop to redis?

<comment><USER21> Not having to implement redis ourselves.

</comment></comment></comment></comment><comment><USER32> _Our initial approach was to store the full board in a single row in Cassandra and each request for the full board would read that entire row._

This is the epitome of an anti-pattern .I sincerely hope that this approach was floated by somebody who had never used Cassandra before.

Even if individual requests were reasonably fast, you are sticking all of your data in a single partition, creating the hottest of hot spots and failing to leverage the scale out nature of your database.

<comment><USER33> This entire project is just an elaborate hack day project. There's no reason to fault them for trying new and interesting hacks to get it off the ground. They realized it wasn't the right method and moved on. End of story.

</comment></comment><comment><USER34> Interesting how big the Norwegian and Swedish flags got, given our small populations.

<comment><USER35> I thought the exact same thing, about the Finnish flag - complete with Moomins.

<comment><USER36> And the almost a dictator of a president Urho Kekkonen. I kind of understand this, I was born in that country...

</comment></comment></comment><comment><USER37> r/Place is really awesome. This is how you grow the community. The 2D and 3D timelapses are super cool to watch, as well. Glad Reddit decided to make this a full-time thing.

<comment><USER38> Full time? I haven't heard that anywhere.

</comment></comment><comment><USER39> This is why experimentation is so important and why I always love when people do things just to do them.

It's literally like exploring the digital universe and reporting on some of your findings.

Great writeup!

</comment><comment><USER40> FULL 72h (90fps) TIMELAPSE: [https://www.youtube.com/watch?v=XnRCZK3KjUY](https://www.youtube.com/watch?v=XnRCZK3KjUY)

</comment><comment><USER41> Thank you for the fascinating writeup! How long did the whole thing take to put together?

<comment><USER42> First commit was on Jan 20 [0], so that provides a lower-bound for how long they spent on it.

[0] - [https://github.com/reddit/reddit-plugin-place- opensource/com...](https://github.com/reddit/reddit-plugin-place- opensource/commit/68498bab9300f43ae4273dd4719dcecb081126f7)

</comment></comment><comment><USER43> This is amazing and I got so many ideas on how to tackle the scaling issues I have with my own multiplayer drawing website. In the aftermath of r/Place I went into some of the factions' Discord servers and posted my site, getting 50-100 concurrent users which caused a meltdown on my server. It was a good stress test but also a wake-up call.

Again, amazing write-up. Thank you!

</comment><comment><USER44> Looks impressive!

How big was the team? How long did it take to complete this project? Is the code going to be open sourced?

<comment><USER45> [https://github.com/reddit/reddit-plugin-place- opensource](https://github.com/reddit/reddit-plugin-place-opensource)

</comment></comment><comment><USER46>
      > At the peak of r/place the websocket service was
      > transmitting over 4 gbps (150 Mbps per instance
      > and 24 instances).

What does Reddit use for serving up this much websocket traffic? Something open source, or is it custom built?

<comment><AUTHOR> It's open source and custom built. [https://github.com/reddit/reddit-service- websockets](https://github.com/reddit/reddit-service-websockets)

</comment></comment><comment><USER47> Now where can we get a dump of all the data.

Like

timestamp, x, y, color, username

<comment><USER48> One of the reddit data scientists dumped it here... [https://data.world/justintbassett/place- events](https://data.world/justintbassett/place-events)

<comment><USER45> > Oops! We can't find that page.

<comment><USER49> my fault :). I have to get a few things ready for a public release of more data

<comment><USER47> Is this data:

[https://www.reddit.com/r/place/comments/6396u5/rplace_archiv...](https://www.reddit.com/r/place/comments/6396u5/rplace_archive_update/)

Complete?

I've been working with that.

<comment><USER50> That data was recorded by the community. It took everyone a while to figure out how to grab a snapshot of the canvas though, so a few hours of data near the start are missing.

</comment></comment></comment></comment></comment></comment><comment><USER51> Weird question - why does that bash script use absolute paths for standard tools like awk and grep (/usr/bin/awk instead of just awk)? Is this some best practice I know nothing about?

</comment><comment><USER52> As someone looking to expand their knowledge of big systems and building at scale this kind of resource is invaluable!

</comment><comment><USER48> If any data science-y folks want to work with the raw data, you can find it here... [https://data.world/justintbassett/place- events](https://data.world/justintbassett/place-events)

<comment><USER50> "Oops! We can't find that page."

<comment><USER48> Looks like the owner changed it to be private :/ Hopefully they'll open it up again.

<comment><USER53> Did anyone download it already? Reshare?

</comment></comment></comment></comment><comment><USER54> Anyone with any insight into how much something like this 'cost' Reddit, resource wise. Is the main outlay in time and the server costs already covered by their infrastructure or does the high traffic add enough to make a difference?

</comment><comment><USER55> /r/place is probably the coolest project that happened this year so far.

<comment><USER56> This year? I'm thinking more like this decade. It's gotta be up in the top 10 of ever. On so many levels, /r/place was fascinating; and I didn't even come across it until after it had finished!

</comment></comment><comment><USER57> Similar to this? [https://www.youtube.com/watch?v=9_uX5yXSOwU](https://www.youtube.com/watch?v=9_uX5yXSOwU)

</comment><comment><USER58> anyone know what framework they used to do visualizations?

<comment><USER21> [https://grafana.com/](https://grafana.com/) for the graphs and [https://www.draw.io/](https://www.draw.io/) and [http://www.fiftythree.com/](http://www.fiftythree.com/) for the diagrams.

</comment></comment><comment><USER59> Good article. One security issue I see is - showing Nginx version in error page - nginx/1.8.0. No need to show details of webserver or its version!

</comment><comment><USER60> Can someone link to the full resolution final image? Been trying to find it. Thanks!

<comment><USER50> Here you go: [https://i.imgur.com/ajWiAYi.png](https://i.imgur.com/ajWiAYi.png).

And for those interested, here's some additional stats:

\- The original announcement about /r/place: [https://www.reddit.com/r/announcements/comments/62mesr/place...](https://www.reddit.com/r/announcements/comments/62mesr/place/)

\- Full timelapse of the canvas over the course all 72 hours __:[https://www.youtube.com/watch?v=XnRCZK3KjUY](https://www.youtube.com/watch?v=XnRCZK3KjUY)

\- Heatmap of all activity on the canvas over the full 72 hours __:[https://i.redd.it/20mghgkfwppy.png](https://i.redd.it/20mghgkfwppy.png) by /u/mustafaihssan

\- Timelapse heatmap of activity on the canvas __:[https://i.imgur.com/a95XXDz.gifv](https://i.imgur.com/a95XXDz.gifv) by /u/jampekka

\- Entropy map of the canvas over the full 72 hours __:[https://i.imgur.com/NnjFoHt.jpg](https://i.imgur.com/NnjFoHt.jpg) by /u/howaboot (explanation: [https://www.reddit.com/r/dataisbeautiful/comments/63kuy6/oc_...](https://www.reddit.com/r/dataisbeautiful/comments/63kuy6/oc_heatmap_of_the_most_pixels_changes_happend_on/dfv3n83/))

\- Map of all white pixels that were never touched throughout the event: [https://i.imgur.com/SEHaUSJ.png](https://i.imgur.com/SEHaUSJ.png) by /u/alternateme

\- Most common color of each pixel over the last...

      - 72 hours**: https://i.imgur.com/C5jOtl1.png by /u/howaboot

      - 24 Hours: http://aperiodic.net/phil/tmp/place-mode-24h.png by /u/phil_g

      - 12 Hours: http://aperiodic.net/phil/tmp/place-mode-12h.png by /u/phil_g

      - 6 Hours: http://aperiodic.net/phil/tmp/place-mode-6h.png by /u/phil_g

      - 2 Hours: http://aperiodic.net/phil/tmp/place-mode-2h.png by /u/phil_g

\- Average color of each pixel over the course of the experiment __:[https://i.imgur.com/IkPOwIh.png](https://i.imgur.com/IkPOwIh.png)

\- Atlas of the Final Image: [https://draemm.li/various/place- atlas/](https://draemm.li/various/place-atlas/) by /r/placeAtlas/ (source code: [https://github.com/RolandR/place- atlas](https://github.com/RolandR/place-atlas))

\- Torrents of various canvas snapshots and image data: [https://www.reddit.com/r/place/comments/6396u5/rplace_archiv...](https://www.reddit.com/r/place/comments/6396u5/rplace_archive_update/**)

\- The post announcing the end of /r/place: [https://www.reddit.com/r/place/comments/6382bb/place_has_end...](https://www.reddit.com/r/place/comments/6382bb/place_has_ended/)

 __It took a while for members of the community to realize what was happening and start recording snapshots of the canvas, so there are a few time periods early on that got skipped

<comment><USER61> This was such an amazing demonstration of human collective collaboration. It sort of makes me fee like humans could do anything, even tho the result is sort of in some sense trivial. As well as simply being enjoyed, this could be studied in so many ways. Competition of memes and cultural representations. 'Evolutionary' convergence upon some optimum. Mainstream vs Fringe. Accepted vs Taboo concepts. Implicit spontaneous emergence of behaviour norms for participants: self regulating systems. I also like this timelapse which contains an overview, and then a close up of each of 12 sections ( 333 x 250 ) - [https://www.youtube.com/watch?v=RCAsY8kjE3w](https://www.youtube.com/watch?v=RCAsY8kjE3w)

</comment></comment></comment><comment><USER62> It's awesome project!

</comment><comment><USER63> > We actually had a race condition here that allowed users to place multiple > tiles at once. There was no locking around the steps 1-3 so simultaneous > tile draw attempts could all pass the check at step 1 and then draw multiple > tiles at step 2.

This is why you use a proper database.

I'd probably add a Postgres table to record all user activity, and use that to lock out users for 5 minutes as an initial filter. Have triggers on updates to then feed the rest of the application.

<comment><USER10> So in that case, each pixel would be stored as a separate row in a relational database? And to query the whole canvas you'd query a million rows on every read?

I lean towards just using the ratelimiting stuff we already have in place (via memcached, which we talked about in a previous post). We just overlooked it.

<comment><USER63> I'd most likely have two tables - one for user activity and one for each pixel (1 million rows only in that table). Selecting a million rows from that pixel table might be 200ms or whatever. I'd still have Redis cache, though, since you're getting 100ms.

<comment><USER64> Consider exactly what you are proposing. One table to store the entire history (one billion or more rows). A second denormalized table, whether updated at the application layer or via triggers, to store the most recent update to each of the one million cells (1000x1000 pixel grid = one million data points).

The simple fact of introducing a one-million-row read for the latest data of each "pixel cell" is fairly insane. You _must_ have a cache for such data. "I'd still have have Redis cache, though" is not even debatable. It doesn't _have_ to be Redis, but is definitely has to be a cache of one kind or another.

<comment><USER63> So, I just did a SELECT * from a table with 1 million single-byte character rows, and it ran in 90.51ms:

      place=> explain analyze select * from board_bitmap ;
                                                            QUERY PLAN
      </comment></comment></comment></comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-----
       Seq Scan on board_bitmap  (cost=0.00..14425.00 rows=1000000 width=6) (actual time=0.009..57.295 rows=1000000 loops=1)
       Planning time: 0.160 ms
       Execution time: 90.510 ms
      (3 rows)

And, with triggers from an activity table, the entire write operation can be atomized so there aren't any race conditions.

I don't think you understand how fast Postgres is on modern hardware. What took a large cluster 5 years ago can be done on a single system with a fast NVMe drive today. We really might not even need Redis in this situation.

And, yes, I have to deal with viral content, so this is right up my alley.

</comment><comment><USER65> This reminds me of [http://www.milliondollarhomepage.com/](http://www.milliondollarhomepage.com/)

</comment><comment><USER66> The front-end UX for scrolling that bitmap was quite frankly horribly badly designed.

<comment><USER21> What was wrong with it?

<comment><USER67> It gets weird when the cursor leaves the box while dragging. Now, when you go back inside, you're still in drag mode since the box did not get the "mouse up" event and you end up selecting and dragging random text.

<comment><USER66> In the end this does not seem to have mattered. Reddit's hardcore "contributors" are the kind of people who enjoy a challenge, even when it's stupid. I think it even turned into some kind of pride for some of them, being able to "master" an idiotically programmed system.

Myself, I just get so frustrated about the idiocy.

<comment><USER68> Reddit has a place for everyone, you might find yourself at home on [https://reddit.com/r/iamverysmart](https://reddit.com/r/iamverysmart)

<comment><USER66> Oh please just go back to reddit.

</comment></comment></comment></comment></comment></comment><comment><USER69> Same was done in Holland several years ago. The one million pixels site. Each pixel was sold for a dollar. All was sold.

<comment><USER70> In real time though?

</comment></comment><comment><USER71> This is all great, but your search never works, still. It has been like that forever.

</comment><comment><USER72> This is awesome but man, reading the canvas portion was a bit distressing. I wonder why they didn't use a game engine to do this? All the work they did has been implemented already in several JS game engines, such as the one I help maintain (it's free and OSS), [https://excaliburjs.com](https://excaliburjs.com). We support all the features they needed including mobile & touch support. They could have also used Phaser ([http://phaser.io](http://phaser.io)) too I bet... that has WebGL support for even faster rendering on supported devices.

<comment><USER73> Hi, I wrote the majority of that part of the project (canvas stuff) & that section of the article – the simple answer is that I have a lot of experience working with the canvas API directly, but little to no experience using any of the popular JS game engines out there (I played around with Phaser years ago, but not very much). I don't think it would've saved me any time to be honest.

<comment><USER72> That's totally fair, I get the sentiment. Great job nonetheless!</comment></comment></comment>


Quebec court upholds $873M judgment against Facebook spammer - <AUTHOR> http://www.theglobeandmail.com/news/technology/quebec-court-upholds-873m-judgment-against-facebook-spammer/article1743467/ <comment><USER2> Looks like he is heavily trying to bank in on his new found celebrity <http://www.adamguerbuez.com/index.php>. After all, assuming his net worth is below a few millions, I guess he is better off with a $873M judgment which gives him tons of press coverage and a Guinness world record than a smaller fine which who would bankrupt him anyway.

</comment><comment><USER3> I wonder what the folks who were in the copyright thread earlier in the day complaining that copyright infringement fines were ridiculously excessive will have to say about a $100-per-spam fine for spamming.

If I'm not mistaken, though, the main effect of this ruling is to completely screw over everyone to whom he owes money. He's already going bankrupt, but if I understand bankruptcy law correctly (and I very well may not) then the ridiculously outsize fine means that all his other creditors now have to make do with a pittance.</comment>


Yet another JVM language? Whiley takes a fresh approach - <AUTHOR> http://www.javaworld.com/article/3137548/scripting-jvm-languages/yet-another-jvm-language-whiley-takes-a-fresh-approach.html <comment><USER2> The Wikipedia page seems like it gives a good overview of Whiley: [https://en.wikipedia.org/wiki/Whiley_(programming_language)](https://en.wikipedia.org/wiki/Whiley_\(programming_language\))</comment>


Kernel Embedding of Distributions - <AUTHOR> https://en.wikipedia.org/wiki/Kernel_embedding_of_distributions <comment><USER2> This seems very interesting, but the overview of this article tests my mathematical knowledge, and I can hardly read the rest without a healthy dose of intuitive reasoning to infer the unknown terms. As far as I can tell, this is a method of statistical analysis which does not require assumptions about the structure of the data in order to perform statistical operations, including comparing data. Conventional information theoretic methods require modeling the predictability (or entropy) of data and then performing statistical operations on the basis of those models, which may be errant, oversimplified or difficult to determine for complex data sets. Instead, the data are represented with an arbitrary number of dimensions in a way that generalizes Euclidean space, and then spatial operations can be performed on the data.

That's as far as I can understand and I'm afraid that there are mistakes even in my simplistic summary. Can someone explain it better?

<comment><USER3> It exceeds my mathematical knowledge, and I gave up trying to read it. My inference was "use of some kind of transformed space when modelling distribution" but the benefit/result versus a direct analysis was absolutely a mystery.

I would still love an explanation of the practical benefits of its use (preferably without the use of numbers or formulae) and IMHO this is what Wikipedia should present. If anyone here is are able to write that description, please do.</comment></comment>


Health officials worry as untraceable virus clusters emerge - <AUTHOR> https://apnews.com/783c7a396adf5f99b8cff399f9478e36 <comment><USER2> I wonder if children are the hidden reservoir? They rarely get ill from this disease and could be passing it around.</comment>


TC50 startup SeatGeek raises money - <AUTHOR> http://www.techcrunch.com/2010/01/08/seatgeek-series-a-funding/

<comment><USER2> These guys got rocked at TC50, deer-in-headlights style. Paul and Marc Andreessen's questions were pretty devastating, given the team should have had answers at the drop of a hat.

Paul asked why they were helping other people buy tickets rather than using their secret price prediction algorithm to arbitrage them and make all the money themselves (assuming the variance is high enough).

They didn't have a good answer to that, although it's plausible to me that the variance is smaller than the transaction costs. But it's a bit like knowing about Black-Scholes 10 years before anyone else and sitting on it.

Andreessen asked them what their TAM was, and it seemed like they didn't even know what that meant. It was pretty bad; obviously their business isn't going to capture 100% of the secondary market, because most of the money is made by the people actually selling the tickets. At most you'd only make some % of that.

In any case, it's good to see them moving forward. TC50 must have helped them refine their pitch a lot, and given the quant nature of the investors they must have seen an opportunity that wasn't totally apparent from their TC50 demo.

<comment><USER3> I can give an obvious answer to that first question. It is because there is no centralised clearing house for tickets, and in order to arbitrage them one would have to buy them from individuals which ends up being very costly. People will not send the correct tickets, they will send fake tickets, they will claim they sent the tickets when you have not received them, etc. It is not a coincidence that almost every auction site out there requires the parties to do the final exchange among themselves.

As far as TAM goes, well I do not think it is fatal if a start up does not know what that particular acronym means. A good investor would calculate that themselves anyway. One would have to be very gullible to believe the TAM estimations of the average company looking for funding (they all say everything is a 100 billion dollar market).

<comment><USER2> Yes, sure, a good answer which amounts to: transaction costs are higher than the variance. That's the answer they should have given Paul.

TAM answers the question, "In the best of all possible worlds, how much money would you make?"

As an entrepreneur you'd better have a good, if not totally accurate, answer to that question. At the very least it shows investors how well you've segmented and sized your target market.

</comment></comment></comment><comment><USER4> Just checked out an upcoming game that I'm planning on attending and wow the interface rocks! At all the other ticket places I have to look at the list of tickets, see what the section is, and then find it on the map to see if the price and location are what I'm looking for. SeatGeek completely eliminates this pain.

<comment><USER5> Thanks, appreciate it! Would love any other feedback you have.

</comment></comment><comment><USER6> I talked with these guys awhile back and they're very nice and have some interesting ideas. I hope they do well.

</comment><comment><USER7> It's kind of pathetic that Arrington labels startups who demoed at TC50 as a "TC50 startup" .. Slyly attaching his name to every startup that sucked up to him enough to get onto the demo roster.

<comment><USER8> Is this that different from YC companies getting the (YC 'XX) whenever they appear on this site?

<comment><USER9> Surely investing!=demoing at.

<comment><USER8> Of course it's not the same, but in both cases the writer is using a short phrase to give context about the company.

Participating in TC50 is not as prestigious as participating in YC, but it is not trivial either. As a reader, I care about whether a company launched at TC50.

<comment><USER9> To be honest, I don't. I don't know 100% how much mentorship input assistance advice, etc TC50 give, but it seems likely that YC or any investor give far more.

The "TC50 startup" thing just seems a bit like newspapers who constantly feel the need to remind you that they broke certain stories "As previously exclusively reported in this very paper" etc.

Just because someone launched at a certain venue, doesn't mean that venue can really claim any ownership. But then it's a common enough thing - you get stars/singers/actors who are constantly labeled as 'being discovered by X' which must be pretty irritating for them.

sorry, went off on a rant there :/ just my 2c</comment></comment></comment></comment></comment>


Text normalization in Go - <AUTHOR> http://blog.golang.org/normalization <comment><USER2> Just a small detail that isn't mentioned in the article:

in NFC form, "base characters and modifiers are combined into a single rune whenever possible"

the interesting detail is "whenever possible": since NFC works by first decomposing, and then recomposing... there're some cases in which if you run NFC normalization on it, the characters will remain decomposed

an example is 𝅘𝅥𝅮 (U+1D160) which its normalized composed form is made of 3 different codepoints

I tried to look at the algorithm for generating the composition table, and it seems it's generated from the decomposition table... if that's so, I can't understand how it could happen that some code points have an NFC form longer than 1

more details: [http://stackoverflow.com/questions/17897534/can-unicode- nfc-...](http://stackoverflow.com/questions/17897534/can-unicode-nfc- normalization-increase-the-length-of-a-string)

does anyone knows the cause behind this?

<comment><USER3> 1\. It's decompose, reorder, compose. So you can see some weird stuff like ḍ̇=ḋ○̣ → NFD=d○̣○̇ → NFD=ḍ○̇

2\. It's not compression, it's normalisation. So it's not compose everything you can. I cannot tell you exact the algorithm off the top of my head, but:

the reason for U+1D160 — it's in CompositionExclusions list.

<comment><USER2> Thanks, after looking up CompositionExclusions I discovered the rationale:

[http://unicode.org/reports/tr15/#Primary_Exclusion_List_Tabl...](http://unicode.org/reports/tr15/#Primary_Exclusion_List_Table)

> When a character with a canonical decomposition is added to Unicode, it must > be added to the composition exclusion table if there is at least one > character in its decomposition that existed in a previous version of > Unicode. If there are no such characters, then it is possible for it to be > added or omitted from the composition exclusion table. The choice of whether > to do so or not rests upon whether it is generally used in the precomposed > form or not.

</comment></comment></comment><comment><USER4> That "café" -> "cafeś" replacement is pretty scary. It looks like the built in strings.Replace function makes the same mistake:

      fmt.Println(strings.Replace("multiple cafe\u0301", "cafe", "cafes", 1)) // multiple cafeś

<comment><USER5> fmt.Println(strings.Replace("multiple cafeterias", "cafe", "cafes", 1))

<comment><USER4> Yeah, I get that. It's just that you might assume that the strings functions would operate on character boundaries (as defined in the blog post) and not based on runes (code points). Leaky abstractions and all that...

<comment><AUTHOR> The purpose of the normalization package is to help you work with text under these constraints. I can't imagine many situations where strings.Replace would be sufficient for reliably manipulating natural language. The cafe example is to demonstrate the why you might need the package.

<comment><USER4> I wasn't thinking that I'd really want to pluralize text like this, but maybe you'd want to turn people's names into links in HTML source or something. If someone's name ends with an accent, and if the unicode isn't normalized, strange things are bound to happen. The blog post is great at pointing this out, and it sounds like people are working on a go.text/search package to help, so that's good. I'm not saying Go is broken, just that this kind of stuff can be really surprising.

<comment><AUTHOR> Yep, working with natural languages is scary. :-)

</comment></comment></comment></comment></comment></comment><comment><USER6> Looks like this issue is pervasive in other languages as well. Out of curiosity ran the same test in Javascript and received the same result.

      s = "We went to eat at multiple cafe\u0301"
      "We went to eat at multiple café"
      s.replace('cafe', 'cafes');
      "We went to eat at multiple cafeś"

Interesting thing is when the text is copy-pasted backspacing first deletes the accent. At least in chrome.

<comment><USER7> FYI:

Node.js - [https://github.com/walling/unorm](https://github.com/walling/unorm) YMMV, but looks good.

It can also serve as a polyfill for the eventual [http://people.mozilla.org/~jorendorff/es6-draft.html#sec- str...](http://people.mozilla.org/~jorendorff/es6-draft.html#sec- string.prototype.normalize)

</comment></comment><comment><USER8> I actually took this a step further a few months back and implemented unicode's "Skeleton" algorithm [https://github.com/mtibben/tr39-confusables- go](https://github.com/mtibben/tr39-confusables-go)

This is useful for example, to ensure that users don't try and spoof each other's usernames. Simply create and store a skeleton string for each username, and keep a unique constraint on it

<comment><USER9> That confusables list is a good starting point, although you'll need to make additions, and probably scale back a couple of the over-zealous ones (eg. rn -> m)

I'm coming at this from a comment spam point of view, not usernames, btw.

</comment></comment><comment><USER10> This document is good, but doesn't mention the case of ligatures. German's "ß" is a problem, and it is not obvious how go handles it.

In javascript:

         "ß".toUpperCase().length !== "ß".length;

Does weiss == weiß ?

<comment><USER11> Does weiss == weiß ?

Yes and no. The swiss would write the former, other German speaking (writing) countries would write the latter. It is incorrect in Germany (after ie, au, eu, ... you must not write ss, unless it's a name, such as the city Neuss)

The upper case of weiß would be WEISS. But it's hard from the upper case WEISS to determine if the lower case is weiss or weiß. (This is why one should never write people's names in bibliographies in small caps.)

<comment><USER12> You can upper case of weiß as WEIß. It is mandatory for taxes and other documents and recommended by the Post.

Technically, Unicode has a capital sharp s since 5.1.0, so we could write WEIẞ.

<comment><USER11> Yes, you can do that. But that's evil and ugly (mixing uppercase and lowercase letters that way). I know it has do be done sometimes.

And I am glad that U+1E9E (LATIN CAPITAL LETTER SHARP S) is not official part of German orthography.

</comment></comment></comment></comment><comment><USER13> Not that there's anything wrong with it, but why are there so many HN articles about Go?

</comment><comment><USER14> Is there an rss for articles of this blog ? Didn't found it.

<comment><USER11> There is an atom feed:

        <link rel="alternate" type="application/atom+xml" title="blog.golang.org - Atom Feed" href="http://blog.golang.org/feed.atom"/></comment></comment>


The x86 Advanced Matrix Extension (AMX) Brings Matrix Operations - <AUTHOR> https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-extension-amx-brings-matrix-operations-to-debut-with-sapphire-rapids/ <comment><USER2> Looks very interesting but ... AVX512 is already problematic cooling wise, this seems even worse.</comment>


Socks Don't Match? How About a Subscription?  - <AUTHOR> http://blogs.wsj.com/digits/2009/03/19/socks-dont-match-how-about-a-subscription/ <comment><USER2> This is why people don't have money. 40,000 people in Europe pay $7.42 a pair of socks?

1\. Collect socks

2\. Figure out how to get people to pay 5x retail for something readily available everywhere.

3\. Profit

 _who has time to shop for new ones?_

Hmm, it is time consuming. I mean, the whole grabbing a package takes forever.

 _There is nothing that is less satisfying to me than going to a store to buy socks._

Yeah, it's just so difficult and disappointing. I guess she's got a really hard life what with her sock problems.

\--

The article compares them to Netflix, but Netflix is different - they're offering a service at lower cost with greater variety. This company is offering less variety at higher cost. This is why people have trouble with their finances - they start spending money they don't need to be spending on services and products they don't need. Yeah, everyone needs socks, but they don't need a service to sell them socks at way above retail. How hard would it be to bookmark socks you like on Amazon and order them when you need them? Or go down to the local store and purchase them?

<comment><USER3> _This is why people don't have money._

I think the point of items like this is not "I am going to spend the rent check on socks" but rather "I have money which is more than sufficient for my needs, but my life is stressful and I have very little time, and I will willingly trade money for time or stress relief, even in doses that seem smaller than some people might appreciate."

My business is built on charging people $30 for something they could do for free. Of course, its only free if you think the extra hour of your life it costs you was valueless. My customers don't.

I paid $3.50 this morning for a coffee and pastry when I could have brewed my own darn coffee and packed a pastry from home for a third of the price. Is that rational? Eh. $2.50 saved a day times twenty work days a month is about $50. That's sort of below my feel-burning-urge-to-change-habits line these days, particularly since the equivalent amount of mental effort of my business would have much better returns.

<comment><USER4> What about buying socks when you are at a clothes shop anyway. Even if its only every two years, you could easily buy 20 pairs at once since you are hardly running the risk that they will be out of fashion next season.

Even rich people go to buy clothes themselves at last once every other year, don't they? Seems like less time in contrast to fill out an online form.

</comment></comment></comment><comment><USER5> I know a lot of people will think this is ridiculous, but I would pay for something like this. I hate shopping and I seem to perpetually be short on socks and underwear. Unfortunately, I never remember when I'm actually somewhere that sells them.

Actually, I take it back. Socks are consumables, but they're not perishable, so it would seem to make more sense to just buy 100 pairs every five years or something.

</comment><comment><USER6> I've been doing something like this for a couple of years -- Picked a popular/long-lived brand of sock, and that's all I buy. No more pairing, no more pairs lost due to losing one, or one getting a hole.

</comment><comment><USER7> I love this idea even though it is expensive. I would never pay these rates, but I think they are missing out on revenue gold. If all the socks you get are the same, like the article says you gather a lot of socks. But if you throw in variety to the sock it is inevitable that people will lose one.

This makes it so that your customers become life long customers as they would always have to replace the mismatched socks. This eliminates customers dropping out after 4 years.

</comment><comment><USER8> Solution: Stop caring if your socks match.

<comment><USER9> That's one solution, but it always bothers me when I know I am wearing socks that don't match. So, I buy big bags of identical socks every so often. $20 lasts for about two years, and since I only own one type of sock, it's not possible to have non-matching socks.

Personally, I think my idea is a lot better than a "subscription".

<comment><USER8> That's pretty close to what I do as well.

</comment></comment></comment><comment><USER10> I really hate matching socks, it seems like such a waste of time. Every time I do laundry I think to myself "There has got to be a way to make the socks match themselves". I mean the dryer is already moving the socks around, and probably bringing every sock by every other sock. Can't there be a way to make like socks attract, stick together?

Alas, 29 years and I still haven't thought of anything :-(

<comment><USER11> Solution, have all your socks be .... wait for it ... THE SAME

I work off 1 big package of socks at a time. Matching them is easy because they're all the same.

Genius.

<comment><USER12> I try to do this, but when I go to buy socks, they never have the kind I bought last time. Instead they'll have pointlessly changed the design, or the color will be slightly darker or lighter, or they'll have changed from sizes of 6-8, 9-11, 12+ to new improved sizes of 7-12 or something, meaning I have to either have socks that fit less well or change brands, which means they don't match any longer... it's all pretty annoying. Lots of cheap, interchangeable goods are like this: if you don't buy them every week, every time you go to the store all the options are different.

<comment><USER13> White socks. I have white socks. There's always socks in the store that are white, or possibly grey in areas covered by shoes.

To notice that my two white socks don't _perfectly_ match, someone would have to come up to me and roll up the legs of my jeans. At this point, I believe they've got me beat on the "social faux pas" scale.

Socks are cheap enough that I throw them out individually if they get holes, or, rarely, if I just don't like how they feel. Most of the time I just get on with it.

I have a much smaller pool of black socks for formal events. Right now I've got "too thin" and "too thick" black socks, but I wear them rarely enough I don't worry about trying to find any that are "just right".

I don't spend much time worrying about socks. I don't spend $10 a year worrying about socks, let alone per month.

<comment><USER14> White socks ftw.

I can't believe people have a problem with this. I learned this from my father when I was still a young boy. Buy white socks, stop worrying.

<comment><USER15> White socks are great for sports, but you don't look very serious wearing a black suit with them...

</comment></comment></comment></comment></comment></comment><comment><USER16> Amazon has something like this. I noticed it as an option for buying iced tea bags, I could select to automatically repeat my order every n months. Don't know what other products they offer it for.

37signals story about it - [http://www.37signals.com/svn/posts/512-amazons- delivery-subs...](http://www.37signals.com/svn/posts/512-amazons-delivery- subscriptions)

</comment><comment><USER17> I vaguely remember hearing a speech by the founder in Switzerland around 2002. If I remember right, he had a single employee and was making ~1M CHF annual revenue at the time. He was so confident of his product that he listed his competitors on his own website - and he would mark each one that went out of business.</comment>


Hacker News: How's that work? - <AUTHOR> http://micheleincalifornia.blogspot.com/2017/08/hacker-news-how-that-work.html <comment><USER2> The article has some insight into the ranking formulae of stories and comments --- or maybe they're just educated guesses. It goes back and forth about whether your karma affects the starting height of your comment. I myself first thought that your comment appeared at the top of the thread for a flash, that rank was briefly boosted by novelty. Then I came to believe that it first popped to the top just for your sake, so that you can see it easily right after posting it. As soon as you reload the page, your comment goes down to its rightful place. Or maybe I'm wrong again. Need to read the source code: [https://github.com/wting/hackernews](https://github.com/wting/hackernews).

And I didn't know about the rare bird known as the vouch button.

The best part about the article is that it encourages you to read Hacker News's own guidelines, [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html). It's a great recipe. So much discussion, here and on other sites, would be better if everyone said the exact same thing minus any personal attack. From the guidelines:

      > When disagreeing, please reply to the argument instead of calling names.
      > E.g. "That is idiotic; 1 + 1 is 2, not 3"
      > can be shortened to "1 + 1 is 2, not 3."

      > Please don't insinuate that someone hasn't read an article.
      > "Did you even read the article? It mentions that" can be shortened to
      > "The article mentions that."

</comment><comment><USER3> I think the downvote doesn't count when you're also replying to the comment. Is that so or am I mistaken?

And I don't know what both flagging and downvoting a comment really does. The downvote makes it greyer, and then the flagging makes it less grey again, so who knows?

</comment><comment><USER4>
      Here are the guidelines. If you have never read them, read them.

Wait for it...

      It is okay to down vote to express disagreement.

The guidelines _don 't say that_.

Have up/downvotes become a popularity contest now?

Has HN become Reddit Orange?

I downvote for one of two reasons:

1) a source or reference is misquoted or made-up altogether.

2) a clear error in _reasoning_ was written (e.g. misinterpretation of the submission of of a stated source).

I don't downvote out of simple _disagreement_. Indeed, I _WANT_ to see well- reasoned arguments that I may disagree with at first glance.

Am I alone in this?</comment>


Show HN: a18n – Automated I18n JavaScript Solution - <AUTHOR> https://github.com/FallenMax/a18n <comment><USER2> Not sure on the name a18n, because it looks like a numeronym, like i18n is, but doesn't seem to be, had me trying to guess what the 18 characters are between a and n.

I get it's a18n like Automated18n like AutomatedI18n.

<comment><USER3> I naturally assumed, and by naturally I mean egrep '^a.{18}n$' /usr/share/dict/words, that _a18n_ abbreviated _anthropomorphization_ , and so in context that perhaps we'd be helping a recently translated program feel more settled in its new repository.

<comment><AUTHOR> Not an English speaker but this is still awkward: I always think "i18n" expands to 18 characters until I see your comment and checked `"internationalization".length` in console to be 20

<comment> 1-more yeah 18 is replacing the middle 18 characters in the word, not including that i and n at the ends. I personally don't like it because it's just an English speaking programmer in-group shibboleth that has bled over to k8s for kubernetes and a16z for Andreessen-Horowitz

</comment></comment></comment></comment><comment><USER4> Shameless self-promotion hijack: [https://github.com/whyboris/JSON-i18n-Editor](https://github.com/whyboris/JSON-i18n-Editor)

A web interface for letting your team translate JSON i18n files - in case you store your i18n in JSON. See the live demo: link on the GitHub page.

<comment><USER5> Nice. Being a happy customer of [https://poeditor.com](https://poeditor.com) I'd like to point out that there's also a service for this out there.

<comment><USER4> I tried POEditor -- it has nice features that my tool does not. I'd like to think my tool is a good-enough free alternative for many cases.

</comment></comment></comment><comment><USER6> From what I can tell, this scans code for strings that can be translated, and then you supply translations from the initial language resulting in a mapping of Chinese to other languages (using Chinese as an example).

What happens if you update that initial translation that every other language hooks off? Do you end up with orphans? At the same time, you may not speak all your languages so you may end up with no -equivalent messages too.

<comment><AUTHOR> Thank you! This concern is valid. If we change original (let's say) Chinese texts in code, and re-run the CLI (wrap+extract). These happens (in my company's workflow):

1\. CLI removes (oldKey -> oldValue) entry from en.json (a18n don't keep stale keys)

2\. CLI adds (newKey -> null) entry to en.json

3\. Developer commits updated en.json (yeah it's versioned)

4\. A script (not in a18n yet) scans en.json, found untranslated text and warns

5\. It's up to the developer to decide what to do with missing translation

Not too perfect, but I think keeping original text in code instead of "some.text.id" also has its benefit:

1\. Devs understand the texts, this brings more context and makes code easier to understand

2\. Some huge projects in my company exists long before they need to support i18n, and it is just too expensive to manually migrate from `text` to `code`

</comment></comment><comment><USER7> This is great. As someone who spent a great deal of time working on i18n with JS for my own software the current state of the art is not amazing. A lot of half finished and abandoned projects, many of which don't entirely work or conform to pre-existing standards. Most don't play well with newer web frameworks and those that do tend to be very limited.

Finding a good solution for VueJS, which has it's own .vue files was particularly hard. I ended up building on top an existing i18n extract tool and integrating it with the VueJS compiler to extract both HTML and JS translations.

What I'd like to see, that I have not seen anywhere (other than a broken and not currently maintained Webpack module) is a plugin for Babel or Typescript compilers that allows the compiler to compile translations into the final output packages and produce one for each language. I think this is preferable to loading translations at runtime and swapping them in.

<comment><AUTHOR> Will definitely look into .vue support!

But for compile translations (per-language), I'm not so sure if added performance can justify introduced complexity (for users' build/serve workflow).

</comment></comment><comment><USER8> Nice work! What's the difference between this and .po/.mo files and the ugettext standard?

<comment><USER9> Thank you for asking this question - it lead me to find this library (and I think the quickstart highlights some differences to how the gnu gettext standard could be leveraged in js/ts vs a "propiatary" solution based on json) :

[https://ttag.js.org/](https://ttag.js.org/)

[https://ttag.js.org/docs/quickstart.html](https://ttag.js.org/docs/quickstart.html)

<comment><AUTHOR> Thanks for feedback! It looks like a decent lib and I would recommend anyone who need richer i18n support (like plural forms) take a look.

As for a18n, it's a deliberate choice to use current format (JSON with extracted key/value pair) instead of standard like ICU messageformat. Here are reasons:

1\. This lib is built to enable automated i18n workflow for existing projects (~200kloc), with minimal human interfere and impose minimal complexity for build configuration, in this regard current format is working great.

2\. This lib is used by both projects, plugins and components in our company, so to keep its footprint down (in case it's not deduped by bundler), it means fewer features (like plural forms, date, list) are supported, and...

3\. ...we originally planned to add support for ICU messageformat if these features are desired. But after two years in production, we found the demand is not too strong to justify the cost.

</comment></comment></comment><comment><USER10> Is it doing anything besides of localization? If not, why did you name i18n (internationalization is a superset of localization domain).

</comment><comment><USER11> Great work.

<comment><AUTHOR> Thanks!</comment></comment>


Ask HN: Fraternities. "There will never be a nigger in SAE" - <AUTHOR>

In light of the recent incident at the University of Oklahoma, [1] I'm interested in speaking with the HN community about fraternities and what role they play in the tech world. I feel fraternities are at a very interesting intersection between the privilege and socialization that is necessary for funding and business success, and yet fraternities are very much at ends with the progressive ideals that so many in the tech community hold.
Many, perhaps even a majority, of successful startups have been founded by people involved in fraternities. From Alex Ohanian, to Zuckerberg, to even Peter Thiel, all were all involved with their college fraternity.
Yet, many of us would agree that fraternities represent a kind of old and outdated institution that is on its way out, regardless of the incidents that have arisen in the past few years.
With scandals surrounding hazing, racist chanting, rape scandals, and loads of superficial partying, it's hard to imagine that the modern tech society has any ties at all to greek life. Forgive me for generalizing, but I tend to find that the tech community is outspoken in their support for meritocracy over status, in their support for gender equality, and with high wishes for higher education to be unpolluted from sports and partying and changed to something more distilled. These are all contrary to what comes from greek life, at least in its current state. There's definitely some cognitive dissonance here.
So, HN, how do you feel about greek life? Are they beneficial to the personal success of their members? Was it beneficial to your career in tech? Will they adapt to modern society? Are they on their way out?
One thing I'm sure we can all agree on, is that the recent video out of OU is unacceptable.
[1]: https://www.youtube.com/watch?v=GBq4_A9nQvw <comment><USER2> This is more or less how all frats seemed to me, admittedly, from a distance, so I never got involved with them. Isn't it obvious that they're a throwback to tribes? Chanting, ritualistic violence, binge drinking, etc. I can't understand how anyone can look at that stuff and be attracted to it. I don't want to be a part of a tribe, I want to dismantle all tribes.

The question of whether it might have helped my "career" is, to me, totally irrelevant. I'd still never ever be involved with something like I saw in the video. I'd much sooner throw my "career" away completely.

I was actually born in Oklahoma. The weirdest thing to me is, even though these kids were saying "nigger", I think if you actually sat down with those kids individually, you'd find out that they don't actually hate black people at all. My best reading of this is that they were actually just sychronizing on some unifying crap, and it didn't really matter what it was. We're all drunk and happy and we're all yelling about the thing that makes us special and different!!!!! They're like dogs in a pack, barking a certain way to know that they _are_ in a pack. They don't hate black people. They don't have any reason to. Just stupid kids being stupid.

Certainly, not all frats are anywhere near that bad. But what is the point? Have friends the normal way. Meet people at your local hackerspace.

I don't think frats are on the way out at all, because humans don't really change. We're the same people who loved the chariot races. And that makes me a little sad.

<comment><USER3> >I want to dismantle all tribes.

I'd say that people who want to do this constitute a (de facto) tribe.

<comment><USER4> I'd argue they'd have to act together and recognize one another.

<comment><USER3> I think it is more contingent on being able to recognize definite outsiders, which they are able to.

</comment></comment></comment></comment><comment><USER5> I loved my fraternity experience and still hold friendships established there as the ones I can count on the most. Fraternities at their best are like sports, they make good men better and bring out the best qualities. At their worst they're binge drinking clubs with little more than mysoginistic, homophobic and racist behavior to bond over. Very tribal, which seems to be the default nature of us.

That said, I know my experience was very different than most. We had a dry house, we had white, black, indian and plenty of others. We were mostly straight, but had a few brothers come out as gay during or after college, to encouragement and support. We ran our own house, doing any repairs and improvements ourselves, we cooked for ourselves, at one point I ran a kitchen that served nearly 100 people a night.

It wasn't perfect, there were plenty of disagreements about how involved we should or shouldn't be on campus, what kind of budget we should have for a band party, how often we should have meetings and a million other things. I can look back and see that these things, and the different ways of thought I was exposed to, made me a better person. I wish others had the same experience but I feel it was far too rare.

</comment><comment><USER6> The only time I ever give fraternities any thought is when a college student on HN asks my opinion on them. Hopefully that hints at an answer to your question.</comment>


Isle of Tune: a music sequencer with a twist - <AUTHOR> http://isleoftune.com/?id=365

<comment><USER2> Michael Jackson: _Beat It_

<http://isleoftune.com/?id=365>

<comment><USER3> That's not Beat It... this one is:

<http://isleoftune.com/?id=819></comment></comment>


The time-saving bias: How drivers misestimate time saved when increasing speed - <AUTHOR> http://journal.sjdm.org/10/10816/jdm10816.html <comment><USER2> I'm in Hokkaido in Japan right now and struck by how slow the speed limits are, even in a rural area with wide straight roads. People get used to driving slow when they are forced to, and society benefits from fewer accidents.</comment>


This Horse Does Not Exist - <AUTHOR> https://thishorsedoesnotexist.com/ <comment><USER2> Something wrong with your fake horses

<comment><AUTHOR> it turns out AIs aren't very good at horses, particularly when mixed with humans in the same shots</comment></comment>


George Gershwin, Then and Now - <AUTHOR> https://www.the-tls.co.uk/articles/summertime-crawford-cambridge-companion-to-gerschwin-calenza-review-russell-davies/ <comment><USER2> [https://archive.md/xHT2y](https://archive.md/xHT2y)

</comment><comment><USER3> If anybody is interested, the 1928 performance mentioned in the article is here: [https://youtu.be/RuU72qM3PJc](https://youtu.be/RuU72qM3PJc)

compare to a later performance: [https://youtu.be/cH2PH0auTUU](https://youtu.be/cH2PH0auTUU)</comment>


I Put In 5 Miles at the Office: Walking Workstations - <AUTHOR> http://www.nytimes.com/2008/09/18/health/nutrition/18fitness.html?ref=fashion <comment><USER2> Has anyone here actually tried this? It sounds interesting, but I'd sure like to suss it out further before making the investment...</comment>


Milestone Based Investing - <AUTHOR> http://www.avc.com/a_vc/2009/08/milestone-based-investing.html <comment><USER2> I agree completely.

Several of my past businesses have gone through tranched rounds and each time I have found it to be significantly destructive.

The VCs who think financially tend to like it for several reasons:

* They are effectively only risking half the capital at any one time reducing their downside. * They are investing the second tranche later which improves their IRR * In their minds they think they have more control over the business.

The reality is, as some the linked and original blog post touch upon:

* The VCs expect you to grow at the same rate as though you'd raised the full amount despite the fact you've only got about 6 months worth of cash in the bank. * It puts loads of pressure to hit short term targets. Both on the company to the VCs and on the VC on the board to their partners. In startups things don't always go to plan (and rarely go completely to plan), and software, markets and deals always take longer. This short-termism is a killer. * It means you are constantly being drip fed and having to worry about manage the shareholders rather than working together about creating a long term view. Very few tech businesses were overnight successes.

In my experience the net effect is considerably to the detriment to of the business, and of entrepreneur/VC relations.

The other thing to realise is the VC is actually getting (assuming 50:50 tranche) X/2 shares and the option to buy X/2 shares in the future if everything at the same price if everything is still peachy. Considering the pretty volatile nature of startups, a rough Black-Scholes calculation will show you're effectively discounting the shared by about 40% by tranching it in that way. E.g. it's better to sell shares at $7 a share untranched rather than $10 tranched.</comment>


The case of the 500-mile email - <AUTHOR> https://www.ibiblio.org/harris/500milemail.html <comment><USER2> [https://xkcd.com/1053/](https://xkcd.com/1053/)</comment>


11 Solid Ways to Improve Your Time Management Skills - <AUTHOR> http://www.dumblittleman.com/2008/02/11-solid-ways-to-improve-your-time.html

<comment><USER2> make lists, make use of down time, reward yourself, concentrate on one thing, avoid procrastination, set personal deadlines, delegate responsibilities, set up a long term planner, employ a program like rescuetime, work in a team, be careful to avoid burnout

Nice submarine!

</comment><comment><USER3> One of the best strategies I've learned is to remove all indications of time from your immediate vicinity when working on a project with a tight deadline. Trying to constantly judge your productivity vs. time is often a huge distraction, and every time you look at the clock is a break in concentration.

</comment><comment><USER4> How can you have a list of 11 things with #4 as "Concentrate on One Thing"?

<comment><USER5> Hrm. My Irony Sense is tingling.

</comment></comment><comment><USER6> <head exploding> Avoid Procrastination at All Costs </head exploding></comment>


In Different Voices (2007) - <AUTHOR> http://bactra.org/weblog/494.html <comment><USER2> I'd thought it uncontroversial that we've been breeding for docility for closer to 6'000 years than "the last one or two centuries."</comment>


Is Ruby Fast Yet? - <AUTHOR> http://www.isrubyfastyet.com/

<comment><USER2> > The benchmark premptively runs the fans and pauses between runs in order to > cool off. These measures give JRuby at least a 25% speed gain. However, > JRuby still might be running slower than it would on a box with appropriate > cooling.

Right. Your computer can't be trusted to not throttle its performance due to crap cooling, and you're trying to _benchmark_ using it?

<comment><USER3> The choice of hardware is quite strange. Should be fun to check the results on normal hardware though.

</comment></comment><comment><USER4> Without knowing what runtimes and parameters are used it's really hard to tell what's going on in this benchmark; JRuby can see 100% and sometimes more improvement on many workloads by tweaking JVM options.

It looks like JRuby is just run under the system JRE (which is what?) with no additional options; that's going to be slow for most workloads.

It'd be nice to see a lot more documentation around the runtime configurations and what the Rails app itself is doing.

<comment><USER5> Down the bottom he states that it is running on Osx. The fan is premptively spun up, which gives Jruby a speed boost. It's probably going to run even faster with proper cooling.

</comment></comment><comment><USER6> Not sure how comparing different versions of rails shows if its 'fast' yet. I'd think setting up some request target and seeing how far RoR is from that target with each version would be better. Or comparing RoR to other platforms is also a good bet, eg [http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/)

I just don't really see how this answers the question other than 'ehh I think it looks a bit faster than it was awhile ago?'

EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason?

<comment><USER7> Amazon says their "m1.small" instance has 1.7 GiB of memory. That's a lot less than most current laptops.

Maybe JRuby uses more memory in total, but can handle more concurrent requests per MB of memory used? That could explain the discrepancy if it's true, though it's just speculation on my part.

Another theory is that the benchmarks are too application-specific. Rails is a complex framework with which you can build many different kinds of applications, each with its own performance characteristics. Conceivably, application A could perform better on JRuby, while application B could perform better on MRI. Depending on which application you use for your benchmark, either interpreter could appear faster.

This calls into question the utility of benchmarking an entire framework. It might be better to do micro-benchmarks, with each one testing a specific scenario. Scenarios could include, for example: A request that loads 1,000 database records into memory; a request for a static image; a request that queries a many-to-many relationship; a request that saves an uploaded file to disk.

<comment><USER8> The thing is, those are all I/O bound operations and in the case of the databases most of the code you're benchmarking is native extensions, so I would expect the difference to be negligible.

Testing whole Rails apps really does seem like the best way to get good data. The interesting speedups are likely going to be in places like string matching / string copying and other places that don't involve network access.

</comment></comment></comment><comment><USER9> JRuby question -- Why is it that much slower? I'd have thought that part of the point was to take advantage of the JVM's built in optimization.

<comment><USER10> My question as well. Every perf benchmark I've seen of jruby over the years, it's much faster than MRI (given time to boot & JIT anyway).

I think this guy's numbers are very suspect.

<comment><USER11> I've benchmarked MRI vs jruby a number of times over the years on real apps and MRI usually cones out faster on average with worse numbers at 95th and 98th percentiles due to GC performance.

I never took the time to figure out why it is faster, but I assume that it is due to C libraries being faster for json serialization and my apps doing primarily json serialization after fetching from the database.

Edit: I should test the performance of the json libs, which would be easy to do

</comment></comment></comment><comment><USER12> Wouldn't it make more sense to test various features of Ruby for comparison rather than comparing an entire web framework? Given that, the URL should be israilsfastyet.com, or more appropriately howdoesrailsbehaveonvariousrubyversions.com

<comment><USER13> Agreed. This is much more of a test of the rails framework then ruby

<comment><USER14> Well, it's a comparison of different ruby versions, using Rails. That's pretty useful to people who use Rails, which is a lot of people.

Of course, it's not a real Rails app presumably, but just a sample/dummy one. So you may or may not see comparable speed-up with a real app. But it's probably suggestive of the upper limit of improvement you can get by switching ruby versions.

"Will I get a performance improvement from my Rails app by upgrading ruby" is a real question a lot of people will have. Micro-benchmarks of ruby wouldn't answer it as well as the OP does.

</comment></comment></comment><comment><USER15> I'm surprised at how negative the comments are here. Graphing how different Ruby versions/implementations perform running Rails is very helpful! Glad to see that MRI continues to get faster with every version. Merry Christmas!

<comment><USER16> The negativity comes from the fact that this benchmark is useless and, at worst, misleading. True benchmarks are hard to get right. Running a bunch of stuff on a laptop isn't sufficient evidence of anything.

Kudos to making the benchmark opensource at least, but even from a quick peak, I wouldn't trust this at all. As others have mentioned, JRuby shouldn't be this slow. Odds are the benchmark is throwing away the JIT gains or the configuration is poor. Add that to the fact that there is no discussion of what a 'rails request' is or what version of Java is being used and there seems to be little to learn from this research.

<comment><USER17> While I'm sure it's not the aim of this site, there is some small value in knowing what sort of performance you get with a naive, untweaked deployment.

<comment><USER18> Website author here. On the contrary, you have nailed it. Noob-friendlyness is a valuable part of a language ecosystem.

</comment></comment></comment></comment><comment><USER19> I've been hacking on a little scripting language of my own lately[1]. To get a handle on its perf, I have a few small benchmarks[2] I run that compare it against Lua (5.1.5), Python (2.7.5), and Ruby (2.0.0p247). (All of those and my language are bytecode-compiled, dynamically typed languages, hence that comparison.)

I happen to have Ruby 2.0 installed, and I was surprised to see it handily beat the other languages. I had expected Ruby to be the slowest of the pack, but they've apparently made big strides in 2.0.

On the off chance that anyone is curious, here's the results of running them on my machine:

        binary_trees - wren   3531  0.28s
        binary_trees - lua    1366  0.73s  258.40%
        binary_trees - python 1420  0.70s  248.64%
        binary_trees - ruby   3170  0.32s  111.39%

        fib - wren            2688  0.37s
        fib - lua             2998  0.33s   89.65%
        fib - python          1390  0.72s  193.38%
        fib - ruby            3645  0.27s   73.74%

        for - wren            9092  0.11s
        for - lua             9387  0.11s   96.86%
        for - python          3171  0.32s  286.74%
        for - ruby            8319  0.12s  109.30%

        method_call - wren    4703  0.21s
        method_call - lua     1780  0.56s  264.20%
        method_call - python   806  1.24s  583.33%
        method_call - ruby    3408  0.29s  138.02%

Those numbers are score, time, and percentage. Score is basically 1/time, just to yield a number where bigger is better. Time is best of ten runs. Percentage is how my language's score compares to that one. So, for example, the 264.20% lua method_call score means my language is about 2.6x faster than Lua on a benchmark designed to stress dynamic method call performance.

[1] [https://github.com/munificent/wren](https://github.com/munificent/wren) [2] [https://github.com/munificent/wren/tree/master/benchmark](https://github.com/munificent/wren/tree/master/benchmark)

<comment><USER20> Oh, man, this reminds me of the pain of writing even the most trivial microbenchmarks in seven or eight languages. How about throwing in some of the fast scripting languages for healthy competition? E.g. JavaScript, LuaJIT and Julia. Feel free to borrow (MIT licensed, but I'm willing to relicense whatever I can):

[https://github.com/JuliaLang/julia/tree/master/test/perf/mic...](https://github.com/JuliaLang/julia/tree/master/test/perf/micro)

<comment><USER19> I'm certainly impressed by those implementations, but I don't think I'm _competing_ with them. :)

I had JS (using node) in there for a while, but it was regularly 10x compared to the other languages. Using a JIT makes it effectively a different class of implementation. It's like entering a car in a bike race.

Thanks for the benchmarks! I might steal some of those. They may be more arithmetic focused than I need (which makes sense for Julia, but less sense for an embedding scripting language where most math is probably done in the host app).

<comment><USER20> It was a turn of phrase, but really I just meant that in my experience it's healthy to have something to aspire to – that's why we always compare to fully optimized C and Fortran. It's easy to fall into an "interpreter bubble" and forget that there's a whole other realm of performance out there. And of course, interpreters _can_ be quite fast – CPython is no slouch and Wren is giving it a run for its money, so kudos. Femtolisp [1] (which we use for Julia's parser) is a great example of a small and simple, yet fast and featureful interpreter. I really want Jeff to write a little book about it, walking the reader through the implementation.

[1] [https://github.com/JeffBezanson/femtolisp](https://github.com/JeffBezanson/femtolisp)

<comment><USER19> > I just meant that in my experience it's healthy to have something to aspire > to

I had actually thought that's what I was doing when I compared it to Lua! I was quite surprised at how well Wren did in comparison. The trick will be to see if I can keep it that fast as the implementation gets more robust. It takes some shortcuts right now that don't scale well, but happen to be faster.

> I really want Jeff to write a little book about it, walking the reader > through the implementation.

I would love that. I've skimmed femtolisp a few times and it's clean but suffers from the almost total lack of documentation that other programming language implementations also seem to have.

<comment><USER20> I don't think that the standard Lua implementation is especially fast – it's the LuaJIT implementation that's so impressively fast.

Programming language implementors do seem to be even less fond of writing comments than most programmers, don't they? I think one of the issues is that internals can change a lot fairly quickly and dealing with comments just makes that worse.

</comment></comment></comment></comment></comment></comment><comment><USER21> >Hardware: Mid-2007 MacBook Pro

Wouldn't it make more sense to do this on generic server hardware with a LTS Linux OS? Once that vintage and customized MacBook bites the dust the website pretty much dies as well.

<comment><USER22> There's nothing magical about a Linux server. If you're arguing an LTS edition will run on newer hardware, well sure but the benchmarks will be just as incomparable.

<comment><USER23> You're right that theres nothing magical, but there is a clear advantage to faster disks, a more modern & faster CPU, newer & faster ram (this is running DDR2 in the benchmark), etc. I'd be curious why this particular machine was used for benchmarking, and if the same benchmarking method yielded different results on EC2 or something like that.

<comment><USER24> You should check out [http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/) where they test on hardware and in the cloud

</comment></comment></comment></comment><comment><USER25> If you're going to call it "Is Ruby Fast Yet", you should really be comparing Ruby implementations to implementations of other programming languages. Asked this way, I believe the answer to the question is a definite "no".

As a small example, I benchmarked a real Scala web app running on worse hardware (a 1GB Linux VM) at over 650 requests/s. There was no degradation in load at this rate of requests, but it was 10x the load we saw in production so I stopped there. The benchmark of Rails here goes to ~120 requests/s.

<comment><USER26> Well there is a JRuby and Rubinius comparison. Those are different implementations of the the interpreter. Ruby will never be faster than C, but this site is aiming to show the improvements over time.

</comment></comment><comment><USER27> FYI we are also working on a similar project

see: [http://samsaffron.com/archive/2013/12/11/call-to-action- long...](http://samsaffron.com/archive/2013/12/11/call-to-action-long-running- ruby-benchmark) and [http://community.miniprofiler.com/t/is-ruby-fast- yet/197](http://community.miniprofiler.com/t/is-ruby-fast-yet/197)

</comment><comment><USER28> I was expecting Ruby benchmarks, not Rails benchmarks.

</comment><comment> w0rd-driven There is one sentence missing from each graph that would've made this a lot easier to grok:

"x is better" where x is more requests, faster startup time, or less memory usage.

I prefer graphs that lead me to what I should expect and that sentence would go a long way to eliminate the "mental math" I had to do to understand each one.

On another note, the 2 year graphs are particularly interesting. They're more consistent than not but those inconsistencies seem a little awkward.

<comment><USER18> The variance is sometimes due to RVM troubles. The benchmark is partly a test of the entire Ruby ecosystem. If RVM is bugging, then people are installing sub-optimal implementations. For example: [https://github.com/wayneeseguin/rvm/issues/1418](https://github.com/wayneeseguin/rvm/issues/1418)

</comment></comment><comment><USER29> 5s startup time for a web framework seems rather high. I wonder how well it compares to similar frameworks in other languages (Django? ASP.NET MVC?)

<comment><USER30> Eh... no it doesn't.

<comment><USER31> It is when your development process involves rapidly iterating the code by running the process many times (i.e. TDD). Anecdotally, when the feedback loop is tighter, I'm much happier. Developing (especially refactoring) in Rails feels like watching paint dry, since it's soooooo slooooow. Every time the process needs to restart, I'm left hanging........waiting......for..............the.............. ..........process.......................to.................... ...................................................s.......... ................t.............................a............... ..............r............................................... ..tttt...........

Also the benchmark is the best case. In real world projects, I've seen Rails take over 60 seconds to start (it took a few days of development effort to bring it down to ~20 seconds). Every time you rerun tests, run a rake task, start or restart your server, etc. It's like the heartbeat of the development process grinds to a halt when you add more features to your Rails server.

And no, tools like Spork are not the answer, since you still have to balance the amount that is resident in memory to fork with coverage of what you can change. It also opens up a fractal of complexity.

<comment><USER32> Oh god. Please edit your comment so it doesn't break the whole layout.

<comment><USER31> No can do. It got downvoted and HN does not allow me to edit negative karma posts. So please upvote my previous post so I can fix the layout of these comments :-)

I'm assuming the negative votes are due to the formatting and not the message. I hope you, dear reader, don't miss the irony of this situation...

If it's downvotes because of the message, well I feel bad for the downvoters and wish them much luck with using their framework.

<comment><USER33> You missed the edit window. If you don't come back and edit your post within one hour, it's locked. Negative karma has nothing to do with it. That's the risk of fire-and-forget posting.

<comment><USER31> Or it's the risk of failing to account for human behavior...

It's not exactly a moral imperative to obsess over your HN comments, IMO.

Most likely, it's a simple bug, that seems to be fixed now :-) No ethics involved. Nothing to see here.

Btw, in the OP, I was trying to express the effect that the increased startup time affects flow and iteration. It's a similar effect to a comment breaking the layout of the page.

I just learned something new today :-)

</comment></comment></comment></comment></comment></comment></comment><comment><USER34> We can at least see MRI has gotten faster with each new version, but we knew that already.

Also, I'm a bit surprised that this doesn't include more implementations of Ruby. Where's Rubinius? IronRuby? REE? I hope the list gets updated.

Furthermore, the benchmark itself could be better; I won't say I am disappointed, but I am not too thrilled either.

</comment><comment><USER35> [serious]

I'm building [http://techendo.co/](http://techendo.co/) and it's running rails on heroku. I was wondering if there was some easy ways to get some better performance out of it immediately now that we're picking up traffic. If anyone could give me some pointers or link me to a page I'd appreciate it.

<comment><USER36> That's an incredibly broad question, so it's pretty hard to answer. Here's some general ones:

* More instances will give you more requests/second, but only so long as secondary system (database, search server, whatever) can keep up with the load

* N+1 queries will kill your performance. They can add a lot of time to processing requests.

* Everything in the most used parts of your site should hit an index on the database. If it's not I can almost guarantee it will be slow.

* Look for work that you can move "offline". As long as they go out within a minute nobody will notice that you aren't sending emails at the exact time they are being generated. Move that work to a background queue so your web server can just add a queue entry and get on to the next request.

* Cache anything you reasonably can. It's a lot cheaper (i.e. faster) to read from a cache than to do a database lookup. It's cheaper to read from a cache than to render an ERB template. Caching can introduce its own difficulties, but the performance gains can be worth it.

Without knowing some specifics of your implementation I really can't give you better advice. Depending on the specific libraries you use it might not even be possible to move to JRuby due to support for C extensions. You might get some performance benefits out of it, but I'd expect it to only buy you some time to execute on some of the above improvements.

<comment><USER37> Tools like [https://github.com/MiniProfiler/rack-mini- profiler](https://github.com/MiniProfiler/rack-mini-profiler) really help with the N+1, and general insight into how long database calls usually take.

Relevant RailsCast: [http://railscasts.com/episodes/368-miniprofiler?view=asciica...](http://railscasts.com/episodes/368-miniprofiler?view=asciicast)

</comment></comment></comment><comment><USER38> As a non-Rails Rubyist, I will have to run benchmarks myself as usual, because people seem to be still equating Ruby to Rails.

</comment><comment><USER39> Compare to what?

And the answer is no, and unlikely to be any time soon:

[http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...](http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=query)

</comment><comment><USER40> There's great data here that's obscured by choosing terrible colours for the graph and showing things that are pretty much irrelevant. A summary chart giving relevant percentage differences would be a lot better than this line chart.

</comment><comment><USER41> I don't like those charts at all. I want to see the progressive Ruby versions on the X axis, I think. The dates are pretty meaningless. If it's the same code, it should run in the same number of cycles; anything else is just noise.

That said, +1 for having a go at it!

Also, yes, Ruby is pretty slow. But it's fast for getting something up and running quickly. If you get wildly successful, you can always swap it out for something else if you need to.

</comment><comment><USER42> If I understand this correctly, this is running periodically on someone's Macbook; people are complaining that's a terrible platform for a benchmark.

It should be possible (if you care enough) to re-run this from the source control history on platforms of your choosing, across whatever date ranges you wish. Anyone really, really care enough?

</comment><comment><USER43> tl;dr: no.

<comment><USER44> I was expecting the web page to just consist of "No."

<comment><USER43> Yep, that was my expectation as well :-)

</comment></comment></comment><comment><USER45> Are web fonts fast yet?

</comment><comment><USER46> Yes, if we're talking about development time! :P

</comment><comment><USER47> benchmarking a web framework on an old macbook pro?

</comment><comment><USER48> this is incomplete and then bullshit, it's comparing a single-core with a multicore running a single-threaded process.

</comment><comment><USER49> I was expecting a one-time-use site such as [http://www.whatwouldmyharvardgrade.be/](http://www.whatwouldmyharvardgrade.be/)

</comment><comment><USER50> Was expecting one of those generic yes/no sites, a la [http://istherejava0day.com/](http://istherejava0day.com/)</comment>


Concentration of CO2 Hits Record High of 416 Ppm - <AUTHOR> https://www.commondreams.org/news/2020/02/12/saddest-thing-wont-be-breaking-news-concentration-co2-hits-record-high-416-ppm <comment><USER2> Not surprising. The growth rate of CO2 in the atmosphere is increasing over time: [https://www.esrl.noaa.gov/gmd/ccgg/trends/gr.html](https://www.esrl.noaa.gov/gmd/ccgg/trends/gr.html)</comment>


Amazon Earnings Amazon beats, stock jumps, its cloud business quite profitable - <AUTHOR> http://uk.businessinsider.com/amazon-earnings-q1-2015-2015-4?r=US

<comment><USER2> I didn't know "beats" was an intransitive verb now...</comment>