Try RethinkDB in the cloud – powered by Docker - <AUTHOR> http://tryrethink.info

<comment><USER2> For those of you wondering what RethinkDB and Docker are:

RethinkDB ([http://rethinkdb.com](http://rethinkdb.com)) is "an open-source distributed database built with love. It lets you enjoy an intuitive query language, automatically parallelized queries, and simple administration. Table joins and batteries included. RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn."

Docker ([http://docker.io](http://docker.io)) is "an open-source application container engine. It gives developers a way to package their app and all its dependencies into a portable container which can be deployed on any modern Linux machine, virtualized or not. Containers are completely sandboxed and do not interfere with each other (think "iPhone apps for the server"), have virtually no performance overhead, and can easily be moved across machines and datacenters. Best of all, they don't depend on any language, framework or packaging system."

</comment><comment><USER3> Hey guys, thanks for your patience. I'm sorry the ride hasn't been as smooth as I'd hoped for so far.

Over 600 of you have created and successfully used Rethink so far. With only a ~7GB RAM overhead (and barely any CPU usage.) That's _really_ awesome.

<comment><USER2> I talked to Nick and we looked at what was causing the problems on his server. A few details on the investigation so far.

1\. It's not caused by RethinkDB :)

2\. It's not caused by cpu, ram or io load. The server is definitely taking a hit, but it still has spare capacity.

3\. No apparent issue on the machine itself (a Linode box)

4\. It's not a docker stability issue either. The docker daemon is happy as a clam, and the containers that _do_ get deployed also run perfectly smoothly.

5\. We suspected that maybe Hipache failed to update its routing configuration fast enough. But that turned out to be a red herring too (unsurprising since Hipache handles vastly larger load at dotcloud).

Our best explanation so far: Docker's default configuration is to assign to itself a /24 ip range. This only allows for 254 distinct addresses, and thus 254 containers (note: the reasoning behind this default was to minimize the footprint on the host system. In retrospect it seems like a silly decision, we'll change it to something larger). Nick then reconfigured docker's interface to a /16, allowing for more containers to be created, and restarted the docker daemon. This causes docker to allocate IPs to new containers from the new, expanded range, while avoiding possible conflicts with the IPs of pre-existing containers. That's where the problem occurs: Docker is doing this correctly, but _very slowly_ , creating a 30 second delay between container creation and successful IP allocation. We're investigating why exactly.

<comment><USER2> I just pushed a fix: [https://github.com/dotcloud/docker/pull/1265](https://github.com/dotcloud/docker/pull/1265)

</comment></comment></comment><comment><USER3> Hi - I'm the author of this site. I'm happy to answer any questions about the technology or implementation!

One of the often asked questions is how to get containers to talk to one another using docker and how to add "runtime configuration." I explore both of these topics and more in the technology piece available at [http://nick.stinemat.es#rethink-docker- technology](http://nick.stinemat.es#rethink-docker-technology)

Thanks for taking a look! Nick

<comment><USER4> Why is "Try RethinkDB now" button NOT working?

<comment><USER2> It worked for me 5 minutes ago.. But now doesn't work for me either. I think the demo server might be overloaded :)

<comment><USER2> It works again for me.

</comment></comment></comment></comment><comment><AUTHOR> For those testing Rethink via this service: once you get an instance, click the data explorer link on top, type "r.expr(1).add(1)" (without the quotes) and hit run. Voila - a simple calculator!

If you'd like to do something more interesting, follow the RethinkDB quickstart -- [http://rethinkdb.com/docs/guides/quickstart/](http://rethinkdb.com/docs/guides/quickstart/).

</comment><comment><USER5> I get a 500 error on the /new POST that occurs when I click the "Try RethinkDB Now" button. Might wanna look into that.

UPDATE: it's working. Load issue of some sort?

<comment><USER3> It's working again. Sorry about that.

</comment></comment><comment><AUTHOR> The whole site seems to be down again. Looks like HN load isn't so easy to handle after all :)

EDIT: looks like it's back up again.

</comment><comment><USER3> Bringing the site down for 5 minutes, brb.

<comment><USER3> It's back up!

</comment></comment><comment><USER6> Is not fast. Is hammered. Cannot test.

<comment><USER3> It's back up. Sorry about that. Such a massive demand (a good thing!)</comment></comment>


Pneumatic tube network at Stanford - <AUTHOR> http://med.stanford.edu/ism/2010/january/tubes-0111.html

<comment><USER3> As the article says, every hospital has these. Without them, I have no idea how we would ever get medications to inpatient floors in a timely fashion.

The tubes are also quite "smart"; all products (including blood products) are barcoded and scanned prior to being put into the tubes for routing to the appropriate destination.

</comment><comment><USER4> I love these systems.

The main branch of the New York Public Library had (still has?) a nice shiny brass system in the research room. You filled out a request on a little card and the librarian would put it into a metal canister and send it down to the stacks. I last used this in 1993 or so when I was a grad student in a literature department.

Then you would go to the big reading room and wait for your number to comeup on the big board. Some wonderful books there ( I remember getting a version of Vincente Huidobro's Altazor that looked like it had a dedication from the author.) And a beautiful old building to sit and read in.

The pnuematic tubes added a nice 'steampunk' feel to the experience (although that word didn't exist yet). I would have been reading William Gibson at the time I am sure though. I know that I read the "Difference Engine" about this time.

It is so remarkable how these systems still exist in the 'older' modernist cities. I say older in reference to NYC because of the fact that it hasn't gone through a war so the intfrastructure is quite a bit older than the larger European or Asian cities. It is different in other cities. When I was in Seoul I would frequently visit a market that was near one of the old city gates. A freestanding market a few thousand years old. That's old - there is nothing in NYC that can touch that. The old stuff in NYC is a few hundred years old at most.

In NYC, the subway is still switched by hand at some places. There is a steam heating system under the streets that, on cold days like this billows up and forms impromptu clouds. And the brass pnuematic tubing at the NYPL.

These systems - especially the older bridges, streets and the subways - <USER2> crumbling. And in some ways the "must have exposed brick in our apartment" nostalgia for an older verdegris patina-ed existence is a false desire for something that never ways.

But you can't help but marvel when crossing something like the Pulaski Skyway in New Jersey (think the Soprano's opening shots) or taking the subway under one of the river tunnels or walking up over the Brooklyn Bridge - that so much was possible with relatively little.

An idea that was beautifully expressed in some of the writing around the recent open-sourcing of the Apollo Lunar Module code from the first moon landing.

[http://atlasobscura.com/places/pneumatic-system-new-york- pub...](http://atlasobscura.com/places/pneumatic-system-new-york-public- library)

</comment><comment><USER5> If these things we're built into new apartments, people would use less cars if groceries could be sendt by tube. Groceries could even be sendt without packaging. Catering could become cheaper and better.

<comment><USER6> While that would be an awesome Jetsons future, this doesn't seem like the kind of system that would well for long-distances, amateurs, heavy items, or fragile foods (would you like your 6-pack of beer shaken or shaken?). It's like the tunnels under Chicago that were supposed to revolutionize delivery, but didn't. (Tunnels: <http://users.ameritech.net/chicagotunnel/tunnel1.html> )

</comment></comment><comment><USER7> I once imagine a tubes network for delivering beverages from the fridge that's located in an outside unit on the property.

Also, a serviceman come by every once and a while to fill up the fridge when it get empty. Of course, security measures are there to prevent unauthorized people from filling the fridge with poisonous beverages or stealing them. The transaction are then already billed somewhere on the internet.

All the programmer have to do is press a button or order from a fridge...and viola! A nice cold beverage is delivered via pneumatic tubes to his desk.

With a little bit of programming, you could have it deliver beverages at regular interval.

Ultimate convenience!

</comment><comment><USER8> Amazon needs to invest in more tube infrastructure.

</comment><comment><USER6> Quite the impressive series of tubes.

</comment><comment><USER9> Tubes are cool.</comment>


Wurstify Your Web Experience - <AUTHOR> https://wurstify.me/

<comment><USER2> i wonder if it's intentional that the review was written by someone with a beard</comment>


ErlPort: Using Python from Erlang and LFE - <AUTHOR> http://technicae.cogitat.io/2014/11/erlport-using-python-from-erlanglfe.html

<comment><USER2> I've update the blog post with these, but here they are in this context as well:

* A more detailed follow-up to that post is here: [http://blog.lfe.io/tutorials/2014/12/03/1828-erlport-and-pyt...](http://blog.lfe.io/tutorials/2014/12/03/1828-erlport-and-python-making-more-calls-from-lfe/)

* There's now an even easier way to use Python from Erlang/LFE that wraps ErlPort: [https://github.com/lfex/py](https://github.com/lfex/py)

Also, the original post was re-blogged here, where the code is much more legible: [http://blog.lfe.io/tutorials/2014/11/21/1508-erlport- using-p...](http://blog.lfe.io/tutorials/2014/11/21/1508-erlport-using-python- from-erlang-lfe/)

</comment><comment><USER3> Here is a good talk on getting erlang to talk with other systems, [http://www.erlang-factory.com/berlin2014/robby-raschke](http://www.erlang- factory.com/berlin2014/robby-raschke). It is of course completely accessible from LFE.</comment>


Where Bean Counting Fails - <AUTHOR> http://ysimonson.tumblr.com/post/2706948721/where-bean-counting-fails

<comment><USER2> The beancounter's acid test is profitability. If public transport were spun off into private hands (perhaps the buses are too big for some routes). Then, allow competition. If then they still made no money, perhaps it's better for people to just own more cars.</comment>


Calls him an asshole, makes him a billionaire - <AUTHOR> http://rogerebert.suntimes.com/apps/pbcs.dll/article?AID=/20100929/REVIEWS/100929984

<comment><USER2> I caught an early screening of this movie last week. I quite enjoyed it. I know this movie will probably be catching some heat for being inaccurate, but I feel like the inaccuracies within the movie are necessary to make an engaging plot.</comment>


Scala vs. Clojure: Concurrency - <AUTHOR> http://bestinclass.wordpress.com/2009/09/17/scala-vs-clojure-round-2-concurrency/

<comment><USER2> _If you think that the Barber problem looks simple, odds are you haven't tried to implement it. There are quite a few things which can go wrong when we have several agents working on shared data, primarily

Deadlocks Race conditions_

A bit off-topic, but I see statements like this all the time, and I always wonder: why is it assumed that you're going to throw together an aggressive design that shoots for theoretically optimally performance, and then trying to ferret out the bugs? Ideally, the solution will be both correct and optimally performant, but if you're worried that you aren't going to get it right, you should start thinking about the best way to fail. It seems wiser to start with a simple dumb solution you know is correct and work carefully towards a more complex, better-performing solution. In that case you would list "crappy performance" as the primary thing that can go wrong in a concurrent system -- but nobody ever does. They're worried about bugs. That means that when they're done with their first attempt, they'll be confident in its performance but not in its correctness.

Here's why I think it's better to start with a simple dumb solution and work from there: you can stop short of an ideal solution. Maybe you only have to optimize it well enough so that it isn't the scaling bottleneck. Or maybe you can just deploy it and tolerate the performance limitations. If you start with code that has the optimal performance characterists but isn't _correct_ , you can't deploy it until you fix all the bugs. You've committed yourself to creating an ideal solution, even though an ideal solution might not be necessary.

<comment><USER3> The chief problem with this is that most programmers' intuitions are calibrated for single threaded processes; when they think of "what could happen" race conditions and deadlocks aren't there. This, I think, is because they are usually emergent global properties, not localized errors, and we're not used to looking for those or, frankly, especially good at them.

There's a very simple way to prevent all deadlocks and race conditions. It's called in some circles the "Global Interpreter Lock" which converts the system from concurrent to single threaded, and is usually the wrong way to deal with the problem. These locks become performance hurdles over time, and people try to replace them with finer grained locks, and what happens _every time_ is that mysterious and weird concurrency errors happen, sometimes for years, until things finally get ironed out. Happens in Python, Linux kernel, and probably a million other places I'm not immediately familiar with.

<comment><USER4> Yes, most programming is a matter of solving a sequential design problem - what comes next in process of building this structure? Threaded programming are more like solving a logic or algebra problem - given certain initial conditions, what could or could not be the case?

I'm not sure whether we aren't good at threaded programming or whether threaded programming is inherently harder.

<comment><USER3> I think the model we use is wrong. My current belief is that the probability of races and deadlocks occurring is a function of the amount of shared data and that conventional shared-everything threading implementations make this far more common than it ought to be. In its extremus this points to a distribution-capable asynchronous message passing architecture á la Erlang, or at least something like Hoare's CSP.

The related criticism that there's too much mutable data is correct in many ways, but can also be seen as a way of reducing the "data frontier"; if it's immutable then there are no concurrency implications.

Now, that said, transactional models don't fit well into that mental model. There can be problems with them--write sync was mentioned in the article--but they seem to be less _inherently_ buggy.

<comment><USER5> doesnt this inherently lower performance?

Message-passing rather than shared data structures implies you need to pass around heavy copies of data.

Are Lock-free data structures (supported probably by atomic hardware operations) the middle ground?

<http://www.audiomulch.com/~rossb/code/lockfree/>

<comment><USER3> Not necessarily, even on a uniform memory access machine; perhaps you pass around a reference to an object that is located close to the data you want. On a non-uniform memory access machine (which is to say all of them these days when you take cache into account) message passing and the message passing formalism can actually save you time.

Anyway, I don't a priori object to shared data structures, but it is important to remember that they are inherently dangerous and the amount of shared memory should be sharply limited to that which is actually necessary, rather than blindly making everything shared.

Lock free data structures won't protect you from deadlocks or races necessarily; they mean you won't have to lock, but you can easily accidentally write a situation with two processes mutually waiting on each other.

</comment></comment></comment></comment></comment></comment><comment><USER3> It seems to me that part of the problem here is that the Scala solution misuses mailboxes--not terribly surprising. I've been unable to find an idiomatic Erlang solution, but I wouldn't be terribly surprised to find that one a) exists and b) works.

<comment><USER6> A not-terribly-idiomatic Erlang implementation I slapped together to test this theory, and also to practice on silly problem:

      -module(test).
      -export([run/0]).
      -import(timer).

      -define(SEATS, 3).
      -define(DAY, 8*60*60).
      -define(PERIOD, 20*60).
      -define(CUTTIME, 15*60).

      barber(Shop, Count) ->
          receive
              {customer, P, N} ->
                  Shop ! accept,
                  io:format("(B) Cutting hair of customer ~w~n", [N]),
                  P ! haircut,
                  timer:sleep(?CUTTIME),
                  barber(Shop, Count+1);
              endtime ->
                  io:format("(B) Closing the shop ~n"),
                  Shop ! close,
                  barber(Shop, Count);
              closed ->
                  io:format("(B) Closed. ~n"),
                  io:format("(B) Served ~w customers. ~n", Count)
          end.
      customer(N) ->
          receive
              haircut ->
                  io:format("(B) Customer ~w got haircut ~n", [N]);
              leave ->
                  io:format("(B) Customer ~w leaving ~n", [N])
        end.
      shop(Seated, Barber) ->
          receive
              accept when Seated == 0 ->
                  io:format("Flagrant Error~n");
              accept ->
                  shop(Seated-1,Barber);
              close ->
                  Barber ! closed;
              endtime ->
                  Barber ! endtime,
                  shop(Seated, Barber);
              {customer, P, N} when Seated >= ?SEATS ->
                  io:format("(S) Incoming customer ~w turned away ~n", [N]),
                  P ! leave,
                  shop(Seated,Barber);
              {customer, P, N} ->
                  io:format("(S) Incoming customer ~w seated ~n", [N]),
                  Barber ! {customer, P, N},
                  shop(Seated+1,Barber)
          end.
      shop_helper() ->
          receive
              P ->
                  shop(0, P)
          end.
      run() ->
          S = spawn(fun() -> shop_helper() end),
          B = spawn(fun() -> barber(S,0) end),
          S ! B,
          make_customers(S, 0, ?DAY).
      make_customers(Shop, _, Time) when Time<0 ->
          Shop ! endtime;
      make_customers(Shop, Id, Time) ->
          Shop ! {customer, spawn(fun() -> customer(Id) end), Id},
          timer:sleep(?PERIOD),
          make_customers(Shop, Id+1, Time-?PERIOD).

This implementation is 68 lines long: twice the Clojure, but still less than the Scala. A big portion of this difference is because the Clojure version cheats, modeling the system as straight-up producer/consumer with queue size constraints. The other two implementations spend about 20 lines dealing with Customer actors, which the Clojure completely punts. The Clojure still wins, but it's not as big a margin as it looks.

<comment><USER7> Actually 68 is about the same as scala. About 30 of the lines are stylistic: one closing bracket to a line and some blank lines for visual grouping. As well several bits of printed status that the clojure version elides.

</comment></comment></comment><comment><USER8> Joshua Bloch, [http://www.scribd.com/doc/33655/How-to-Design-a-Good-API- and...](http://www.scribd.com/doc/33655/How-to-Design-a-Good-API-and-Why-it- Matters), I believe had as one of his rules: "Example code should be Exemplary" (I don't see it in the slides, but it was in the video as I recall)

Sample code lives forever; make sure you get it right!

Of course, I didn't go to the primary sources for the article, so I could be completely wrong about the situation.

edit: While I really like clojure, that isn't all its syntax, less macros. There's stuff like ` #() _ etc., although two out of three of those examples are shorthand for things you can do with the syntax he listed (quote and fn). I think you could easily fit it on a page, though.

</comment><comment><USER9> The use of actors with mailboxes that annoys the author is:

1\. Take a message from the mailbox (that may have been queued for a long time).

2\. If the mailbox still has more than N items, then throw away the message.

This is a really dumb way to approximate a fixed size mailbox, and doesn't allow for an immediate refusal.

The questionable Scala code is pulled from recently written O'Reilly book on Scala. I haven't read it myself, but it's probably not as good as "Programming in Scala" (Odersky et al), which covers Actors+Concurrency in Chapter 30.

Needless to say, Scala also has access to all the Java concurrency primitives and libraries. I believe there's a Scala STM library as well.

<comment><USER10> Yes I like the look of the clojure one, but actors are just one of many (flawed) libraries in Scala, I am sure a similar solution could work there.

</comment></comment><comment><USER11> Definitely one of the better postings I've seen on HN.

In very little space it teaches you a lot about clojure + scala + concurrency. And it seems to me (coming from a scheme + java background, so with no particular horse in the race) to be a rather honest comparison.

I'm eager to see what the Scala community responds with (and hoping they take the time to show code).

</comment><comment><USER12> That design is poor. There should be an actor modeling the seats (passed a customer and as a non-blocking operation either seats them or ejects them) and an actor modeling the barber (loops doing a blocking poll of the seats for a customer and doing the haircut).

Mailboxes are not queues!</comment>


Merry Christmas to HN - <AUTHOR> Hello HN,<p>I become active Hacker news at the start of year, its been incredible journey so far. I have learn so many things from computer science to psychology to economics etc. Thanks for all the contribution and best of luck for future. Happy Holidays. <comment><USER2> Merry Christmas, everyone. I tend to lurk a lot, but this is one of the few programming communities where I don't expect to be met with negativity and condescension every time I post something. And beyond that, just reading everyone's discussions has easily helped shape me into the person I am today. Thanks for the past few years and many more to come!

<comment><USER3> You must be new here. Wait until the negativity and condescension kicks in.

<comment><USER4> If there is some of those, it's still at a dramatically lower rate than from any other boards that I've been looking at.

This is refreshing and always good for the mind, scientific or not that it might be.

</comment></comment></comment><comment><USER5> And a jolly Festivus to the Restuvus!

<comment><USER6> Have a wonderful fleas on my dog!

</comment></comment><comment><USER7> Merry Christmas guys. May you and yours be blessed with peace, prosperity, and happiness, today and always.

</comment><comment><USER8> I read HN since 3 years. I've created my startup 1.5 year ago. I think that says it all. Oh and I had revenue from day 70, currently cash-flow-positive, and I've taught a lot of people what bootstrapping means. Thank you, community.

</comment><comment><USER9> This seems as good a place as any to point this out: take a look at the numbers on the frontpage.

<comment><USER10> At the moment now it's default orange again. Xmas theme turned off?

<comment><USER11> That was me. I turned it on in the REPL last night, but did it wrong. Leapt out of bed this morning thinking "Doh! I bet it switched off." Which it had.

<comment><USER10> No worries. That's why we have 2 Christmas days.

</comment></comment></comment></comment><comment><USER12> 메리 크리스마스 HN. Thanks for all the insights, day in and day out. Love ya.

</comment><comment><USER13> Thank you! Merry Christmas to all of you :-)

And thank you for being a big part of my small world :-)

</comment><comment><USER14> Gleðileg jól!

To get a jumpstart on my New Year's resolution, time to air some grievances (with myself)!

1\. Dammit Xan, when are you going to finish up those unit tests for tänzer? You're holdin up the actual 0.1 release!

2\. I can't believe you haven't started implementing your bytecode VM yet! Are you waiting on a freaking miracle, or just piddling until you figure out whether you want to make classes and functions defined at object code load time, or have opcodes for registering classes and functions at runtime?

I can't go home until I wrestle myself in this year's Feats of Strength and get code for my VM, birchvm, up and running.

</comment><comment><USER15> Merry Christmas and thanks for being with me every single day in 2014.

</comment><comment><USER16> Merry Christmas to you as well.

</comment><comment><USER17> Merry Christmas! I've thoroughly enjoyed lurking at HN for the last year. Looking forward to another year of interesting articles!

</comment><comment><USER18> One of the few places left where comments are worth reading. Thank you, all, and Happy "whatever makes you happy". :-)

</comment><comment><USER19> Merry Christmas! I share with you my santa hat! [http://santahat.me](http://santahat.me)

</comment><comment><USER20> Merry Christmas everyone!

[http://xmas.flatout-technologies.com](http://xmas.flatout-technologies.com)

</comment><comment><USER21>
      map (\x -> chr (x + 32)) [45,69,82,82,89,0,35,72,82,73,83,84,77,65,83,1]

<comment><USER22> explain :)

<comment><USER23> Python equivalent for convenience:

      map(lambda x: chr (x + 32), [45,69,82,82,89,0,35,72,82,73,83,84,77,65,83,1])

Just paste it into a python terminal and run it :)

<comment><USER24> Python 2/3 compatiblity:

        "".join(map(lambda x: chr (x + 32), [45,69,82,82,89,0,35,72,82,73,83,84,77,65,83,1]))

<comment><USER25> Or as a list comprehension if that's your thing:

        "".join([chr(x + 32) for x in [45,69,82,82,89,0,35,72,82,73,83,84,77,65,83,1]])

</comment></comment></comment></comment></comment><comment><USER26> HN is the reason I'm a reasonably good programmer. Happy Holidays everyone!

</comment><comment><USER27> Nollaig shona daoibh go léir!

<comment><USER28> Nollaig chridheil agus bliadhna mhath ùr à Alba.

<comment><USER27> Go raibh maith agat, cara :) Agus tú féin

</comment></comment></comment><comment><USER29> HN is very informative and boosts the confidence of readers in their respective areas of working. I wish all HN contributors to include articles which will be of interest to professionals in all walks of life.

</comment><comment><USER30> "Nollaig chridheil agus bliadhna mhath ùr" from Scotland.

</comment><comment><USER31> Merry Christmas all, I hope your holidays are wonderful

</comment><comment><USER32> Wesołych Świąt Bożego Narodzenia!

All the best to the community. It's the only place on the Internet where comments are often more insightful than articles :)

</comment><comment><USER33> Merry Christmas all!

</comment><comment><USER10> Vrolijk Kerstfeest!

</comment><comment><USER34> Merry Christmas!

How about having a little new year party this 31st ?

</comment><comment><USER35> [https://www.youtube.com/watch?v=NsLdRAYxa1o](https://www.youtube.com/watch?v=NsLdRAYxa1o)

</comment><comment><USER36> Merry Xmas to all of you! It's been a pleasure to read and be influenced by great people and minds. Love to Computer Science!

</comment><comment><USER37> 4d 65 72 72 79 20 43 68 72 69 73 74 6d 61 73 21

</comment><comment><USER38> Merry Christmas to each one of you here on HN.

</comment><comment><USER39> Merry Christmas!

</comment><comment><USER40> Feliz Navidad!

<comment><USER41> Here's a Christmas present for you, feel free to reuse as often as needed: ¡

¡Feliz navidad! :)

</comment></comment><comment><USER42> Merry Christmas to all from Houston!

</comment><comment><USER43> Glædelig jul!

</comment><comment><USER44> Срећан Божић!

</comment><comment><USER45> Ungil Kurismas (Palauan)! Probably my favorite post on HN right now. Happy Holidays to all.

</comment><comment><USER46> Merry Xmas ya glorious basterds!

</comment><comment><USER47> Merry Christmas to you as well

</comment><comment><USER48> Merry Christmas HNer's :)

</comment><comment><USER49> Merry Christmas to you as well

</comment><comment><USER50> Merry Christmas to you too!

</comment><comment><USER51> Merry Christmas, everyone!

</comment><comment><USER52> Счастливого Рождества! :)

</comment><comment><USER53> Happy Holidays! Here to the many more successful years to come!

</comment><comment><USER54> Happy Grav-Mass, folks!

</comment><comment><USER55> Schöne Weihnachten :)

</comment><comment><USER56> မေရီ ခရစ်စမတ်ပါ HN ..

</comment><comment><USER57> ಕ್ರಿಸ್ಮಸ್ ಹಬ್ಬದ ಶುಭಾಷಯಗಳು ! With regards from India :)

</comment><comment><USER58> Merry Christmas HN!

</comment><comment><USER59> Fröhliche Weihnachten & feliz natal

<comment><USER60> Feliz Natal!

</comment></comment><comment><USER61> Love this community, Merry Xmass <3

</comment><comment><USER62> Joyeux noël :)

</comment><comment><USER63> Buon Natale!

</comment><comment><USER64> გილოცავთ შობას! from Georgia

</comment><comment><USER65> Bon Nadal!

</comment><comment><USER66> 圣诞快乐!

</comment><comment><USER67> Lmao at learning computer science, psychology and economics from Hacker News...

<comment><USER60> I think it was obvious he meant "learning some from these subjects", and not "I actually became as knowlegeable as a psychologist and an economist".

</comment></comment><comment><USER68> Vrolijk kerstfeest!!

</comment><comment><USER69> Merry Christmas!

</comment><comment><USER70> Veselé Vianoce!

</comment><comment><USER71> Hyvää joulua!

</comment><comment><USER72> සුභ නත්තලක්!

</comment><comment><USER73> happy xmas and new year !

</comment><comment><USER74> many thanks to hn

</comment><comment><USER74> many thanks

</comment><comment><USER75> Merry Christmas everyone! Ooh look alternating article numbers in holiday colors! Isn't it amazing what technology can do?? If only it could <blink/> like the olden days

</comment><comment><USER76> All I want for Christmas is the three lines of CSS it would take to make HN responsive.

If that's not possible, I'll take world peace instead.

<comment><USER77> I'd like collapsible comments more :)

<comment><USER78> There's several extensions folks have written to add layers of functionality to HN -- collapsible comments being a major one.

I use HN Enhancement Suite: [https://chrome.google.com/webstore/detail/hacker- news-enhanc...](https://chrome.google.com/webstore/detail/hacker-news- enhancement-s/bappiabcodbpphnojdiaddhnilfnjmpm?hl=en)

</comment></comment></comment><comment><USER79> The thing I dislike most about the holidays is the erosion of intellectual discussion and infectious somatization even on its few remaining bastions like HN.

Oh, Huxley.

<comment><USER80> > erosion of intellectual discussion and infectious somatization

Santa got you a thesaurus for Christmas?

<comment><USER79> The only weird one there is Soma, and that's a Huxley reference you won't find in a thesaurus.

I wish though. A thesaurus would be a most endearing gift.</comment></comment></comment>


A man who shared his books and multiplied them (2012) - <AUTHOR> http://blog.woodpie.com/the-man-who-shared-his-books-and-multiplied-them/ <comment><USER2> My Seattle neighborhood has over 30 Little Free Library ([https://littlefreelibrary.org/](https://littlefreelibrary.org/)) book exchanges. The owners tell me that they simply seed the exchanges with an initial collection of books and do little in the way of curation or maintenance after that.

<comment><USER3> My local library sponsors something similar.

</comment></comment><comment><USER4> Thanks for this story. It is a nice counterpoint to a lot of what i read. It all starts at home.</comment>


The plummeting values of unicorns prompts soul-searching among investors - <AUTHOR> https://www.wsj.com/articles/silicon-valley-adjusts-to-new-reality-as-100-billion-evaporates-11574764205?mod=rsswn <comment><USER2> Wow, I just watched the ColdFusion video on We this morning. It's interesting watching this huge disaster in real time, instead of years later.

The blame seems to be on Masayoshi Son, of Softbank, for anointing their founder Adam Neumann. He sounds like a character from the tv show Silicon Valley.

[https://youtu.be/QHQTzeve7OM](https://youtu.be/QHQTzeve7OM)

<comment><USER3> At this point I'd rather take money from Russ Hanneman, at least his character and motivations are well known and unlikely to change.

</comment></comment><comment><USER4> Evaporates? It wasn't there in the first place.

<comment><USER5> The value was only there on Masayoshi Son's fevered brow, has subsequently evaporated.

</comment></comment><comment><USER6> How can I comment if I can't read the article ? Is this link a promo for WSJ subscription?

</comment><comment><USER7> Both outline.com and archive.is can't bypass paywall. I don't read wsj often: is there still a valid workaround?

<comment><USER8> Yes: Your local library might have article content available online.

SFPL link: [https://search-proquest- com.ezproxy.sfpl.org/docview/2317834...](https://search-proquest- com.ezproxy.sfpl.org/docview/2317834265/4A4609EB630D4883PQ/1)</comment></comment>


Async/await: It's Good and Bad - <AUTHOR> https://medium.com/@benlesh/async-await-it-s-good-and-bad-15cf121ade40#.t59d6u4ws <comment><USER2> The author has a beef with the ES5-compatible code that Babel emits for async/await, but you don't have to go that far.

If you can get away with ignoring IE and Safari, the latest versions of Edge, Firefox, Chrome, and Opera all have native support for promises and generators, which can implement async/await style concurrency much more tersely: [https://babeljs.io/docs/plugins/transform-async-to- generator...](https://babeljs.io/docs/plugins/transform-async-to-generator/)

There's also Kneden ([https://github.com/marten-de- vries/kneden](https://github.com/marten-de-vries/kneden)) which, combined with a Promise polyfill, should transpile async/await into relatively clean chains of promises.

</comment><comment><USER3> I don't really buy the con of async/await allowing you to write really long functions. You can do the same thing in completely synchronous logic. Additionally, I don't really buy the chopping up of a cohesive process into itty bitty functions. Certainly small, simple function are nice, but when small for smallness sake, you basically end up with unreadable spaghetti code.

A benefit the author didn't quite touch on is what a boon async/await is for branching processes. I've got a block of logic that takes credit card info for Stripe, and it needs to validate user info, tokenize payment info, add it to the user model, and then add the tokenized card to the Stripe customer associated with the card. There are a lot of places this can go wrong or data can fail a validation. In promises, branching mid-flow is a huge pain.

The much more interesting issue with callback-driven approach to async, no matter what the sugar, was raised by [http://journal.stuffwithstuff.com/2015/02/01/what-color- is-y...](http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your- function/).

</comment><comment><USER4> The fact that the generated javascript doesn't look as neat is a pretty poor argument. Does the author avoid other language features that get "lowered" ([http://ericlippert.com/2014/04/28/lowering-in-language- desig...](http://ericlippert.com/2014/04/28/lowering-in-language-design-part- one/)) into less elegant code as part of the execution?

In time, we'll not see this code as it'll all happen within the javascript engines, but all features of javascript are still abstractions over lower level code.

How about writing a loop over asynchronous functions? Now that is unpleasant to look at, and most examples I've seen use recursion which is pretty limiting.

That is one of the best reasons to embrace async/await

</comment><comment><USER5> Does it really matter what the generated code looks like?

We use Typescript and I don't care what the final Javascript looks like. I don't plan to ever touch that code manually.

<comment><USER6> It matters if you're concerned about keeping the size of your JS payload in production to a minimum. In the Babel example, you're looking at multiple times more lines per async function compared to promises, plus extra execution via regenerator when compiled to ES5.

That said, I'm a fan of async/await -- just not yet for projects where I need to optimize for filesize.</comment></comment>


European Parliament asks member states to grant protection to Edward Snowden - <AUTHOR> http://i100.independent.co.uk/article/europe-has-voted-to-offer-edward-snowden-asylum--W1eMqoEsDOe <comment><USER2> Misleading title. They have voted for a. Non binding resolution agreeing that he should not be extradited from the EU. It is a far cry from asylum.

<comment><USER3> It is obviously far less than what the title implies or what (some) people would like. But the power of parliament's non-binding resolutions to define the debate shouldn't be underestimated.

The Parliament has, in the last 10 years and especially under its current President Schulz redefined the balance of power vis-a-vis the Commission and the Council. One prominent recent example was the so-called "Spitzenkandidat- System" whereby the parties nominate a candidate for the Commission presidency and then whichever party wins the most seats in parliament gets its nominee appointed.

Note that there were no laws mandating such a system and that it was actively opposed by quite a few powerful people and institutions, such as Chancellor Merkel.But Schulz et al did an excellent job of (a) exploiting parliament's democratic legitimacy as the only elected EU body and (b) showing the heads of states some fine instruments by which Parliament could have stalled the agenda.

Sorry for the wall of text. tl/dr: the EU parliament has acquired an aura that makes even non-binding resolutions much more relevant than, say, a few op-eds or press releases.

</comment></comment><comment><USER4> Relevant seeing as the 'i' didn't link to it (third paragraph):

[http://www.europarl.europa.eu/news/en/news- room/content/2015...](http://www.europarl.europa.eu/news/en/news- room/content/20151022IPR98818/html/Mass-surveillance-EU-citizens%27-rights- still-in-danger-says-Parliament)

Discussion here:

[https://news.ycombinator.com/item?id=10471172](https://news.ycombinator.com/item?id=10471172)

</comment><comment><USER5> _>...asked member states to grant him protection from extradition_

Incredibly misleading. Key word there is "asked". The EU Parliament has no power to compel such, nor indeed can it pass legislation by itself.

</comment><comment><USER6> So, who wants to volunteer to lose all trade with the United States?

</comment><comment><USER7> The EU is not a country and has no power to offer anyone asylum.

<comment><USER3> Asylum is within the jurisdiction of the European Union, which, for example established minimum standards for the asylum process in 2004. These rights can and have been held up by the courts. In this regard, the European Union has created the law that "offers people asylum" even if the executive individual decisions are left to the member states.

There's also the Dublin Agreement which (even if it's falling apart right now for practical reasons) is in principle applicable, binding law.

So, no, the EU isn't going grant Snowden asylum. But it is certainly a major player in refugee law.</comment></comment>


Ask HN: Is it ok to outsource your resume to an HR professional? - <AUTHOR>

Started updating my resume and just thought that there might be staff that I just don't know that cant make my resume 10x more solid. <comment><USER2> Usually it couldn't hurt to have more people look at your resume and give advice. Each of us can only know so much, and while you might not take your advice, it could be useful to hear.

Be wary that if you pay someone to work on your resume, it's no guarantee that you're more likely to get a job. And the resume is still just the key that opens the lock, you still have to get your foot in the door by doing well on the many levels of interview and screening.

</comment><comment><USER3> Posting a link has often proven useful for garnering feedback from HN'ers in threads like this. Considering that many HN'ers have seen a lot of resumes, it may not be a bad idea.

Good luck.</comment>


Run a private DNS over HTTPS service - <AUTHOR> https://fly.io/docs/getting-started/dns-over-https/ <comment><USER2> What would be my motivation doing so? Not sure what is the advantage of running my DNS over HTTPS.

>> DNS over HTTPS (or DoH) is a protocol that makes browsing more private. Browsers typically resolve domain names with an unencrypted protocol, allowing nosy neighbors and internet providers to snoop on some internet activity. DoH creates an enecrypted connection between browsers and the DNS resolver to make it difficult to even see what domains a user is loading.

This is simply not true (typo aside). My resolver runs locally in my apartment or in our office. "nosy neighbors and internet providers" are not a threat to us but surveillance companies are, who coincidentally pushing for DoH. And DoH is a protocol that makes browsing easier to be controlled by your browser vendor. These guys just forget that not all DNS traffic is originated from web browsers. Google (Chrome) already ignores resolv.conf and trying to force its way to 22.214.171.124. Basically fighting for the last bit of information they can get about you. Thanks, but no thanks.

<comment><USER3> Does your local resolver not have to query an upstream provider? Is that recursive query private?

<comment><USER2> You can configure it two ways:

\- full recursive (meaning you are going to query the root servers and all the other servers until you get your answer)

\- custom upstream provider who is not a surveillance company

The first is as secure as the protocol to use to query. You can use DoT to make it more secure.

"DNS over TLS (DoT) is a security protocol for encrypting and wrapping Domain Name System (DNS) queries and answers via the Transport Layer Security (TLS) protocol. The goal of the method is to increase user privacy and security by preventing eavesdropping and manipulation of DNS data via man-in-the-middle attacks."

The second case is a bit better when DoT is not available for all the nameservers, using Cloudflare's DNS service as an upstream gives you what you want (it also supports DoT AFAIK).

As far as I am concerned DoT is the way to go and the best would be if all DNS servers support that all the way to the root servers and I could run my home service with ad filtering on while other people could just use Cloudflare or whoever they want and not leaking out what they are querying for companies in between. DoH is pointless in my opinion if you have DoT.

</comment></comment></comment><comment><USER4> I wrote a DoH implementation [https://github.com/fanf2/doh101](https://github.com/fanf2/doh101) based on OpenResty which is running in production on Cambridge University's central DNS resolvers. Not many people are using DoH, though! DNS-over-TLS is _much_ more popular, because Android will try to use it by default. If you run a recursive DNS server, enable DoT!

<comment><AUTHOR> OpenResty seems like a great way to do that.

<comment><USER4> It was my first OpenResty app, and it worked better than I expected. My only qualm is that the OpenResty packages are not updated very frequently, which makes me worried about security support.

</comment></comment></comment><comment><USER5> The article is more about "how to use Fly", which is cool and looks useful, but not really for me.

Thanks for the doh-proxy tip off though, might be worth chucking that on the box that runs my DNS :)

<comment><AUTHOR> doh-proxy is pretty neat and tiny. The flyio/doh-proxy docker image is pretty much ready to go if you want to use it somewhere else. It's just a little harder to get the "global" bit.

</comment></comment><comment><USER6> I am not a network guy so please bear with me. I use a large hosts file on Windows filtering out massive amount of crap. I don't think it's possible to keep using it while running DNS over HTTPS? (I directly connect my PC to the modem provided by ISP).

<comment><USER7> There should be PHP sscipts; DoH servers. Install it to your https web site. Use it as a proxy to another DoH server. But before you forward the query to the other end check the domain against your black list.

Or you can use ads blocking DoH server [https://blahdns.com](https://blahdns.com)

<comment><USER8> NextDNS[1] is an alternative with customizable blocklist.

[1]: [https://nextdns.io/](https://nextdns.io/)

</comment></comment></comment><comment><USER9> You can also do this on the same network which runs 22.214.171.124. Cloudflare (where I work) allows you to deploy your own serverless functions for free [1] on the same machines which serve DNS traffic, and the Workers can query [https://cloudflare-dns.com](https://cloudflare-dns.com) without a problem.

1- [https://workers.cloudflare.com/](https://workers.cloudflare.com/)

<comment><AUTHOR> At the risk of hair splitting, but I'm not sure I'd call custom JavaScript in front a shared backend DoH origin "private". It seems like you may as well just use 22.214.171.124 to do DoH if you're content with CloudFlare.

<comment><USER10> ...in fact, the default DoH provider in Firefox is CloudFlare.

</comment></comment></comment><comment><USER11> >Create a fly account

...nope!

<comment><AUTHOR> There's no good way to host your apps without creating an account somewhere!

You can totally use the Docker image in the article without doing that though. And maybe we should figure out how to make "login with your Hacker News" work.</comment></comment>


Unity Analytics and GDPR meltdown - <AUTHOR> https://forum.unity.com/threads/unity-analytics-and-gdpr.513112/ <comment><AUTHOR> I really wonder what's going to happen if Blizzard (aka hearthstone) or someone else big (Steam) get shot with the GDPR bullet 1st. GDPR has special and more stringent provision when dealing with Minors data. Everybody look at the adtech industry as the one with the highest risk regarding GDPR but the Gaming industry (especially for online and mobile) is the one that can be shot first because of the Children's provisions of the regulation.

</comment><comment><USER2> This won't be pretty. Backend engineering efforts to address this only began a few weeks ago, given unclear "technicalities" that had to be addressed in interpreting the meaning of "right to be forgotten". I'm guessing other companies are facing similar issues, especially if any part of their user data collection pipeline is backed by the "cloud", and these access points are heavily throttled in terms of read/write requests.

You would think that migration efforts would've happened sooner, but alas, like global warming, it's one where it seemed like "waiting until the last minute" was a preferred approach to "invest $$ in this now and use up engineering resources on something that we really don't want to do because it hurts our bottom line".</comment>


HackerNews exceeds Slashdot in unique visitors - <AUTHOR> http://siteanalytics.compete.com/slashdot.org+ycombinator.com/

<comment><USER4> <http://news.ycombinator.com/item?id=2400939>

</comment><comment><USER5> Not positive, but I think newobj is making fun of how inaccurate compete.com numbers are, in light of comments on this post <http://news.ycombinator.com/item?id=2399570> and especially TC's take on compete.com's numbers, <http://techcrunch.com/2011/02/09/compete-on-compete/>

</comment><comment><USER6> Call me skeptical, but I've never trusted Compete's numbers. Their service gives stats without citing sources, so I don't know what kind of magic or data interpolation they use.

<comment><USER7> Without Google Analytics to do a 1-1 comparison, any metric will be suspect. Alexa will suffer from its standard bias of only counting people with the Alexa toolbar installed, which won't be many for either site. Compete's metrics are compiled from a variety of sources [1], including a toolbar, none of which seem particularly definitive.

[1]<http://en.wikipedia.org/wiki/Compete.com>

</comment></comment><comment><USER8> So what caused the massive Slashdot dropoff? I generally gravitate to where the most intellectual conversations are (and hacker news is that place), but Slashdot has been consistently decent for ages.

I will say, Slashdot seems to be mostly made up of a cranky, close-minded older crowd that fails to comprehend current trends. Maybe it's the user base that's the problem?

<comment><USER9> Regardless of the veracity or not of Compete stats, I personally completely removed /. from my daily visited sites about 2 months ago, after 10+ years of it being my #1 morning destination. I have nothing tangible to pin this on - they haven't done the digg-style algorithm implosion; their recent redesign wasn't a gawker-esque breakage which made it unnavigable. It's more subjective, but here are some of the points I can find which I think contribute to my decision, in no particular order:

The 1-2 day lag has increasingly become an impediment to the value of the discussion. There have been comments on /. for a few years such as 'I read this 2 days ago on X'. 4-5 years back there were few viable alternatives for quality discussion where those experienced in the field participated, so this could be tolerated. HN, Twitter and to a lesser extent reddit and specialist forums have meant the discussion has often already happened, and for /. it's more recap on topics hand-selected by the 'editors'...

...against whom the vitriol has been rising. While 10 years back the sources were varied, /. has increasingly become 2-day old metadiscussion on whatever was getting pageviews on the 'tech' linkbait sites - TC, Gizmodo, etc. This has trended, AOL-way style away from real tech and more towards shiny/flamey/'socialmedia-y' puff-pieces. This is a fundamental problem with not trusting your community to select the stories, but relying on hand-chosen entries. Not just the lag, but it's too open to skewing/editorial bias/corporate control. Idle was an early indicator that the corporate overlords were after this, and even ghettoising it there didn't stop the creep of topic away from the more hardcore tech which had initially attracted...

...the greybeards. The dotcom boom created a lot of techs. Many had no experience, and were looking to the older hackers for advice. Now that 'boomer' generation is 10+ years in, and have been developing successful enterprise and their own communities, they have considerably more to offer. These communities have also learned from some of the flaws of /. - <USER2> mentioned the hand-selection, but also...

...+1 funny. This is critical. While sometimes HN can appear a little dry, the actively enforced rule against one-liner witticism forces those of us who are given to facetiousness (myself included) a reason to pause and hopefully construct something more thoughtful. This was always an issue with /. - <USER3> never got Karma for funny, which led to the crazy 'insightful' mods to reward humour. This gaming of the comment system IMO devalued it.

So that's a few things off the top of my head. The primary reason for me leaving wasn't these per se, but more that I found increasingly that I would visit and not see interesting headlines, or would read a thread and find nothing but snarky jibes, meta-discussion about /. and the editors, etc. I felt this degradation of quality exponentially rose in the second half of 2010, and so around Feb this year I completely stopped going. I am sure I'm not the only one and this sort of decline is self-perpetuating.

I'm not happy about it - I credit /. with a huge amount of my versatility these days as a tech, and broadening my horizons on a great many issues. I see HN in some proto-stages of this decline at the moment, and I'm hoping the control structure here will allow the community be self-aware enough and proactive enough to take this experience and make some tweaks to avoid the negative trends that growth in readership/participation bring to all online communities (Usenet, /., Reddit have all seen this happen).

Old stories. Less insightful than HN. Lame.

</comment></comment><comment><USER10> Slashdot isn't what it once was. <http://www.google.com/trends?q=slashdot.org>

<comment><USER11> I see stories go by on Slashdot that were linked on HN days earlier. I used to read Slashdot to get up-to-the-minute tech news. They need to move faster and get rid of all of the cruft on their page. As much as I hate seeing people say cliche'd things like "this is 2011, get with the times", this is 2011, their site design harkens back to their launch and it's just not what I would consider the forefront of usability.

</comment></comment><comment><USER12> Quantcast shows ./ has an order of magnitude more traffic

<http://www.quantcast.com/ycombinator.com> \- 21.3k monthly people <http://www.quantcast.com/slashdot.org> \- 261k monthly people

</comment><comment><USER13> Alexa seems to agree that HN has caught up with Slashdot and is about to overtake it: <http://www.alexa.com/siteinfo/slashdot.org+ycombinator.com>

<comment><USER14> only non-techieidiots have the Alexa spyware toolbar installed.

That only shows that more non-technically proficient visitors check out ycombinator and is not indicative of the total number of users.

</comment></comment><comment><USER15> I don't see anything restricting this to HN, either. Am I missing something? While I imagine that HN is the majority of traffic to YC these days, it'd be silly to pretend the rest of the site wouldn't have visitors HN doesn't.

</comment><comment><USER16> I think this says more about Slashdot than HN...

</comment><comment><USER17> So, where are the cool kids hanging out these days? I like True Reddit, but it's a bit slow...

</comment><comment><USER18> why does 'spam "by moultano"' show up as a top search term?

<comment><USER19> I don't know, but moultano's HN profile says this:

"I work at Google trying to make your search results not suck."

<comment><USER20> He was also pretty active in the discussions about the personal blacklist and ranking change to block out contentfarm sites.

</comment></comment></comment><comment><USER21> I think it's the eventual consequence of the mutation of the dev community. I guess and hope that we won't need to create a new website for better community sharing the content we're interested in. I wish we had a mixage of Torrent and Github to share any kind of content; article, links, video, picture, even spam texts...</comment>


IPhone 4G, Google Wave, Google Voice; Collaboration Transformed - <AUTHOR> http://iphonecto.com/2009/06/30/2010-iphone-4g-google-wave-google-voice-collaboration-transformed/ <comment><USER2> I am always suspect of these type of articles about bleeding edge technology transforming business. Steve Gilmour is notorious for this w/ twitter and xmpp.

While all the jargon sounds great, it is very detached from the reality of most IT departments who only buy software because other well known companies are using it - not because its a great transformative technology.

</comment><comment><USER3> _But this particular convergence will also enable new types of communications processes to become part of every-day scenarios._

Eeek. I think he means "People will use it to communicate."</comment>


Stop and Seize - <AUTHOR> http://www.washingtonpost.com/sf/investigative/2014/09/06/stop-and-seize/ <comment><USER2> I didn't find this listed, so obligatory Last Week Tonight with John Oliver link[0].

The extent to which authority figures intervene in normal citizen life in the US always seems like some medieval European feudal lord stuff to me.

[0] [http://www.youtube.com/watch?v=3kEpZWGgJks&list=PLmKbqjSZR8T...](http://www.youtube.com/watch?v=3kEpZWGgJks&list=PLmKbqjSZR8TZa7wyVoVq2XMHxxWREyiFc)

</comment><comment><USER3> Organizing the police to be parasites rather than predators of narcos. A few steps away from 'payola' but still.

</comment><comment><USER4> Yet another reason why you _never_ talk to the police, and _never_ consent to a search.

[https://www.youtube.com/watch?v=6wXkI4t7nuc](https://www.youtube.com/watch?v=6wXkI4t7nuc)

<comment><USER5> Good luck with that. I agree with you, but it's hardly a solution given that you can be arrested on practically any pretense.

<comment><USER6> Unless you know the right magic spells to invoke in order to keep the demons away. I know one that goes, "I'm not consenting to any searches. Am I free to go? Am I under arrest?" If you keep chanting it in the right tone, there's a good chance you'll escape.

<comment><USER7> "...there's a good chance you'll escape."

I'd love to see a citation on that, because I can easily see that going the other way: cop considers you a smart-ass and decides to teach you a lesson. Maybe you appear nervous or he/she claims that your eyes are glazed, and now has probable cause to detain you further and get a warrant to have your car torn apart looking for drugs or other contraband.

Yes, you'll eventually win in court if you're truly innocent and you keep saying you don't consent to searches, ask if you're free to go, and ask for a lawyer. Of course, that doesn't mean the officer can't make your life hell during and after the stop if they so desire. The point is, they have the power, not you. You can't even leave until they say you can. The best you can hope for is that the cop maybe gets a reprimand after you complain, and good luck getting any compensation beyond that.

</comment></comment></comment></comment><comment><USER8> 1 in 12 households are unbanked - [https://www.fdic.gov/householdsurvey/](https://www.fdic.gov/householdsurvey/)

I'd bet far more of these people are getting being hurt by this than the "bad guys"

<comment><USER9> So about 8%. Bordering on market insignificance. Insignificant in a lot of areas but definitely not the norm.

<comment><USER10> >So about 8%. Bordering on market insignificance.

That's crazy talk.

<comment><USER9> Some web development companies drop support for browsers when their versions drop below 10%. Not crazy at all.

</comment></comment></comment></comment><comment><USER11> Highway robbery.

</comment><comment><USER12> Old news, but who's doing anything about it?

<comment><USER13> I did something - I flagged it as being off topic in that it's 1) politics and 2) mainstream news.

[https://news.ycombinator.com/item?id=6120530](https://news.ycombinator.com/item?id=6120530)

I'd also recommend that people get more involved in politics to stop stuff like this. _However_ , please do that elsewhere - there are _tons_ of places on the internet to talk politics, but only one Hacker News.

<comment><USER14> I see you are being down voted. I do understand your view point, but I don't see where politics comes in and unfortunately this story is far from mainstream. Now maybe if this happened to Kim or Miley then it would be mainstream, off topic, and something would change.

<comment><USER13> I don't care if you folks downvote me. I'd rather that than you all transforming HN into a site for politics and "outrage" articles, and ruining it.

The Washington Post is a fairly mainstream news organization, isn't it?

It's very, very much a political and "outrage" story. Does anyone really feel 'intellectually gratified' after reading it? I know I don't. I feel angry. To me that means it's not really something that belongs here. This isn't a site for political activism.

</comment></comment></comment></comment><comment><USER15> What's interesting to me, in addition to the obvious major point, is _how_ this process is taught: in private seminars.

I have seen this in other areas involving questionable practices, especially involving law enforcement. Training agencies put together professional seminars- just as any of us might take a programming seminar- and teach lessons that "improve" the officers' skills, but at the same time, in the privacy of the seminar, strain the boundaries of what's legal.

That's the part that I find fascinating. In contrast to larger-scale fascist movements, in which anti-national behavior is spread from the top-down (I'm thinking of slave-era American and modern Arabic racism here or the classic example of Nazism), this type of nation- and rights-destroying behavior emanates from numerous, distributed "educational facilities."

It's fascinating to me because there's no single identifiable point to put the finger of blame on, because the erosion happens so gradually, the damages to the common good take place in such tiny increments and because it's not motivated by a common, uniting goal (although law enforcement is the simulacrum), but by numerous distributed organizations motivated by making cash via the trainings and who motivate the participants the same way. It's as if multi-level marketing were getting applied to professional gang-banger training.

And through the logic of "training seminar," why exactly _not_ produce week- long seminars in gang-banging? It's not illegal to teach people how to break the law, there is a need in the form of customers who want to know how to do it and there is money to be made in educating others in the arts of carjacking, robbery and drug sales using slides, role-playing and case studies. It is up to the customer whether or not to make use of the information...

One final point is that there is an implied escalation of consent in the "professional" veneer of the "training seminar." This happens due to our assumptions that 1. spending money on a seminar results in at least some useful information 2. information learned in a private seminar is not expected to be unethical or useless. This leads to training seminars on law enforcement (and several other topics) operating as an "offshore zone" where unacceptable behavior gets legitimized: The police chief could never train their officers to perform manipulative, borderline illegal practices, but he can easily sign off on a training where these practices occur and become part of their department[s repertoire. The kinds of conversations and education that could never take place in a public-supported institution such as a police station can easily happen outside the publicly-supported sphere, in the conference room of a hotel, led by a small, scarcely memorable, unaccountable training organization.

<comment><USER16> >It's not illegal to teach people how to break the law

I'm sorry, do not pass go, do not collect $200, go straight to jail.

[https://www.techdirt.com/articles/20130908/00044524444/eight...](https://www.techdirt.com/articles/20130908/00044524444/eight- months-jail-teaching-people-how-to-pass-lie-detector-test.shtml)

[http://www.theregister.co.uk/2013/09/09/polygraph_countermea...](http://www.theregister.co.uk/2013/09/09/polygraph_countermeasures_man_jailed/)

[https://news.ycombinator.com/item?id=6308878](https://news.ycombinator.com/item?id=6308878)

<comment><USER17> > I'm sorry, do not pass go, do not collect $200, go straight to jail.

That's a disproportionate amount of bluster given that 1. That was a _single_ counterexample. 2\. The article you posted says:

Federal District Judge Liam O'Grady said: "There's nothing unlawful about maybe 95 per cent of the business he [Dixon] conducted" but found criminal fault in his willingness to assist would-be applicants and others to lie to federal agencies, the Washington Post reported.

In other words, the polygraph-beating information was not illegal, but he was prosecuted for helping people lie to federal agencies (a crime itself). Undercover agents ensnared him via a technicality. But if the class were restricted to polygraph-beating, without persuading students to lie to federales, he wouldn't have served his relatively minor sentence of 8 months.

Contrast _one guy_ going to jail for 8 months with the _hundreds of millions_ of dollars seized from people not charged with crimes in the original article. Consider the zillions of materials and resources available for learning to assault someone, to hack into computer systems, to break locks, websites on "how to rob someone" (google it) and make explosives and the distinction is clear.

<comment><USER16> That is a single counterexample simply because I decided to post a single one. Working in the firearms industry for years has shown me that if the ATF/FBI don't like what you are doing they can legally make your life hell. 'Someone' makes an anonymous tip, judge rubber stamps a warrant to search all your premises. Two parts from unrelated weapons in the same building, conspiracy to build a machine gun.

What you don't understand is, you don't understand the full technicalities of the law and your adversary does. Does it really matter if they don't convict you in the end if you exhaust all your resources fighting them on lawyers?

</comment></comment></comment></comment><comment><USER18> Is this an opportunity for a startup to automate the process of retrieving money?

Like [http://www.getfixed.me/](http://www.getfixed.me/) but on a more serious scale!

(I'm not in the US so not something I can do)

<comment><USER19> This is a good and socially valuable startup idea. The next time HN has a thread about startup ideas, I hope that you (or someone else reading this) posts this.

Kudos for posting this.

</comment></comment><comment><USER20> Previous discussions from submission ~month+ ago. [https://news.ycombinator.com/item?id=8280889](https://news.ycombinator.com/item?id=8280889)

</comment><comment><USER21> Related discussion from 2 days ago [https://news.ycombinator.com/item?id=8444785](https://news.ycombinator.com/item?id=8444785)

</comment><comment><USER22> This has been going on to some extent for decades. Now that John Oliver (a satirist) does a story it's all of a sudden front page news?!

<comment><USER23> I first read about this in Reader's Digest in the 1980's or early 1990's. Yes I am annoyed by the fact that nobody gave a shit until a TV funnyman talked about it decades later, accompanied by a million Facebook posts of "why is it that only comedians talk about this!!!1" Contrary to popular opinion, this is the sign of a public that doesn't read, doesn't actually care about news, and isn't informed. John Oliver gets his story ideas from the news, it's all there if his fans cared to look.

<comment><USER24> I think it's really plain to see how even the most educated youth today think they're getting culture enough from TV series, documentaries, Facebook links and Wikipedia trips. Who ever reads a longform piece of good journalism or (gasp!) a book anymore? I guess the answer is those who are force-fed this ancient and boring medium by means of still being able to afford a citizenship-minded Liberal Arts degree that would otherwise implicate them in more debt than their projected lifetime employment value.

</comment></comment></comment><comment><USER25> If you need to carry "seizable" amounts of money around, you should put that in an envelope along with a written statement that describes the purpose. And if possible, the envelope should be sealed by a solicitor.

I like to see what happens then. It should be a sufficient hurdle.

<comment><USER26> You mean like in multiple of the examples that are cited in the article? Lay church members carrying funds from their congregation, going to purchase property for the church (well, the police said "both of them disclaimed ownership of the money, so it was seized" - no kidding, because it was for the church).

Or the gentleman buying a car, who had the car ad, contact information for the prospective seller, who confirmed that he had an appointment to meet that person? "No, based on my investigation I have concluded this is money for a drug purchase".</comment></comment>


Why it's time to support only Android 4.0 and above - <AUTHOR> http://dannyroa.com/2013/10/17/why-its-time-to-support-only-android-4-0-and-above/ <comment><USER2> Site is giving me a database error, so I'm committing the sin of posting based only on the title :(

Hipster Mountain View/Palo Alto is not the world. Most people who manage to afford to buy a smartphone, use the thing until it dies. I personally _just_ upgraded from an Android 2.3 device, and my latest Apple device is on iOS 2. And I read Hacker News for christ's sake.

Maybe for your app, your community the norms are different. But be careful not to bias your data - you better be sure that you are not unknowingly turning away people who would be customers if only you supported their device.

<comment><USER3> Even in the world outside of "hipster palo alto", 70% of active Android users are on 4.0+: [http://developer.android.com/about/dashboards/index.html](http://developer.android.com/about/dashboards/index.html) It seems to be growing 5% per month, so already by the end of the year 2.3 market share might fall below 10%.

Starting from 4.0, _many_ APIs were cleaned up, many bugs were fixed, and it seems that many carriers/manufacturers have stopped shipping bizarrely broken custom modifications.

Unless you have a large development team and absolutely must reach every possible customer, targeting 4.0 and upwards seems to be the most efficient choice.

<comment><USER4> FWIW, remember that these figures are biased in construction.

Less importantly, they only take into account devices that have the Play Store (which you might consider your audience, but there are tons of people in the world happy to buy things from alternative markets available on their devices).

However, and this is somewhat insidious: they measure devices used in the last seven days. How often do normal users "actively visit the Play Store"? I find it highly unlikely that every normal user does so at least once every week.

In fact, I would be much less surprised by a universe where people only "visit the Play Store" once a month at best, and only when there's something specific for them to be doing (maybe they hear about a new product, and go to download).

I would then further submit that this frequency decreases the older the device is: during the first month of owning your device, you probably have tons of things you suddenly realize you want, because the device comes with no apps installed.

With an expectation curve on the usage of the device thereby working against older devices, you will see a chart that only looks at the most recent 7 days biased towards recent versions as the probability of older versions dips.

Let's look at what happens then if we posit that the average 2.3 user is using the Play Store once every four weeks, while the average 4.x user is using the Play Store once every two weeks: this mild skew makes 30/70 become almost 50/50.

Are these people who are using 2.3 really outside of your target market because they only download an app every month instead of every couple weeks? If you market to them, and make it clear your app supports them, will they not buy it?

While that might be true, it isn't something you can support with just these numbers from Google. It's even worse, in fact, when you remember that when Google was tracking "active devices" instead of "devices actively visiting Play" the 2.3/4.x split was so far in the other direction as to be unbelievable, and when Google decided "fine, we are limiting it to active users" the numbers jumped.

And now, we see the numbers creeping better and better... but did you notice that they changed the time range from 2 weeks to 7 days in the few months since Apple lambasted their uptake at WWDC? If not, you aren't paying enough attention ;P, as that's a really critical detail: Google keeps moving the goal post, so you can't take any solace in "we are making progress".

</comment></comment></comment><comment><USER5> It's amazing to me that we're saying that a three year old OS is too old to have software written for it. Especially given that a part of the laptop market has metamorphosed into the tablet market.

I guess that's not a problem if you buy new tablets/phones every year, which is what we're moving towards.

<comment><USER6> Maybe it will not be the same when Android 4.0 turns 3-years old.

Android 4.0+ is just much more mature than 2.3 in terms of UI and API.

<comment><USER7> > _Android 4.0+ is just much more mature than 2.3 in terms of UI and API._

Maturity is a very important point. Between 2.3 and 4.0, Android became scale- able across tablets and handsets, with unified APIs enabling a unified code- base.

Compared to that big change, dealing with API differences from Android 4.0 to 4.4 is a relatively small matter, comparable to testing for hardware capabilities (and often tied to hardware capabilities). You can test for the API version on the fly and keep a unified code base.

</comment></comment></comment><comment><USER8> I feel that supporting Android 2.3 at this time is roughly equivalent to supporting IE6. For the majority of people, just stop targeting 2.3 support as a requirement. Perhaps it'll net you some additional marketshare in emerging markets, but for the most part, ≥API14 is ubiquitous to a point where this is a sensible decision.

<comment><USER9> <API14 is 30.8% of the user base.

[http://developer.android.com/about/dashboards/index.html](http://developer.android.com/about/dashboards/index.html)

IE6 is 4.9%

[http://www.ie6countdown.com/](http://www.ie6countdown.com/)

They're completely different.

<comment><USER10> Yea, I think that's a pretty bad comparison. If you look at the IE graph you posted, it's less than 1% in the US.

Another graph that might make things easier to visualize (the blue and above is who we're talking about abandoning): [http://en.wikipedia.org/wiki/File:Android-dist-by- dessert.pn...](http://en.wikipedia.org/wiki/File:Android-dist-by-dessert.png)

<comment><USER8> I will grant that it is a bad comparison. But, not for percentage based reasons. I initially wrote the comment that abandoning <14 now is like abandoning ie6 back in 2010, when google dropped support[1], but percentage wise, that's not fair either.

Since Android API 14 is just celebrating its 14th birthday, I'd guess (nothing to back this up) that you're going to see a sharp dropoff of Android 2.3 marketshare as people replace their old phones. Between developing for the future, and incentivising people to switch away from old technology (people don't want to buy an old phone that can't run the latest goodies), I think this is still the correct path to choose.

Kill it with fire. I don't take much issue with making Android 2.3 a second- class citizen at this point.

[1] [http://gs.statcounter.com/#browser_version-ww- monthly-201001...](http://gs.statcounter.com/#browser_version-ww- monthly-201001-201001-bar)

<comment><USER10> >I'd guess (nothing to back this up) that you're going to see a sharp dropoff of Android 2.3 marketshare as people replace their old phones... Kill it with fire. I don't take much issue with making Android 2.3 a second-class citizen at this point.

You could very well be right, but I think you have the wrong impression of the users.

We view IE6 users as a completely technology inept minority. They could upgrade to a better browser in seconds (not including the enterprise).

Android users on the other hand? I've been an Android user for _4 years_. I got tired of getting gouged by Verizon and just switched to Republic wireless. So I have a _brand new phone_ and I'm on 2.3.

Even leaving the percentages out of it, I don't think it's appropriate.

<comment><USER8> I'm not going to lie--I think buying any device running 2.3 at this point is a giant waste of money, comparable to buying a laptop with Windows XP on it. There is no reason. Nexus 4s are recently out of stock at $200, and the Nexus 5 will probably compete in that area too.

It's a shame that republic wireless is handing out 2.3 devices--that really should stop.

<comment><USER10> It seems like there is some backstory behind the phone and the versioning (this is the only phone currently offered until the Moto X next month, which has 4.2) [http://en.wikipedia.org/wiki/Motorola_Defy#Manufacturer_crit...](http://en.wikipedia.org/wiki/Motorola_Defy#Manufacturer_criticism)

Sorry you think it's a "giant waste of money." I need a phone and I'm now saving something like $40/month over my previous plan.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I'ma let him finish when his site comes up, but here why I generally recommend targeting Android 4:

1\. If you want to make a UI that scales across all device sizes, and you don't want to use hacks like ActionBarSherlock that will become a maintenance headache after the moment of convenience, build your app skeleton around Android 4 APIs.

2\. Targeting Android 4 avoids use of the Support Library for back- compatibility, and therefore avoids using FragmentActivity. This isn't as bad as some of the back-compatible action bar hacks, but it is still an unnecessary complication in most cases. N.B. you may still want to use the Support Library for new APIs like PagerAdapter.

3\. Tablets are going to increase in importance, and for some apps, tablets are the main target. Center your implementation strategy around tablets as the ideal experience, with multiple fragments on each tablet-size layout that are "unrolled" into sub-activities when running on a handset.

4\. If you think the above point implies a unified code-base, yes, it does. Targeting Android 4 and above will simplify your unified code base.

The exception is apps that MUST target every user, and every device in the field. For example, my consultancy had an engagement with a dating site that needed to upgrade their Android app. After weighing the pluses and minuses, the conclusion was that it was better to put in more effort and incur some technical debt to reach every possible user. On the other hand, if you are writing an enterprise app for a green-field deployment, don't even consider supporting old versions.

<comment><USER6> Site is back up!

<comment><USER7> Excellent article! I'll link it on G+ and on the _Programming Andorid_ page.

I would add to the list the fact that people buy apps most often when they have just bought a new handset. Supporting old versions is like making a retrofit product. The market is inherently limited and more costly to support.

</comment></comment></comment><comment><USER11> Just remember that in some countries Android 4.0 is yet to be the norm...</comment>


Bad Washington Post Story on Apple and Google's Exposure Notification Project - <AUTHOR> https://daringfireball.net/2020/05/washington_post_exposure_notification_story <comment><USER2> On the other hand, there is some evidence from an app already in widespread use in Iceland that says it might be basically useless:

[https://news.ycombinator.com/item?id=23204410](https://news.ycombinator.com/item?id=23204410)

<comment><USER3> To be fair, the app was released at the tail end of the infection curve so there wasn't much pandemic to affect anymore. Could be useless, but I don't think this was a good test to generalize with.

</comment></comment><comment><USER4> There is spin in this analysis.

Apple and Google have a huge amount of power when it comes to what capabilities they allow apps to use on their device. So when the article mentions "sovereign power" and "unbending stance", it is referring to that. It is impossible for a user to run an app that collects tracing and location data in the background given the restrictions the OS puts in place. This definitely is better from a privacy standpoint, but it is these companies making the call that it's better for society.

Google and Apple are the ones that decide what data is collected and who it is shared with. It is pretty remarkable how much faith we must have in them.

<comment><USER5> > Google and Apple are the ones that decide what data is collected and who it > is shared with. It is pretty remarkable how much faith we must have in them.

No, Google and Apple are one of a number of entities that have the ability to put a maximum on the data shared. Governments are free to pass laws that limit data collection and sharing (and some have!), and individual organizations are free to limit their collection and sharing. To an extent, individuals are also free to limit what data is collected about them, though I'll grant that this is difficult for many users.

Do you really want to go the route of compelling Apple or Google to collect data on behalf of a government?

</comment></comment><comment><USER6> It's intuitive that contact tracing APIs, as Google and Apple PMs have conceived them, somehow help you. Some health officials have an opinion on what kind of software would help them, saying something extremely well publicized by Apple and Google will not. This conflict plays out with masks, taking flights, not being able to smell as a diagnostic, and any number of widely reported guidance that has some evidence, some intuition, tremendous exposure in social and traditional media but no conclusive, registered control trialed (RCT) results.

A line of logic that goes, "in the absence of conclusive evidence, play it safe" doesn't give you an answer for "track locations and surveil, to collect more data and answer questions better, versus collect a minimal amount of data in the interest of some other kind of safety called privacy." There's no universal rationalist approach to your opinion here. You have to decide whether to trust John Gruber or whatever health officials WashPo quoted.

RCT is a real line for people in medicine, at least for people who aren't looking to make a buck developing a drug. In the same way health officials don't strictly care if the stock market goes down, they wouldn't strictly care if Google and Apple's brand would be impacted by detailed location tracking. At the end of the day health officials, like the UCSF prof quoted here, just want people to be safe and healthy.

The counterpoint is, what do Apple and Google want? To do something? The appearance of doing something? They are not health organizations. John Gruber isn't a doctor. These are different perspectives and goals, not an argument over how to achieve common goals. Why crank out and die on the hill of a giant company's PR team?

<comment><USER7> > At the end of the day health officials, like the UCSF prof quoted here, just > want people to be safe and healthy.

If only it were so simple. Publicizing private data is a one-way operation that has longer term effects on safety and health, and we should certainly not do it without a very strong reason to do so. Health officials will not be accountable for the long term effects of their actions. Apple and Google will, so they are somewhat more incentivized to act to protect their users' privacy.

</comment></comment><comment><USER8> The misinformation has been extensive, even in this site where some supposed technologists don't even understand the privacy protections.

</comment><comment><USER9> The Daring Fireball guy is very hurt hat someone is criticizing his beloved Apple, but there's at least a pinch of truth in the criticism: Apple and Google feel they're not only entitled, but own, the data they have on us. That they're not willing to share it even in a health emergency even with democratic governments is iffy.

This should be at least user choice, if not decided at the ballot box. Certainly not by shareholders. Especially since it seems the app in question is, as a result, not very useful.

<comment><USER10> The data isn't stored by Apple or Google, it remains on the device. Users have to decide whether to share their location history with the government or health authorities. Instead of automatically pushed every few minutes as one their sources insisted was necessary.</comment></comment>


Do you live near a meth lab? - <AUTHOR> http://money.cnn.com/interactive/news/meth-lab-map/

<comment><AUTHOR> This was referenced in an article linked from this thread:

[https://news.ycombinator.com/item?id=8199503](https://news.ycombinator.com/item?id=8199503)

Looks like they're using D3.</comment>


What causes Ruby memory bloat? - <AUTHOR> https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html <comment><USER2> The author never made clear if they are measuring _virtual_ memory usage, or _physical_ memory usage. Having a lot of virtual memory does not "cost" much: it's just having permission to use a lot of memory if you so wish. It's possible to have huge chunks of your virtual address space with no physical memory backing them. Physical memory is the amount of physical RAM your process is using.

Memory managers tend to greedily request a lot of virtual memory because there's usually little harm in doing so, and the act of asking the kernel for more permission (read: more virtual memory) is slow.

Minor quibble: it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." That is the memory allocator for C, and Ruby just so happens to be implemented in C. The glibc allocator will use the system calls `brk()`, `sbrk()` and/or `mmap()` to request memory from the kernel ([http://man7.org/linux/man- pages/man2/brk.2.html;](http://man7.org/linux/man-pages/man2/brk.2.html;) [http://man7.org/linux/man-pages/man2/mmap.2.html](http://man7.org/linux/man- pages/man2/mmap.2.html)). Nothing really changes per the punchline.

<comment><USER3> > Having a lot of virtual memory does not "cost" much

To some extent, I agree, at least on 64 bit systems (on 32 bit systems, there was the risk of running out of address space).

However, the memory in question here is, for the most part, not freshly allocated, but was in use once. This means that it used to be backed by physical memory, and before that backing can be withdrawn, the page has to be written to disk.

It seems to me that the best solution would be to call madvise(MADV_FREE) on these regions, in which case they can be unbacked without further ado. I'm somewhat surprised that the memory allocator does not do this itself already.

<comment><USER2> > However, the memory in question here is, for the most part, not freshly > allocated, but was in use once.

Because the author does not differentiate between virtual and physical memory, I can't agree with that. It's quite possible most of the memory "freed" was never in use. In which case, there's not much benefit. And there is the probable downside that allocation heavy applications will pay a lot more.

<comment><USER3> > It's quite possible most of the memory "freed" was never in use.

Given the allocation patterns shown, it would seem to me that if two blocks are still in use, it's fairly likely that the region in between also were in use once (there may be exceptions due to pools etc, but generally memory is parcelled out in a linear fashion).

> And there is the probable downside that allocation heavy applications will > pay a lot more.

What would the cost be? All that would happen is that the free pages are marked as clean. I'm sure that's not entirely free, but bound to be considerably cheaper than paging the page out and in again.

<comment><USER2> > What would the cost be? All that would happen is that the free pages are > marked as clean.

That's a minor page fault. You get an OS-level exception, switch to kernel mode, process the page fault by marking the page as loaded, then switching back to user mode. That is expensive if you do it a lot.

> but bound to be considerably cheaper than paging the page out and in again.

Yes, of course, a minor page fault is cheaper than a major page fault. But both are more expensive than _no_ page fault, which is what happens if you just never free the page to the OS and there's plenty of available physical memory.

</comment></comment></comment></comment></comment><comment><USER4> Here are some benchmarks on an 8-year-old, 4-core i7 running OSX Sierra. Parsing a 115Mb log file for lines containing a 15-character word (regex: \b\w{15}\b) we have:

      LANG / TIME* / RAM

      JS (Node 11.11) / 8.4s / 100Mb
      Ruby 2.6 / 19.1s / 14.8Mb
      PHP 7.3 / 4.4s / 5.6Mb
      Python 3.7 / 24.7s / 4.2Mb
      Perl 5.26 / 14.0s / 1.0Mb

*These figures are for runtime, ie. with startup time deducted. Ruby's startup time (0.55s) is much longer than the other languages (Python:0.06s, Perl:0.02s, PHP:0.14s).

Ruby's memory usage is 3.5 times that of Python for only a 30% speed gain. Perl uses 1/15 of the RAM used by Ruby and is 35% faster but it could be argued that Perl 5's lack of built-in OO accounts for some of this ..... until you look at PHP which has built-in OOP and uses 2/5 of the RAM used by Ruby whilst performing 3.2 times as fast.

I love that Ruby is designed from programmer happiness but the shine starts to wear off when you look at its memory usage. Slow is bearable as it's only marginal but Ruby's memory usage is orders of magnitude higher it seems. Matz's goal of making Ruby 3 times faster is only half the battle, maybe even only a third. If an increase in speed comes at the expense of even greater memory use then Ruby will not survive.

<comment><USER5> If memory mattered that much we'd all still be using Perl.

<comment><USER6> Some of us still are. And not just because of memory. :)

</comment></comment></comment><comment><USER7> In classic glibc form, malloc_trim has been freeing OS pages in the middle of the heap since 2007, but this is documented nowhere. Even the function comment in the source code itself is inaccurate.

[https://stackoverflow.com/questions/15529643/what-does- mallo...](https://stackoverflow.com/questions/15529643/what-does-malloc- trim0-really-mean)

<comment><USER8> I'm used to a random comment on stack overflow being the source of truth for angular, django and the like. But this is the first time I've seen it for glibc!

</comment></comment><comment><USER9> The reason glibc malloc doesn't like to free random pages in the middle of a mapping is probably because that inflates the number of PTEs needed to describe it. Say you have one mapping and free a page in the middle of it - now you need to PTEs to describe that. Similarly the mapping shown in the last image probably requires a few dozen PTEs to accommodate the holes.

That isn't free (it requires cache & TLB space), but it's entirely possible that Ruby is slow enough on the interpreter and data model level for this to not matter much.

Edit: Turns out malloc_trim doesn't actually modify the mapping but rather uses madvise(DONTNEED), so a higher address resolution cost probably only materializes under memory pressure.

</comment><comment><USER10> It really makes sense that something like this would be the case.

All the experts say "oh, Ruby uses lots of memory for [reason] and it can't really be fixed", so no one even tries.

Until someone comes along who is either motivated, smart, or ignorant(!) enough to try to fix it anyway, and finds that the commonly accepted answer was wrong.

This happens all the time, especially in science. Trust, but verify, I suppose.

<comment><USER11> > All the experts say "oh, Ruby uses lots of memory for [reason] and it can't > really be fixed", so no one even tries

This isn't true at all. It's well understood that jemalloc 3.x exhibits lower resident set size because it more readily releases pages.

This idea has been around for at least 3 years: [https://bugs.ruby- lang.org/issues/12236](https://bugs.ruby-lang.org/issues/12236)

</comment></comment><comment><USER12> In the context of Rails applications hosted on EC2, then I've not found Ruby's memory usage to really be an issue. In my experience most Rails apps range between 150-500MB per instance.

My current employer typically uses M5 instances which have a ratio of 1 vCPU : 4 GiB Ram.

Running Unicorn you'll probably only want 1.5 instances per vCPU. Even a memory heavy Rails app is probably only going to utilise ~20% of the available memory.

Running threaded Puma, you probably want only a single process per vCPU and maybe 5-6 threads. In my apps running 5 threads per process typically increases memory of the process by 20%. So in that instance you'd only utilise 15% of the available memory on a M5 instance.

If you are having memory issues on Rails, then quick wins are upgrading your Ruby version. I saw 5-10% drop in memory usage with each of the major version 2.3.x -> 2.4.x -> 2.5.x.

Also if it is an old app, check you've not built up cruft in your Gemfile. Removing unused gems can be another quick win for reducing memory usage.

</comment><comment><USER13> I love it. When you think you know what the issue is,

      * prove it is really the issue
      * fix it
      * prove that you fixed it

</comment><comment><USER14> I know that a Ruby shop naturally wants to use Ruby for everything, but when the job described is:

a simple multithreaded HTTP proxy server written in Ruby (which serves our DEB and RPM packages)

then I would reach for Linux ipvs or haproxy, and apache or nginx to do the serving. Good tools already exist for these things, it's a shame not to use them.

(And we have a Ruby dev group, so please don't accuse us of having a phobia or hatred of Ruby.)

<comment><USER15> This is a pretty old piece of software, it might predate nginx. Or if it doesn't, it probably does some non trivial url rewriting or other logic that would be a pain to do in nginx.

In any case it's just something thrown together to solve a need quickly and effectively. The performance characteristics might not even have been an issue, they just caught his eye.

<comment><USER16> nginx has been around since 2004. haproxy since 2001. httpd since 1995.....

Re-inventing the wheel is almost certainly harder than learning the configuration syntax of any of these.

<comment><USER15> Writing an http proxy is 5 lines in node.js, and not much more in Ruby. I could do it in either without even consulting a reference. I spent hours learning nginx configuration, and could spend hours more. If all I need is a simple proxy with some logic why do it? It's not reinventing the wheel, it's building a wheel that's good enough, using the materials you got.

</comment></comment></comment></comment><comment><USER17> I recently spent weeks investigating a very similar issue in a Haskell program, giving deep into its memory manager and glibc's malloc.c (where I found multiple bugs, showing that much code in there appears to never have gotten proper review in decades).

Writing a memory visualiser is exactly what I needed and planned to do next, so this is a great contribution for anybody working on problems like this.

</comment><comment><USER18> Tracking the MRI enhancement at: [https://bugs.ruby- lang.org/issues/15667](https://bugs.ruby-lang.org/issues/15667)

</comment><comment><USER19> This was a really interesting read. It makes me wonder though - are other languages affected by this? I haven't heard any similar reports from, say, Java or Python.

<comment><USER20> I have experienced similar memory fragmentation problems with Python 2.x.

Java has a compacting garbage collector. So it should not be affected.

<comment><USER21> Except that heap fragmentation was _not_ the issue the author identified...

</comment></comment></comment><comment><USER22> Interesting, I read about this from here: [https://medium.com/@floriendrees/a-poodle-is-a-dog-ruby- is-n...](https://medium.com/@floriendrees/a-poodle-is-a-dog-ruby-is-not- rails-83f8d1bb4f0e)

</comment><comment><USER23> Excellent post! I didn't quite understand why MALLOC_ARENA_MAX=2 outperforms this solution (slightly) and what are the trade-offs exactly ... can anyone shed more light on this?

<comment><USER24> I'd have to look more deeply myself, but it likely means larger allocations up front, to avoid having to call out to the OS to make smaller allocations. So if you know you're going to use the larger allocations anyway it'll probably be almost no trade off but if you aren't sure you'll probably use more than you needed.

<comment><USER25> It doesn't mean larger allocations, it means more allocations. glibc will try to use different arenas for different OS threads to avoid lock contention, if you allow a lot of arenas then malloc will allocation a lot of chunks of memory assuming you have lots of threads requesting memory.

</comment></comment></comment><comment><USER26> It looks like OP is victim of something similar to [https://sourceware.org/bugzilla/show_bug.cgi?id=23416](https://sourceware.org/bugzilla/show_bug.cgi?id=23416), and malloc_trim is only papering over it.

</comment><comment><USER27> Source code of the visualizer: [https://github.com/FooBarWidget/heap_dumper_visualizer](https://github.com/FooBarWidget/heap_dumper_visualizer)

Best thing is this can be called directly from Ruby with FFI.

</comment><comment><USER28> I call GC.start() manually in various places. It seems to tame memory usage.

<comment><USER29> When you GC.start() you are probably forcing a round of garbage collection. I assume it is slow to run GC.start(), but after gives you some runtime advantages, but only for a bit (until you call it again), but then the heap fills up again. You can tune the RUBY_GC_HEAP_GROWTH_FACTOR in GC.stat after initializing and load testing a few times to avoid having to call GC.start() and manually force garbage collections by allocating enough space for the system to run its own processes, while minimizing initializing unutilized memory or having too small of a heap size, which will trigger too many garbage collection runs, both requiring expensively slow kernel system calls.

</comment></comment><comment><USER30> I hope someone submits this as an issue to the ruby repo, bc this is awesome.</comment>


Has HTC completely abandoned the Hero? - <AUTHOR> http://www.htc.com/mea-en/search.aspx?cat=0&page=0&q=hero No trace of it on the website. <comment><USER2> [http://shop.sprint.com/en/software_downloads/pda_smartphone/...](http://shop.sprint.com/en/software_downloads/pda_smartphone/htc_hero.shtml)

</comment><comment><USER3> They just released an update to 2.1 yesterday.</comment>