Post mortem of a failed HackerNews launch - <AUTHOR> http://www.gigpeppers.com/post-mortem-of-a-failed-hackernews-launch/ <comment><USER3> A few things have caught my attention in your post.

Your biggest problem was that the configuration of your services was not sized/tuned properly for the hardware resources you've got. As a result of this your servers have become unresponsive and instead of fixing the problem, you've had to wait 30+ minutes until the servers recovered.

In your case you should have limited Solr's JVM memory size to the amount of RAM that your server can actually allocate to it (check your heap settings and possibly the PermGen space allocation).

If all services are sized properly, under no circumstance should your server become completely unresponsive, only the overloaded services would be affected. This would allow you or your System Administrator to login and fix the root-cause, instead of having to wait 30+ minutes for the server to recover or be rebooted. In the end it will allow you to react and interact with the systems.

The basic principle is that your production servers should never swap (that's why setting vm.swappines=0 sysctl is very important). The moment your services start swapping your performance will suffer so much that your server will not be able to handle any of the requests and they will keep piling up until a total meltdown.

In your case OOM killing the java process actually saved you by allowing you to login to the server. I wouldn't consider setting the OOM reaction to "panic" a good approach - if there is a similar problem and you reboot the server, you will have no idea what caused the memory usage to grow in the first place.

<comment><USER4> Appengine.

You're a development shop, not scalable system builders. Deciding to build your own systems has already potentially cost you the success of this product - I doubt you'll get a second chance on HN now. If you were on appengine, you'd be popping champagne corks instead of blood vessels, and capitalising on the momentum instead of writing a sad post-mortem.

I'd recommend you put away all the Solr, Apache, Nginx an varnish manuals you were planning to study for the next month, and check out appengine. Get Google's finest to run your platform for you, and concentrate on what you do best.

<comment><USER5> I wish I could vote this comment up 10 times over.

I know that I know little to nothing about sysadmin, so when I built a recent app I used AppEngine for this very reason. And when it got onto the HN front page it scaled ridiculously easy without any configuration changes. (No extra dynos, no changes at all.)

And when I've occasionally screwed up and done stupid stuff, it still doesn't go down. (To be honest, I first saw the problem when I noticed my weekly bill was ~$5 instead of the baseline $2.10. It helped that being a paid app pushed the limits up a lot higher.)

</comment></comment></comment><comment><USER6> 1\. Reduce keepalive, even with nginx 60 is too much (unless it's an "expensive" ssl connection).

2\. set _vm.swappiness = 0_ to make sure crippling hard drive swap doesn't start until it absolutely has to

3\. Use _IPTABLES xt_connlimit_ to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddos. If you are reverse proxying to apache, connlimit is a MUST.

<comment><USER7> Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth.

<comment><USER3> _> Don't bother reducing keepalive, just disable it altogether. Unless you have a very specific use case it is more trouble than it is worth._

Bad idea. This way you're actively increasing the latency of your site. This way, for each asset that has to be fetched you're forcing the client to open a new connection, which can add more than 150 ms of delay per item (thanks to the three way TCP handshake).

What I would suggest is setting the KeepAlive timeout to a value that could handle each individual page-load. This way all the page elements will have a chance to use the connections that has been already opened.

<comment><USER8> Actually, I've tested this. You get 10ms of extra delay per request, not 150.

There might be some magic value at which KeepAlive will be helpful during non- peak periods without crippling the server during peak periods, but for a well- engineered site, the extra 10ms delay per request shouldn't be a big enough deal to warrant risking a full-on site outage later on.

Also, this has already been discussed _to death_ on HN: [http://www.hnsearch.com/search#request/all&q=keepalive&#...</a>

<comment><USER9> Light travels less than two thousand miles in 10 ms, and TCP requires three one-way trips before starting the first request on a new connection. Anybody more than 620 miles away (about half a time zone) is guaranteed to have a higher ping time than that.

<comment><USER8> You're absolutely right, I didn't think about that. However, I did perform the test(s) from Sacramento, CA to a server in Newark, New Jersey -- a distance of 2,810 miles according to Google.

I was fairly careful with the test(s), and the 10ms difference seemed to be consistent. So that's odd. I need to investigate that further.

</comment></comment></comment></comment></comment></comment><comment><USER10> If anyone owns a blog or site that they suspect _may_ appear on HackerNews (especially if you're posting it), then please take the small amount of time to put an instance of Varnish in front of the site.

Then, ensure that Varnish is actually caching every element of the page, and that you are seeing the cache being hit consistently.

You should expect over 10,000 unique visitors within 24 hours, with most coming in the 30 minutes to 2 hours after you've hit the front page on HN.

You need not do your whole site... but definitely ensure that the key landing page can take the strain.

Unless you've put something like Varnish in front of your web servers, there's a good chance your web server is going down, especially if your pages are quite dynamic and require any processing.

<comment><USER11> Oh, Well then let me share this.

A few weeks ago I got on the front page and within a 24 hour period was hit with 29,000 unique visitors with 38,000 page views. The page itself is image heavy with 1.3 MB on first load. I'm running Wordpress with the Quick Cache plugin by PriMoThemes. I'm hosted on a shared 1and1 server.

I've been hit before and went down, that's when I installed the Quick Cache plugin. Also 1and1 moved me to another server at some point but I'm not sure if it was before or after. Either the cache plugin is really good or I'm on a rockin server all by my lonesome. Or both.

If you're self hosting a wordpress site grab the Hyper Cache plugin or the very very simple Quick Cache plugin by PriMoThemes.

[http://www.tutorial9.net/tutorials/web- tutorials/wordpress-c...](http://www.tutorial9.net/tutorials/web- tutorials/wordpress-caching-whats-the-best-caching-plugin/)

<comment><USER12> Thank you. This is pretty much exactly what I read the post for and was a little disappointed not to find. People talk about getting frontpaged on HN, Slashdot or Reddit, and how you need to be sure you can handle the load, but never give any useful figures on what that load is.

Knowing that I can serve 10 requests per second and likely withstand a frontpage on HN is more useful to me than knowing that I need a way out if and when my web-servers are being totally overwhelmed.

<comment><USER13> On an unexpected trip onto the front page, here's what I saw (hits at ~1 Hz on a Friday evening in the top slot):

<http://measuredmass.wordpress.com/2012/10/12/hacker-news/>

Graphical post-mortem here:

[http://measuredmass.wordpress.com/2012/10/20/more-hn- numbers...](http://measuredmass.wordpress.com/2012/10/20/more-hn-numbers/)

</comment></comment></comment></comment><comment><USER14> I'd argue the opposite of your headline, that this was a very successful launch. Since HN isn't your target audience having your site fail from the traffic was far better than having it fail from a launch in your market. You shook out some important bugs before you lost real users. Plus you got to do this followup which will bring even more traffic.

</comment><comment><USER15> First off, best of luck with your project. Secondly, kudos on writing the post-mortem, as I know it takes some guts to own a "failure".

I think, however, the need to write something like this speaks to an incorrection assumption: you need a "launch". Of course, TC and HN can give you a nice bump in traffic and even signups. However, in the long run, this really doesn't accomplish much for you. It gives you the kind of traffic that will likely leave and move on to the next article, skewing your metrics. There's certainly qualified prospects in there, but it's hard to decipher with all the noise.

Again, the concept of a "launch" speaks to poor business models. It really benefits businesses where the word "traction" is more important than "revenue". Build a business that provides a service that others will pay for and grow as fast as the business can bear, bringing in those visitors that are truly valuable to you.

<comment><USER16> When we did the beta, I posted the launch details on HN and you'll be surprised by the amount of constructive feedback and users that we got. Cucumbertown now has users from devs to CEO's who came in through HN and are now engaged users.

Cucumbertown has some notions like 'forking recipes' – called "Write a variation" which enables you to take a recipe and fork and make changes. Additionally Cucumbertown has a short hand notation way to write recipes(think stenography for recipes) – for advanced users. Things like these appeal to the HN crowd a lot.

Also, don't you think quite a few hackers like me are also cooks!

<comment><USER17> Wow, forking and shorthand are great features! I had no idea from your homepage. Maybe I'm not your target audience but you should make that clearer the moment someone lands on your homepage. ("Why we're different and maybe better than AllRecipes or XYZ: ...")

<comment><USER16> Thanks for the feedback.

Cucumbertown is very UX focused and from our research we came to a conclusion that our "aha" moment is to get you to the fastest possible way to write a recipe and give you that bout of joy. Now being hackers, we'd want to see forking & stenography in front of us. But that's been a struggle we've been trying to showcase between simplicity to the "aha" moment and differentiation with others. Our primary audience comes to Cucumbertown because they are frustrated writing recipes in dropbox, google docs, wordpress & tumblr blogs as "blobs of text".

</comment></comment></comment></comment><comment><USER18> Thanks for this post, there were some nice tips in there. Although, I do have some nitpicking about your writing style. Maybe it's just me, but I found that your use of "+ve" instead of just saying "positive" and of "&" instead of "and" did not have the intended effect of speeding up reading, quite the reverse actually.

<comment><USER19> Seconded. Initially, my brain told me that +ve was the name of the site, so I was confused when I looked for a product page link only saw "Cucumbertown".

Granted, that's mostly laziness -- apparently I've got a rule that matches "strange words near the top of the post" to "probably the name of the product".

<comment><USER20> I dont know about anyone else, but when I saw "+ve", I just thought to myself, "what is that?" for about a half a second before giving up and moving on.

<comment><USER21> It doesn't even parse for me. If "+ve" means "Positive", what does "+" mean? Positi?

Regardless of what you do, a little bit of respect for English is always a good thing to have.

</comment></comment></comment></comment><comment><USER22> Running with swap enabled is a terrible idea. The authors mention how it was only once solr crashed that they were able to actually log in and start fixing problems; having swap means that rather than the OOM killer terminating processes, instead your whole system just grinds to a halt.

(it's strange that they recommend enabling swap when they also recommend enabling reboot-on-oom, which is pretty much the complete opposite philosophy)

<comment><USER23> In my experience, the linux kernel handles no swap at all very badly, so you need a small amount.

Increasing the swap, which is the suggested solution, is however, a terrible idea. As soon as you hit high memory usage, your IO load will go through the roof, and everything will grind to a halt.

The solution here is separation of services - i.e. put Solr on a different box, so that if it spirals it doesn't take out other services.

The OOM killer is your friend for recovering from horrible conditions, but as soon as you hit it or swap, somethings gone wrong.

<comment><USER16> You are right. The best solution is separation of services. But for a startup than runs 7-2 services like this – it's a close call. You'll often have to run 2-3 services together, else $100 * 7 machines is too much burn

<comment><USER3> It's not a problem running several services on the same box as soon as each of them is sized appropriately. What I suggest at least roughly calculate how much eg. RAM could each service use at peak time. This usage should be limited so that the sum of memory used by all services at peak time is less than the amount of RAM you've got on your server.

</comment></comment></comment></comment><comment><USER24> 1\. So nginx didn't cache because of cookie?

2\. Isn't swapping bad? I don't think I've ever had a situation in which swap more than say 100MB was helpful. Once the machine starts swapping, a bigger swap just prolongs the agony.

3\. If you couldn't ssh, why didn't you just reboot the machine?

Edit:

1\. What did you use for the graphs?

2\. What is the stack?

<comment><USER16> We use DataDog - <http://www.datadoghq.com/> . It's a statsd, graphite manifestation but with much more capabilities.

Stack is Python, Django, PostgreSql, Redis, Memcache etc.

</comment></comment><comment><USER25> Stress test, load test before launch!

It doesn take more then an hour, and you quickly know what your upper limits are, and where the bottlenecks are.

I use gatling in favor of JMeter: <https://github.com/excilys/gatling>

</comment><comment><USER26> I find it very difficult to believe that this person worked on any sort of performance team, given that what they discovered is pretty much "Handling Load 101".

Running everything on one box? Using swap? No caching? It's like a laundry list of junior admin mistakes.

<comment><USER27> Generally that's kind of how these smaller 'startups' work... and then I get a call and charge my standard rate per hour ;)

</comment></comment><comment><USER28> This post mortem has me thinking about the best way to handle the situation in which you can't SSH into your server. The OP decided to trigger a kernel panic/restart on OOM errors, but I have a couple of concerns about this approach:

* If memory serves correctly, if your system runs out of memory, shouldn't the scheduler kill processes that are using too much memory? If this is the case, the system should recover from the OOM error and no restart should be needed.

* OOM errors aren't the only way to get a system into a state where you cannot SSH into a system. It would be great to have a more general solution.

* Even if you do restart, unless you had some kind of performance monitoring enabled, the system is no longer in the high-memory state so it will take a bit of digging to determine the root cause. If OOM errors are logged to syslog or something, I guess this isn't a big deal.

I suppose the best fail-safe solution is to ensure you always have one of the following:

* physical access to the system

* a way to access the console indirectly (something like VSphere comes to mind)

* Services like linode allow you to restart your system remotely, which would have been useful in this scenario

<comment><USER29> * In linux-land, there's an OOM killer (<http://linux-mm.org/OOM_Killer>) that would have started taking processes out. You have to exhaust swap for it to really take effect, and once you hit swap, your entire machine suddenly becomes hugely IO bound - in shared or virtual hosting environments, this usually makes the machine totally unresponsive.

* I've never seen any sort of virtual hosting service without either a remote console or a remote reboot. Usually both.

</comment></comment><comment><USER30> I clicked on the link to Cucumbertown and was immediately greeted with a picture of Italian seasoned chicken thighs.

I think I really like your website. I really like the simplicity of the presentation to the user.

</comment><comment><USER31> This is a great way to make lemonade out of the lemon of getting hosed by a lot of traffic. Write an informative post-mortem and resubmit! I know I missed the original submission and clicked through to the site, and there you have it. I'd say being humble and trying again is never a bad idea.

</comment><comment><USER32> Whatever was the _real_ cause for your issues, Linode's default small swap space is a plague. A system starts to misbehave much gently if there is enough swap.

<comment><USER3> For a production server I think that the opposite is a better _general_ advice - reduce the available swap, because if your server gets to a point to need it, the performance will suffer so much that your server will become completely unresponsive. Having less swap will allow the OOM to kill the run- away process and allow you to login and fix the problem instead of rebooting the server or waiting in vain for it to recover by itself.

 _edit: typo_

<comment> X-Istence Drop your swap on a separate drive from your main drive that is serving your data. Solves the problem nicely.

</comment></comment></comment><comment><USER33> I am currently building a site and this is definitely an experience that I can learn from. I am wondering, why was the homepage not being cached?

<comment><USER3> There was a cookie set for CSRF protection and the headers specify that content should not be cached if there is a cookie (or more precisely - <USER2> cached content includes the cookie as a cache key, so each request with a different cookie gets a cache-miss).

<comment><USER16> Slightly different.

"Note that in 0.8.44 behaviour was changed to something considered more natural. As of 0.8.44 nginx no longer caches responses with Set-Cookie header and doesn't strip this header with cache turned on (unless you instruct it to do so with proxy_hide_header and proxy_ignore_headers). "

<http://forum.nginx.org/read.php?2,126312,126316#msg-126316>

<comment><USER3> Thanks for the clarification.

</comment></comment></comment></comment><comment><USER34> [ not that you asked for it here, but I've got some frontpage UI feedback: ]

I think you should put a description up front to describe what Cucumber town is. I think that main image should be a slider with multiple feature images, and I think the Latest Recipes should be the first section after this. Just my 2c!

Screen: <http://cl.ly/image/3R2Y131Z433L>

<comment><USER16> This is wonderful feedback. We'll definetly look into this.

</comment></comment><comment><USER35> I have been having some of the same issues on a site I run ( <http://www.opentestsearch.com/> ). Under heavy load solr will grind to a halt if you don't have enough ram available.

Putting a dedicated Varnish server in front of the search servers helped a lot. Using a cdn may also be a viable option, but haven't tried it myself.

<comment><USER27> Well, and this is why I recommend running solr on a standalone instance. It (and java/jetty/tomcat/etc.) are very memory hungry in general, so it is worth your while and money to spend a bit more and spin up a separate instance or whatever type of services you are using to run solr. It'll also run faster.

One last thing you can do if none of that is possible is use a better VM like Jrockit ([http://www.oracle.com/technetwork/middleware/jrockit/overvie...](http://www.oracle.com/technetwork/middleware/jrockit/overview/index.html)). Jrockit with the right GC in my experience is much better about running in lower memory type situations.

</comment></comment><comment><USER36> That's why I like to use Heroku/EC-2 for launching new webservice. If shits hit the fan, you can jack up the processing power/database/RAM/whatever to scale to your demand. Once you have a good idea of the traffic it generates, you can then move it to a cheaper service.

Obviously, it's easy to say that when you're on the bench. Congratulations on the launch by the way.

<comment><USER16> Cucumbertown co-founder here. Actually I dislike this idea though we should have been better prepared.

At my previous firm we had this culture that whenever traffic peaks we spin up new instances. And tools like RightScale & Chef make it ridiculously simple. So our style was to do that than to optimize strains in code paths. Because this is so so convenient.

And before you know it, this notion of __hardware is cheap __becomes a culture. Soon enough if you grow you'll be serving 100K users with 250 machines.

<comment><USER36> I understand what you are saying.

I do agree that Chef [and RightScale? Never used it) makes it easy to spawn new instances and through load balancing average your load.

I was talking in term of tradeoff in the first few weeks of a new service with a MVP. Obviously, you re-assess your need before you get to 100k user, and probably uses something else than EC-2.

In any case, both ideas are equally good. I don't claim better knowledge in any way.

</comment></comment></comment><comment><USER37> Wow, nice post with real data, graphs, and helpful tips. As the Germans would say, I have nothing to complain about.

</comment><comment><USER38> Once memory goes to swap you already lost. Personally I rarely configure swap on servers, save the DB. I would reconfigure your services to not grow past physical free memory. After that you are going to have to scale servers horizontally.

</comment><comment><USER39> lamesauce.

1\. HN should let you pay them $10 and let them hammer your server(s) before your story goes live. good for you. good for them.

2\. there's a deal at lowendbox right now for a 2GB VPS for $30 a YEAR. you could have a healthy server farm for pretty cheap.

</comment><comment><USER40> This is really interesting, thanks for sharing. I love this kind of transparency.

</comment><comment><USER20> Is there a way to run simulated traffic to determine how your server will react based upon heavier load to try and determine how many people it can serve?

<comment><USER41> Blitz is also worth a mention.

<http://www.blitz.io/>

<comment><USER42> Can also connect newrelic with it and get stack traces logging how much time each component of your app is taking including DB queries.

</comment></comment></comment><comment><USER43> Please, fix your CSS for mobile use. It's impossible to read because, if I zoom, the sidebar gets bigger too

</comment><comment><USER44> well, you got it right this time :)

</comment><comment><USER45> Your project, Cucumbertown, is a cooking/recipe site/platform/network. Hacker News is not your audience/customer. Any "launch" on Hacker News is a fail, regardless of downtime.

<comment><USER46> Couldn't disagree more. HN community is the most (brutally) honest and it's feedback can prove very valuable - regardless whether HNers are the target market or not.</comment></comment>


How Vblock Saved Pivotal Developers Time, Decreased Frustration Levels - <AUTHOR> http://blog.pivotal.io/cloud-foundry-pivotal/case-studies-2/how-vblock-saved-pivotal-developers-time-decreased-frustration-levels

<comment><USER2> A lot of the folks at VCE (including me!) were really pleased to see this post. I've been thinking about Vblock Systems since 2010 as a way to do the dev / test / qa / qc / prod / scale /etc... pattern in the _same_ investment for initial spend and ongoing operations.

There are thousands of Vblock enabled teams out there in the world now so this is a great data point for those that have not yet considered the non prod options on the same environment.</comment>


Show HN: Best-Man Speech Danger Checker - <AUTHOR> https://newlywed.science/speech-checker.html <comment><USER2> You can add some examples for different cases it was able to detect. This would help us better analyze the website.</comment>


Using React 16 to open new window with shared state - <AUTHOR> https://hackernoon.com/using-a-react-16-portal-to-do-something-cool-2a2d627b0202 <comment><USER2> This is a bad idea for any serious app. I inherited a framework that did this, and it performed terribly. Not to mention, it's an exercise in discovering browser edge cases you would have otherwise never had to worry about. Use postMessage or localStorage to communicate between windows. Don't use window handles.

<comment><USER3> I will block react with ublock

<comment><USER4> Ok

</comment></comment></comment><comment><USER5> It's probably just me, but I don't get what is cool about rendering in another window. Also, I generally don't like it if an app uses more than one window, so I'm not sure what this is useful for.

<comment><USER6> Can be useful when you have multiple monitors and don't want to clutter the main window, especially for single page apps or visualization/monitoring things.

I have done my own take at this since some times but vanilla only [https://github.com/grz0zrg/wui](https://github.com/grz0zrg/wui) where any of the dialog widget can be detached off the main window keeping the same content & state. A feature i never saw in any web apps but saw often on desktop apps, nice to see that some peoples think of this sort of thing with cleaner approach!

<comment><USER7> > Can be useful when you have multiple monitors and don't want to clutter the > main window, especially for single page apps or visualization/monitoring > things.

Yep, this is precisely the use-case I would have for this in loom-viewer[0], a SPA data viewer I'm developing for the Loom file format[1] (think large genomic data matrices + metadata). It has a lot of scatter plots and bar charts, and for people with multiple monitors using a separate window as a pop-out would be quite beneficial. It also would make it easier to _print_ out the graphs (which I find silly, but it's a thing researchers do).

I've looked at golden-layout before, but it doesn't play nice with redux at the moment[1]. Since I'm already on React 16 this might be what I'm looking for.

[0] [http://loom.linnarssonlab.org/](http://loom.linnarssonlab.org/)

[1] [http://loompy.org/](http://loompy.org/)

[2] [http://golden-layout.com/](http://golden-layout.com/)

<comment><USER8> Hi, I'm a Redux maintainer. Could you clarify what you mean by "doesn't play nice with Redux"?

<comment><USER7> Hi!

Well, note that I say that golden-layout does not play nice with Redux, not the other way around. It's not Redux being problematic here, as far as I can tell.

There's an open issue on their github repo asking for examples of how to integrate it:

[https://github.com/deepstreamIO/golden- layout/issues/280](https://github.com/deepstreamIO/golden-layout/issues/280)

Maybe things have improved since I've last looked at it, but when I tried a few months ago, it seemed like it would require quite a bit of change in my code to get it up and running

By comparison, this new React 16 solution looks like it's a lot simpler to integrate in my existing code.

</comment></comment></comment></comment></comment><comment><USER9> It's a really cool read. I thought it was neat and a bit more elegant than doing something similar with Window.postMessage ([https://developer.mozilla.org/en- US/docs/Web/API/Window/post...](https://developer.mozilla.org/en- US/docs/Web/API/Window/postMessage)).

</comment><comment><USER10> This was always possible in React. I used it to render the redux-devtools in its own window (before there was a browser plugin): [https://gist.github.com/tlrobinson/1e63d15d3e5f33410ef7](https://gist.github.com/tlrobinson/1e63d15d3e5f33410ef7)

<comment><USER11> That's establishing a separate component tree. This is one single component tree, hosted in the parent window. That was never possible until portals landed.

<comment><USER12> You could have it like this:

        <Parent>
          <MyPortal root={element}>

Howdy!

          </MyPortal>
        </Parent>

Where MyPortal actually renders its children in another element. Single component tree, multiple DOM roots. What portals add is the propagation of events through the portal.

<comment><USER12> I'm out of my edit window, so here's an example of what I mean:

[https://flems.io/#0=N4IgZglgNgpgziAXAbVAOwIYFsZJAOgAsAXLKEAG...](https://flems.io/#0=N4IgZglgNgpgziAXAbVAOwIYFsZJAOgAsAXLKEAGhAGMB7NYmBvAHgBMIA3AAgjYF4sGCGgB8LAPQdOogDpp2XXgIAOtAE7EMUcVK5y08kAF8K6bLkQEAVgip0GTYnjABXNNWIR63AAoatKAgALxgAChV1WhU4AEpuYHlubgAlGAxPABEAeQBZfHUmNhh1CKiY- GpCaDZCtApuSOi4AtpaYlik7kLiV3U0bjRXKCh5Y3l5Nw8vHwBBVwAPbjLm+MSB7phe- sbylqqauoBuMZPDD3o4Ym5Cbn5U9M9KwoxGAFFYHAYJtCnPbwGKTaxDCay6PT6A0IYXmCwaQxGFC6yWhAHIVKj4cMoA1USJuEIRKjYkiNslkTcwv5NNoQjAGsAou1EI0AtpTBSUWF0ZjBtjcfi1DSoMTSclkiSup00ONzmkMsQcvk6sVStCge0SQThGhpSYzCBMDg8JU4HYaPRGMwrNJEBgwIx1AkugAjDIAawA5lF3GwALR0KAaFkAd2qjDOyRdGlVLIATColmx2ow2Js2JHuNH1Kq- eoMBxXHAWQBGAAMibOXQcVuILNexFKfFiVY2YCDrxZ6ggnpImbAlr9IZgPZILOjUAzXRUBY4aE9LLL3HLlbGP2kzo22dj3AAzInuHBaEE04Up1uYyU8wWIEXSxWFpnqLAMOpx+1CJmhOpPSJ76uNhnNg5wXZcHzOWUTAAXSoII0HdBAUHMY0rBeJ5ik4GAgxUL5iHwWxKBAPpyCsEhiBiRAJAkdwVC9SpaCwCQ0OIAABEsADZ8BLfAy2orA2CYx48IwrDolw- DzWIABPFRLBAAjTGQuTmL9ZMsHwETsPEgiqGIvAyIoqiaLouhGJUtS2M47jeNcfjBIVVSGI0mBMK0pwJMI6TZLwBSoOMIA)

</comment></comment></comment></comment><comment><USER13> This interacts poorly with the built-in script blocking in chrome. You can't enable JavaScript in the about:blank window, so the buttons in it don't do anything. This is a pretty serious issue.

Probably more of a chrome bug though I guess, it should inherit script blocking from the spawning page.

</comment><comment><USER14> Ah this is cool, like ember-wormhole in the ember world!

How were React devs doing modals, etc before?

<comment><USER15> Just use ReactDOM.render into a separate, top-level
 that is usually empty. And then have the close button of the modal call ReactDOM.unmountComponentAtNode.

Alternatively, don't bother with all of this and just mount the modal into your children. Use CSS `position: absolute/fixed` to position it. Of course this assumes your ancestor elements don't have certain properties like `position: relative` or `transform` set, but this is usually not a problem.

<comment><USER16> A pop-up can be dragged onto a second monitor, for example, whereas a modal cannot.

<comment><USER15> OP was asking about how to do modals, not comparing a modal vs a pop-up.

</comment></comment></comment></comment><comment><USER17> Another reason you might want to use this technique is to render something into an iframe

</comment><comment><USER18> Could this be exploited for state rehydration across multiple tabs?

</comment><comment><USER19> How are portals implemented? Are they using broadcastchannel API or postmessage, etc, to communicate between the windows in the same app?

</comment><comment><USER20> React 16?

Anglar 1 was around for a while. There was a big fuss about Angular 2 being incompatible, and before I have had a chance to look at it, we are on Angular 4.

Now we are on React 16.

As if the JavaScript ecosystem wasn't bad enough, we now seem have an extra problem of completely silly release numbers. (I do 80% back end work, but in this day and age its hard not to use JavaScript. I took an interest in these frameworks a couple of years back and didn't gt much chance to follow up since then. Version numbering like this discourage me from finding out more.)

<comment><USER21> honestly just caused me to realize I'm not even positive what version my last react app was in.. I thought I was 'current' but I know it was a single digit number so I guess I wasn't.

<comment><USER20> If it's backward compatible, its not so bad, but whole number releases usually imply that there will be some breaking changes (or they do to me at least).</comment></comment></comment>


Uber on Slack - <AUTHOR> http://uberonslack.com/ <comment><AUTHOR> This came out of a hackathon[1] and offers a bunch of useful commands:

/uber ride [origin] to [destination] -- Request a ride from [origin] to [destination].

/uber estimate [origin] to [destination] -- Get price and time estimates to drive from [origin] to [destination].

/uber status -- See the status of the last ride you requested on Slack.

/uber share -- Get a link to share your ride's progress.

/uber cancel -- Cancel the last ride you requested on Slack.

/uber help -- Get a list of available commands.

[1] [https://devblog.uber.com/uber-slack-a-weekend-a-story-of- ope...](https://devblog.uber.com/uber-slack-a-weekend-a-story-of-open-apis/)</comment>


Roasting coffee beans with Python, a bread maker, and a heat gun - <AUTHOR> http://coffeesnobs.com.au/YaBB.pl?num=1259627839

<comment><USER2> Andrew Tridgell, who built this coffee roaster a.k.a. chocolate truffle melter, gave us Samba and Rsync.

</comment><comment><USER3> Ok now that it's hooked up to a computer, how about using a genetic algorithm to find the perfect roasting settings? (As a bonus it could evolve over time to match your gradually changing tastes.)

<comment><USER4> As a home coffee roaster myself (including building my own roaster) I'm guessing that experimenting programatically with roast times would cause the smoke alarm to go off regularly.

I guess you could hope that later generations get better at this, but there's always the risk of a 5 alarm mutation ;-)

<comment><USER3> Hook up a smoke detector to turn off the roaster. If smoke doesn't clear in 1 minute, have it shoot off a fire extinguisher. Problem solved.

</comment></comment></comment><comment><USER5> RubyRoast tastes better.</comment>


Overfitting, Regularization, and Hyperparameter Optimization - <AUTHOR> http://dswalter.github.io/blog/overfitting-regularization-hyperparameters/ <comment><USER2> This is a well-written article, and the concepts are explained clearly, thanks for sharing. I'd just like to add a caveat.

When the author says "if a researcher runs 400 experiments on the same train- test splits" \- then depending on what he means by 'test' set, that researcher is _wrong_. In pretty much all machine learning literature I've come across, it's drilled into you that you _never look at your held-out test set_ until the very end. Hyperparameter optimisation and/or model selection happens on the training set and only when you've tuned your hyperparameters and selected your best model do you run the model on your test set to see how it's done.

Once you've run the model on the test set once, you can't go back to tweak your model because you're introducing bias and you no longer have any data left that your model's never seen before.

To avoid overfitting, you can use cross-validation to effectively re-use your training set and create multiple training/validation splits. (As an aside, I find it frustrating how liberally different sources switch between 'validation set' and 'test set', it's really confusing).

<comment><AUTHOR> OP here: I probably should have made it clearer that the using the same train- test splits is verboten.

Your second and third paragraphs are also exactly correct. I attempted to make those points in the post, but you've done so more effectively here.

<comment><USER2> Yeah I just thought it was worth hammering it home, especially because of some literature's use of "test set" to mean "validation set" which can really throw off a beginner like myself.

</comment></comment></comment><comment><USER3> I spent a few months recently on a tough ML project, and pretty much got my ass kicked the entire time. It seemed like every hurdle that I overcome was met with another as I turned up the "voltage". I came to regard every decision that I made (this includes all kinds of hyperparameters, but also design decisions) with extreme suspicion. I don't really think there is a convincing way around this: any kind of optimization has a context outside of which the optimization no longer makes sense. And so one tries to include this context, but this turns into a meta-optimization with a meta-context assumed, and so-on in infinite regress. I guess I am agreeing with the author: "if two algorithms achieve the same performance on a task, the one with less hyperparameter optimization is generally preferable."

It really seems like there should be more of a theory around these issues. Even a dreadfully abstract and/or terse VC-dimension-scariness level of a theory.

I keenly await the author's next post.

<comment><USER4> There are some formal methods and rules of thumb for avoiding trouble in this regard. Take a look at bayesian and entropy-based maximization strategies. Hyperparameter optimization can be looked at using either strategy. Also take a look at [https://en.wikipedia.org/wiki/Minimum_description_length](https://en.wikipedia.org/wiki/Minimum_description_length)

There's definitely a lot of theory, but it hasn't yet been turned into readily usable R libraries.

</comment></comment><comment><USER5> I think you really get into this sort of mess when you want to squeeze the last ounce of predictive performance out of an algorithm (or ensemble of algorithms). When you just want performance that is better than a plain old regression, I've found that just picking sane defaults for some hyperparameters (e.g. RBF kernel for SVM) and doing a small grid search for others (e.g. slack parameter for SVM, cost-complexity for trees) works very well.

In Python this is easy enough with scikit-learn, and in R the caret package makes semi-automatic tuning really easy.

So the question becomes: how far do we really need to go to create business value, and does it actually make sense to go all Kaggle on the problem?

<comment><USER2> I'm only a data science student at the moment so I don't have much "real world" experience with machine learning, but this is what I would have thought. If you're trying to get quick value from a dataset, you can probably run a "vanilla" random forest or something and get pretty good results. Then, if you want to use it in production somehow, you can go back and "go all Kaggle" (I like the expression!) on it.

</comment></comment><comment><USER6> This was a really helpful article, thank you for posting it. I've been cognizant of these issues for some time but I hadn't seen any articles encapsulating them so cleanly. Thank you.

</comment><comment><USER7> Basic question(s) as I am not a data scientist but have just taken a machine learning course ([https://www.coursera.org/learn/machine- learning/](https://www.coursera.org/learn/machine-learning/) )

Won't trying different combinations of hyper parameters/lambda (over a small range) help us arrive better instead of manually tuning it? Or is that what the author meant by manual tuning?

<comment><USER8> I'm not a data scientist per se, but I've been working with some (boss and co- worker) to get some stuff operationalized and into production, so I've been responsible for generating inputs, helping analyze/visualize outputs, and building linear optimization models, so I've got some very basic experience.

As I understand it, one of the pitfalls of automatic tuning is that it becomes hard to account for seasonality and you will likely end up with useless parameters - for instance a customer ID is rarely a good parameter to optimize on, even as a categorical variable, except in very specific cases. It is probably a proxy variable for one or more other ones that you need to tease out of the rest of the data.

(warning, potentially me talking nonsense coming up) Automatic tuning is no substitute for a talented analyst who knows the data well and understands the goal. But if you've got hundreds to millions of parameters, you may not have another choice really.</comment></comment>


Why Are Marriage Rates Down? Study Blames Lack of 'Economically-Attractive' Men - <AUTHOR> https://www.studyfinds.org/why-are-marriage-rates-down-study-blames-lack-of-economically-attractive-men/ <comment><USER2> From the Wiley press release [1], not the blogspam:

> "Most American women hope to marry but current shortages of marriageable > men—men with a stable job and a good income—make this increasingly > difficult, especially in the current gig economy of unstable low-paying > service jobs," said lead author Daniel T. Lichter, PhD, of Cornell > University. "Marriage is still based on love, but it also is fundamentally > an economic transaction. Many young men today have little to bring to the > marriage bargain, especially as young women's educational levels on average > now exceed their male suitors."

I don't know, that seems like such an incredibly 1950s take to me. Is that really the way most people view marriage? Sure, people who are seeing success in one area of their lives (educational, professional) are more likely to see success in other areas, but I don't think most people (or most women) view marriage as a "fundamentally economic transaction". That reads kind of like incel flame-bait to be honest.

[1] [https://newsroom.wiley.com/press-release/journal-marriage- an...](https://newsroom.wiley.com/press-release/journal-marriage-and- family/do-unmarried-women-face-shortages-partners-us-marriage-mar)

<comment><USER3> > view marriage as a "fundamentally economic transaction".

You dropped the critical word 'also'. Marriage is 'also a fundamentally economic transaction'. At some level, you should understand what your future looks like with this person, how much will you make, will that make both of you happy, how will you handle a divorce, etc. What happens if one of you gets sick and the other has to care for you?

</comment></comment><comment><USER4> This echoes what's been going on in Japan since the 90's.

What we hear about men in China needing to own an apartment in order to be a well qualified potential partner seems to be an amped up version of this.

And most importantly

"These synthetic husbands were also 30% more likely to be employed than real single men and 19% more likely to have a college degree"

I really enjoyed the nomenclature of "synthetic husbands"

<comment><USER5> Are marriage rates up in any Western country?

</comment></comment><comment><USER6> Poor people have existed in every time and era, yet we don't hear of low marriage rates in the 1800s, for example. It can't be just about the money.

I personally think it's good that marriage isn't the only choice for women, or men for that matter. People like their independence and without the social pressure to marry just for the sake it, it ultimately results in happier people that don't have to do things just because of some vague, background expectation of society - which is ultimately composed of more people that don't care about you than do.

Unless you are poor, but if you are poor you are likely unhappy whether you are married or not.

Could universal basic income save marriage?

<comment><USER7> > Poor people have existed in every time and era, yet we don't hear of low > marriage rates in the 1800s, for example. It can't be just about the money.

It is now socially acceptable, and financially feasible, for women to avoid marrying. That was certainly not the case in the 1800s. It was barely the case in the 1950s.

</comment></comment><comment><USER8> I feel like an increasing rate/normalization of hookups and arrangements (sugar) makes it perfectly possible for men to get the transactional/on demand aspects they generally seek and ladies get the economic aspect they often seek.

I (still happily) married right before Tinder came around. I really think if I had access before being in LTR with my wife I'd go this route and not have married. Hookups and companionship when I want. None of the stress, pressure, expectations, and drama of being a husband.

<comment><USER9> This misses the actual upsides of marriage, you don't get married for the easy casual sex (well I mean some people probably do at first). You get married for the commitment. You can rely on the other, till death do you part, to support you, pooling your resources etc. and your families an social network sees the commitment as a sign of maturity and a signal to invest in your future as well.

To look at marriage as purely transactional either sexually or financially is disingenuous.

<comment><USER8> I'm not really imposing this on anyone else just voicing my thoughts on how I may choose differently given second chance. I'm happily married so know the pros and cons. Fact of it is I was never gunning for marriage. I dated my wife for a very long time and she eventually gave me an ultimatum. I didn't want to leave her so I married her. That said I think it's the best decision I've ever made. However, it's challenging because I'm also selfish and want to focus on myself and my interests way more than couple stuff. Fact is I don't need any of that stuff you listed. I make plenty of money, I don't give a shit what my family or friends think, I really sometimes wish I had the resources I have now without the responsibilities that come with wife, kid, family. It's probably the definition of mid life crisis.

I'm thankful to be happily married. I know a lot of people in similar situations but not happy and refuse to put their kid through a broken home or they don't want to lose half or the social/family shame part. It seems miserable.

</comment></comment></comment><comment><USER10> I've often joked that getting married is like saying "I love you so much, let's get the government involved".

I feel like so much of the draw for many people to get married is the pageantry and narcissism involved.

As a divorced man who is at the age where all his friends seem to be getting married I can't help but make mental wagers on how many of those marriages will actually last.

</comment><comment><USER11> The pay gap says that men make more money, but this study would have me believe that they don't make _enough_ "more money" to settle down with? Someone please correct my wrongthink.

<comment><USER12> The "pay gap" is about people who have jobs. With some more or less defensible assumptions about what jobs are comparable.

But this study is about employment, income and academic characteristics of available vs desired men. And it too makes some assumptions. One, that women looking for husbands won't consider other women. And that they exclude men of other races. Both are obviously false, but arguably not false enough to render the study meaningless.

<comment><USER11> Unemployment for men is under 4% so I'm not sure how that separates the issue of the pay gap from the men in this study. Put another way, >96% of men have income from employment. And according to the study, desired income vs real income was the biggest difference according to the women. Clearly, >96% of men are making far less than desired according to the women surveyed.

<comment><USER12> > Unemployment for men is under 4% ...

Official unemployment data aren't reliable. Mainly because they ignore people who aren't actively looking for work. Typically because they've given up, or won't accept what's on offer.

Women do have high standards, according to this study. Somewhat implausibly high, I admit.

<comment><USER13> I think peeps are forgetting the long term impact to one's career from bouts of long term unemployment. How many bounced right back from 2008? Combine with student debt, rising home prices, and the fact you're not getting any younger, then in 10 years you're not 28, but 38. You have a job, but you're making less at 40 then you did at 30. And you're not supposed to 'trust' anyone over 30. (>_<)</comment></comment></comment></comment></comment>


A Live Cricket Steers Mattel's New Autonomous Toy Car - <AUTHOR> http://www.gizmodo.com.au/2015/12/a-live-cricket-steers-mattels-new-autonomous-toy-car/ <comment><AUTHOR> At least with this self driving car we know where the bugs are.

<comment><USER2> Did you submit this article solely to make this comment? If so I'm fully behind you

</comment></comment><comment><USER3> PETA's going to hate this... and for once I'll agree with them. Sending kids down the path of capturing and imprisoning creatures to use for entertainment doesn't strike me as a good thing to do.

<comment><USER4> Let me introduce a different point of view.

A toddler will try to grasp any small and 'cute' animal at sight. Is part of their human inquisitive nature. Children do not keep animals because all of they are little sadist jailers; they normally do this because are curious, want to understand any novelty around and have the basic impulse to protect, make lots of 'pet friends' or even experiment playing parent roles (with their 'subrogated sons').

And this is good, not bad. Even if some random insect or snail die eventually. This is totally the opposite of a sadistic behaviour.

Children feel the normal urge to explore, control and manipulate their environment. A child that do not show any interest for the real world is either autistic, sick, or have a serious problem that needs to be addressed (like being abused for example). Therefore, parents have to teach them about the typical outdoor dangers. This includes what animals to avoid at any cost, and how to manage the other animals that could deliver a bite leading to a sored thumb, like crabs or crickets.

To make a children responsible of the life of a pet and to teach him/her to adopt a routine (of feeding, changing water and caring of their beloved pet) is an invaluable experience that any boy and girl in the planet should have once at least. Humans and errors are linked concepts. I think that is much less traumatic for a boy to kill a short-lived cricket or grasshopper because they forgot to refill a bowl with water the last weekend, than to kill a canary or a cat by a similar mistake.

I think also that children need to understand that sometimes illness and death happen. To avoid a children any (reasonable, controlled, human) exposure to death will probably lead to a shelfish or stunned adult that will not touch their ill old parents with a ten foot pole. Everybody will die and our sons need be armed psychologically against this moment of mourn and lose, and have the opportunity to learn from family and adult friends what to do about this.

A pokemon or virtual pet can not provide the same experience. Life is not like a video-game that can be reiniciated as many times as you want, and children need to understand that when a pet die will not return to life again pressing some buttons, that is lost, and that to feel bad about this is normal.

<comment><USER4> And now lets talk about the toy. I thing that is ugly, smart and reasonably well designed.

In some countries to keep crickets as pets is not a such bizarre idea. When I was young I shared a few black shiny and big-headed cricket pets with my brothers. After hoping for hours in shorts, up and down the meadow, we returned to home and show our treasures: a wall lizard nest that we keep warm and wet until eclosion, three yellow and white butterflies, several grasshoppers and crickets... Males and females are easy to identify. Two males can not be put in the same cage and want to live isolated most of the time. We hunt and keep them for a while before to be released again in the meadow. Boys nowadays just keep excited pressing buttons all the evening instead, a really sad way to spend a glorious summer evening in my opinion. We liked to hear it singing in the balcony at night, is a relaxing sound. You could go to a news stand then and buy small plastic cages specially for housing crickets. (Cardboard boxes are just not good enough, because crickets will chew a hole in a coin and escape).

I can't help to point that this toy is a luxury palace for a cricket by comparison with our small cages. Well shaded with generous inner space and imitating a burrow. For most children living in a big city in a crowded apartment those are ideal pets for newies, more convenient and less demanding than a bored cat or dog.

</comment></comment></comment><comment><USER5> The two comments posted (so far) on Gizmodo nail it for me:

> This seems sort of sadistic.

> > Yeah, just another case of man using world's creatures for his amusement. > It's a bit sick really.

<comment><USER6> Is it much worse than an ant farm? I think it is probably a bit worse, since at least with an ant farm ants are doing ant-related things, even if it's behind plastic.

<comment><USER7> Don't ants have food in their ant farm?

<comment><USER8> They mentioned there being a "feeding chamber" on this toy as well.

</comment></comment></comment></comment><comment><USER9> Seems like a really odd target market: kids who don't think bugs are gross, have moms who'll allow a box of live crickets in the house, have easy access to a source of live crickets, have parents who aren't totally weirded out by the idea of feeding a constant stream of crickets into this thing for a child's mild amusement as it rams into a table leg over and over again.

<comment><USER10> Most likely they'd be playing with it outside (if kids still do that nowadays).

<comment><USER9> Yeah but they're probably going to store their little collection of cricket pilots in a box inside the house... and if I know kids, they'll plop down at the kitchen table to load up a cricket before heading outside to play.

<comment><USER10> Perhaps. I used to collect bugs in a jar (then release them later) on my parents' farm, but never brought them inside. However, can't speak for all children though.

My brother on the other hand, I could see him doing something like what you described when he was a kid though.

</comment></comment></comment></comment><comment><USER11> That is obviously a joke. Mattel would never produce such a crappy ad.

</comment><comment><USER12> This seems fucking wrong.</comment>


Sneak Peek: Visualization on Rescale - <AUTHOR> http://blog.rescale.com/sneak-peek-visualization-on-rescale/

<comment><USER2> VNC over SSH is by no means an optimized solution. Depending on the client ISP, remote desktop is very usable. I tested this on a relatively slow 2Mbit up, 12Mbit down DSL connection, and basic "monitoring" and post-processing tasks are very usable.

Applications which require higher frame rates are, on the other hand, unusable with VNC on a connection like this.

I have tested a RDP connection on the same network which performs marginally better to VNC over SSH.

We're currently exploring integrating solutions/technologies like DCV to improve the 3D/frame rate performance of remote desktop.

There are in general 2 performance areas we're looking at: interaction responsiveness and frame rate/rendering performance. Interaction responsiveness is not an issue at this point. The frame rate and rendering performance is something we can and will definitely improve on by integrating different technologies.

</comment><comment><USER3> Great write-up, and as always, really impressive. Handling GPU-intensive visualizations remotely can be really tough. I've dealt with similar in the EDA space. I realize this is new and in still in beta, but do currently have any performance characteristics? Is the application latency/responsiveness tolerable?</comment>


The 20£ limit with NFC Cards bypassed by researchers - <AUTHOR> http://gizmodo.com/report-a-flaw-in-visas-contactless-card-lets-anyone-ch-1653974432

<comment><USER2> £20* but that's quite a flaw. good job visa said it's safe</comment>


Watson's medical expertise offered commercially - <AUTHOR> http://news.yahoo.com/watsons-medical-expertise-offered-commercially-173205800.html <comment><USER2> I hope everyone realizes this is the beginning of the end of the practice of allowing doctors to offer their own diagnoses. Doctors already cannot offer treatments they invented themselves (the evidence-based revolution in medical practice now only allows treatments vetted by clinical research), but computer-based "expert systems" appear to be able to offer more reliable diagnoses, given a clearly expressed set of symptoms. And Watson wasn't the first expert system to show this ability -- an expert system named MYCIN (early 1970s) performed better than clinicians with whom its results were compared:

[http://www.it.bton.ac.uk/staff/lp22/CS237/CS237MedicalXSys.h...](http://www.it.bton.ac.uk/staff/lp22/CS237/CS237MedicalXSys.html#Evaluation)

It is important to add that, notwithstanding MYCIN's demonstrated abilities, it was never used in actual clinical practice:

<http://en.wikipedia.org/wiki/Mycin>

A quote: "MYCIN was never actually used in practice but research indicated that it proposed an acceptable therapy in about 69% of cases, which was better than the performance of infectious disease experts who were judged using the same criteria."</comment>


Azithromycin Hydroxychloroquine Coronavirus Mortality Rate - 0.4% (5 in 1300) - <AUTHOR> https://twitter.com/RiganoESQ/status/1242835657237635083 <comment><AUTHOR> Professor Didier Raoult describes the toxicity of hydroxychloroquine treatment and covid-19 mortality rate.

Out of 1300 patients treated, he had 5 deaths.</comment>


Ask HN: 'flag' should have a confirmation - <AUTHOR>

I've rarely, possibly never, 'flagged' an article for deletion here. Today I think I did that by accident.<p>I would suggest that either the flag link should have a confirmation step, or that it should not appear in the summary (New, Ask). Instead, it should only appear on the discuss page. <comment><USER2> I believe you can unflag?

<comment><AUTHOR> ... You are in fact correct, and that means that I didn't accidentally flag the article. Thanks!</comment></comment>


UK government pulls back from rule 'gagging' researchers - <AUTHOR> http://www.nature.com/news/uk-government-pulls-back-from-rule-gagging-researchers-1.19775 <comment><USER2> Related: [https://news.ycombinator.com/item?id=11519438](https://news.ycombinator.com/item?id=11519438)</comment>


Ask HN: By when will majority of homo sapiens become true atheists? - <AUTHOR> While reading Homo Deus, I was intrigued by this question of when will a consensus be drawn among us that we are just an other animal and there is no one called God. a slow transition to this singularity can be observed reviewing our history, but I am curious to know your opinion on when we will reach there and what might be a supplant to God. Off course there will be anomalies in all casses, let's root them out as of now.Let's Discuss more in the comments section. <comment><USER2>
      a slow transition to this singularity can be observed
      reviewing our history

Citation? Atheists, using the modern definition, were known in Plato's time. Many so-called atheists today _promote_ facially superstitious beliefs. For example, the anti-GMO movement.[1] Indeed, I don't think I've personally ever met a single individual (myself included) who didn't systematically exhibit superstitious beliefs.

A propensity to superstition might be an unavoidable byproduct of abstract reasoning or of complex social organization. The first question shouldn't be when do we become true atheists, but whether we want to be. The next question might be, how do we devise processes and institutions that independently address our biases and superstitions, either by mitigating or reshaping them.

[1] Because lack of evidence is not evidence. Neither is appealing to uncertainty or supposedly unacceptable risks (annihilation of all humanity) without _quantifying_ those risks. In rational discourse there's no avoiding accepting the limits of existing knowledge. But people aren't very good at abstaining from drawing conclusions.

<comment><AUTHOR> "Archaic hunter-gatherers were just another species of animal. Farmers saw themselves as the apex of creation. Scientists will upgrade us into gods"

I think you misunderstood the context I am really emphasizing on, here. it's not about whether this world consists of "true" atheists or not rather it is about when can at least a majority of us become agnostic about God and supernatural powers. As @tomhoward mentions below superstition may be a byproduct after all ,but I agree with your take on the dire requirement needed to mitigate the rampage of unscientific belief being capitalized by demagogs knowingly or unknowingly. Mere education seems to be working but personally, I can't withstand its dawdling struggle against gov funded conventions. What pragmatic steps do you think we can take to ramp up the processes of awareness without being ousted by the society or of that sorts.

</comment></comment><comment><USER3> I started my militant atheist phase about 10 years ago, then emerged from it about 4-5 years ago as I started to understand enough about the world/universe to realise that much of the true nature of existence is way beyond our ability to comprehend.

That doesn't mean I have any belief in a theistic, interventionist god.

It does mean I have an acceptance and appreciation for the transcendent and the unknowable, similar to what Einstein and Darwin described of themselves.

I'd recommend exposing yourself to the writings of physicists like Roger Penrose and Freeman Dyson, and recent podcast appearances by physicist Eric Weinstein, to understand how religion and spirituality can (and indeed, must) co-exist with physics.

To answer the question in the title of your post: we are so far away from understanding the fundamentals of the nature of reality, that such a question doesn't bear any serious consideration at all.

<comment><AUTHOR> " I started to understand enough about the world/universe to realise that much of the true nature of existence is way beyond our ability to comprehend."

Can you quantify this with one of your own experiences? I will learn from it vicariously if possible. I am not militating theists and neither am I propagandizing atheism. Surprisingly enough these superstitions have saved animals and humans before (Ex: Hinduism, Bhudisim, Jainism, etc.) but caused more harm than good. I will look up people you referred and their writings. We will always be unbeknown about God or some super power because no matter how much technically advanced we become, there will always be layers deeper to dig into and we can never declare that we know everything to prove that God doesn't exist.

We can only fail to accept God but never truly reject him.

This is a universal truth.

You have all the freedom to be an agnostic(which I am too) and not an atheist, but what I am urging on more people should be educated enough to be agnostic and have the courage to question an action made in the name of God the almighty.

The question was to rekindle thoughts among HNs as to where we really stand in this path towards unknown waters.

<comment><USER2> Replace "God the almighty" with "homeland", "honor", or "science". Does it make a difference?

If it doesn't make a difference, then what does any of this matter? If the issue is people using imaginary persons, relationships, and qualities to justify bad behavior, I don't see how the response should be promoting a reactionary identity that fundamentally commits the same error. Some communists committed unimaginable horrors in the name of suppressing religious ideas. I'm not trying to equivocate history, but am trying to point out that replacing theism with anti-theism or even skeptical-theism isn't necessarily going to produce different outcomes in the long run. God exists, such as he does, whether you say "there is a God" or you say "there is no God" or you say "I don't know if there's a God". The only way to purge the idea is to say nothing at all.

If there is a difference, then there might something distinct and salient about such a concept that might be beneficial and worth preserving, notwithstanding all the harms.

</comment></comment></comment><comment><USER4> Extrapolating from history: never

Based on the power of HN comments to change humanity: never

We have to start by not capitalizing "god" and improving our writing skills to better argue with the believers.

</comment><comment><USER5> when we meet sentient aliens and they laugh at us?</comment>


Sabotaged by Polish orthography - <AUTHOR> http://blog.plover.com/lang/ogonek.html <comment><USER2> While we're in this thread: notice that pączki is a plural. Please remember this and don't say 'pierogies'—it's just pierogi.

There's no need to know the singular for pierogi, because no one has ever eaten just one.

In return, I promise to continue working to stop Polish people from pluralizing potato chips as "chipsy".

<comment><USER3> > There's no need to know the singular for pierogi, because no one has ever > eaten just one.

Ain't that the truth! My Polish friend also introduced me to 'pierogi rooski'? Apparently, a Russian take on pierogi, with more meat. Do I have the spelling correct?

> In return, I promise to continue working to stop Polish people from > pluralizing potato chips as "chipsy".

As a Jamaican, where banana chips are like a national snack, it was amusing to see "chipsy bananowe" for sale. Fun times; I plan to go back.

<comment><USER4> > My Polish friend also introduced me to 'pierogi rooski'? Apparently, a > Russian take on pierogi, with more meat.

If you mean "pierogi ruskie", they don't have any meat in them, just quark with potatoes and onion, though they're often served with bacon. And this dish comes from "Ruś" (now Ukraine), not "Rosja", otherwise it would be called "pierogi rosyjskie". I hear that in Ukraine they call this dish "Polish pierogi".

> As a Jamaican, [...], it was amusing to see "chipsy bananowe" for sale.

I believe "it" was not a Jamaican, however amusing it was.

<comment><USER5> > "pierogi ruskie" [...], though they're often served with bacon

Blasphemy!

</comment></comment></comment></comment><comment><USER2> Polish orthography is a disaster partly because of the choice of Latin over Cyrillic. The latter would have been a much better fit for the sounds in the language.

Making matters worse, there are homophones ( _ż_ and _rz_ , _u_ and _ó_ , _ch_ and _h_ ) that depend on when the word entered the language.

Like so many things wrong with the country, you can comfortably blame the Catholic Church for this orthographic train wreck.

<comment><USER6> And yet - somewhat hilariously, Polish is one of the easiest languages out there in terms of knowing how to pronounce the written word.

Spend a week or two practising a pronunciation guide and that's pretty much well it.

Reading something out loud for the first time? Native level accuracy 95% of the time, easy.

Compared with the absolute crap shoot which is English or - my nightmare - French, reading Polish out loud is an absolute walk in the park.

<comment><USER7> My mouth actually hurts when I speak French. Such unusual shapes for a native English speaker :\

<comment><USER8> In a famous polish novel "Krzyżacy" French language is compared to shaking of tin bowls, so even Polish speakers have troubles.

</comment></comment></comment></comment><comment><USER2> A small correction to the OP: _paczki_ means packages and not boxes ( _pudełka_ ). It used to be a much more common sign in Polish neighborhoods in the US before Poland leveled up to the first world.

<comment><USER9> .. and the of course it depends how you want to use the objective boxes (pudełka).

I will throw you some pudełka. I am busy with those pudełkami. There is something wrong with those pudełkom. Whats the size of these pudełek?

And its funny: the newest IOs still don't have "ą"

</comment></comment><comment><USER10> In comments poles often omit those squiggles which lead to interesting nuances:

"Laske mi robi." means "to condescend/deign to do something for somebody" (ł) or "to do a blowjob" (l).

</comment><comment><USER11> Similar to how Spanish will not accent capital letters. The México passport for example says MEXICO on the front.

<comment><USER12> How strong a rule is this? Or is it regional?

[http://buscon.rae.es/dpd/srv/search?id=BapzSnotjD6n0vZiTp](http://buscon.rae.es/dpd/srv/search?id=BapzSnotjD6n0vZiTp) is from the Diccionario panhispánico de dudas, 2005. It shows several examples of accented capital letters, like LA NACIÓN in the headline of a newspaper, and the phrase "ESTÁ PROHIBIDO FUMAR DENTRO DE LAS DEPENDENCIAS DE LA EMPRESA."

That said, La Nación itself doesn't use accents when capitalized. On the other hand, you can see "EL PAÍS" several times at [https://elpais.com/](https://elpais.com/) . On the third hand, the El País in Uruguay doesn't use the accent [http://www.elpais.com.uy/](http://www.elpais.com.uy/) . But it does uses ALCANZÓ in a headline [http://a2010.kiosko.net/02/06/uy/uy_elpais.750.jpg](http://a2010.kiosko.net/02/06/uy/uy_elpais.750.jpg) .

[http://procedimientospolicialesargentina.blogspot.de/2016/04...](http://procedimientospolicialesargentina.blogspot.de/2016/04/19-de- abril-dia-de-la-policia-de-la.html) has an interesting mix with the blog title "DIA DE LA POLICIA DE LA NACION ARGENTINA" and a poster image saying "DIÁ NACIONAL DEL POLICÍA".

<comment><USER11> The reason it's similar is because it's not a rule. It's just sometimes left off.

<comment><USER12> My apology, I misinterpreted "will not accent" as implying some sort of rule.

</comment></comment></comment></comment><comment><USER13> For the benefit of French speakers, the ą in Polish seems to be pronounced like the "on" sound.

So "pączki" is pronounced "pon-tch-qui" while "paczki" is "patch-qui".

</comment><comment><USER14> I lived in a Polish neighborhood in Brooklyn for a few years, and absorbed as much orthography and pronunciation as I could during that time. I ate a few pączki, too.

While I learned to speak a little bit, it was not enough to be functional beyond basic greetings and ordering food. It was always a lot of interesting fun, though.

The main outcome of all this is that for ten years, whenever I see a Toyota Camry, my weird brain thinks "hmm... tsahm-rih..." and I roll my eyes at it.

</comment><comment><USER15> Ugh, this reminds me of character encoding issues... I hope I never meet the guy who invented ISO-8859-2 or CP1250 or other such nonsense for that guy's sake...

</comment><comment><USER16> This brings the memory of Internet chats in the 90s when almost nobody used Polish letters with diacritics (because it is faster to type without them and there were a lot of different incompatible encodings). Usually you can understand the meaning from the context but sometimes the same kind of funny misunderstanding would happen as there are quite a few words that without diacritics become completely different but valid words.

</comment><comment><USER17> Wikipedia gives "Zażółć gęślą jaźń" as an example containing all the Polish diacritic letters.

Is there a standard way to write this when limited to ASCII?

For example, Danish/Norwegian replace æ, ø, å with ae, oe, aa. It seems less likely that something could exist and be reasonably readable for Polish.

[https://en.wikipedia.org/wiki/Polish_orthography](https://en.wikipedia.org/wiki/Polish_orthography)

<comment><USER18> It is quite common to remove the diacritics if you are lazy or don't have access to the diacritics. That phrase becomes "Zazolc gesla jazn".

Most search engines find "Zazolc" and "Zażółć" equal because of that. This becomes a problem in case of the words like "paczki" (boxes) and "pączki" (donuts), which have their own separate meaning - as explained in the article.

In contrast to most European countries, in Poland we use American keyboard layout with "Polish (programmers) layout" keyboard setting in OS.

You press ALT+A, ALT+E, ALT+L, ALT+S, ALT+C, ALT+Z, ALT+X to write "ą", "ę", "ł", "ś", "ć", "ż", "ź", respectively.

</comment></comment><comment><USER19> Its curious how simultaneously we are stubbornly holding onto diacritics (and current orthography in general) while also having continuously lots of issues actually reproducing/handling them. One would think that we either would have gotten better at dealing with them, or dropped them altogether. Especially considering that modern fixed orthographies are generally relatively recent phenomenon.

<comment><USER20> Usually the diacritics exist because of something that is contrastive in the language's phonology. This post is mainly about a concrete example of this, where pączki and paczki refer to two different things that a shop could sell. So it's super-helpful that shops can make that distinction in writing!

Or for example

[https://en.wikipedia.org/wiki/Minimal_pair#Stress](https://en.wikipedia.org/wiki/Minimal_pair#Stress)

In Portuguese, my strongest foreign language, I can think of examples like

a 'the (feminine)' / à 'at the (feminine)'

nó 'knot' / no 'in the'

dá 'gives' / da 'of the'

nós 'we' / nos 'in the (plural)'

sê 'I should be' / se 'oneself' / sé 'see (Catholic)'

pode 'can' / pôde 'could (past)'

avô 'grandfather' / avó 'grandmother'

tem 'he/she has' / têm 'they have'

among many others.

At least a couple of these reflect vowel differences, although some would be homophones in speech. Every language that holds on to diacritics would have pairs like this where the diacritics make a difference to the meaning. So people may really appreciate having writing systems that can reflect these differences in order to avoid confusions that would otherwise occur.

<comment><USER19> Obviously naively just stripping diacritics won't work. But that doesn't mean that diacritics are the only option for orthography; for example in nordics ä and ö (or the equivalent norwegian/danish ones) could be substituted with ae and oe, because those do not occur otherwise in the language, and so the orthography remains unambiguous.

Maybe a good example of related development is the disappearance of Þ (thorn) from english orthography, being replaced with th.

<comment><USER20> > Maybe a good example of related development is the disappearance of Þ > (thorn) from english orthography, being replaced with th.

This didn't have quite the full benefit that you describe, though, because of compound words like "cathouse", "hothouse", "lighthouse", "outhouse", etc. And of course digraphs can be extra-risky whenever languages accept loanwords.

</comment></comment></comment></comment><comment><USER21> Nit: it's not pawnch-kee if you're speaking American English, it's more like pown-chkee. But still not quite.

<comment><USER22> Yeah, we poles are as obsessed with our prononciation as much as italians are obsessed with their food.

People form other countries just _can 't_ get it right :)

<comment><USER23> It is pronunciation with u, not prononciation...

<comment><USER24> I wouldn't be so sure about that.

</comment></comment></comment></comment><comment><USER25> this reminds me of what seems like a neverending debate about the actual pronounciation of paczki. Is it 'pawnch-kee' or 'poonch-kee'. haha.

Which lead me to this interpretation [https://www.youtube.com/watch?v=zdNsFOPYMzE](https://www.youtube.com/watch?v=zdNsFOPYMzE)

<comment><USER26> It should be a vowel sound similar to the English words "thought", "dawn", "fall", and "straw", except nasalized. So "pawn" is probably closer than "poon". (If any one of those have a different vowel sound than the others in your dialect, use the more popular version.)

<comment><USER27> FYI: The way I explain it to those who speak French is that "ą" is similar to the French "on".</comment></comment></comment>


Ask HN: HN inbox, feature to send/receive messages to/from HN users - <AUTHOR>

Few times I was in need to contact other HN users. But not all HN users have mentioned contact info in their profile.<p>Would it be nice to have Reddit/Quora like inbox feature for HN users, as part of HN itself? <comment><USER2> This idea comes up from time to time. And majority of the community doesn't want it.

One of the HN's most appealing side is it's simplicity.

Anyway, those that want to be contacted puts their e-mail/website into their profile.

</comment><comment><USER3> Maybe just have it push to email, and make it easy to turn off. I, for one, don't want another inbox to check.

That way, you can contact anyone, but everyone's email is not visible.

</comment><comment><USER4> What I wish is a notification when someone responds to my comment. Nothing more. I love tracking comments and having constructive discussions, but without notifications, it becomes very difficult.

<comment><USER5> Check out [http://hnnotify.com/](http://hnnotify.com/)

<comment><USER4> This is great, thanks! Didn't know it.</comment></comment></comment>


What is NoSQL for? - <AUTHOR> http://blog.cubrid.org/web-2-0/what-is-nosql-for/ <comment><AUTHOR> Pretty good overview for those who want a high-level catchup.</comment>


A Globe, Clothing Itself with a Brain (1995) - <AUTHOR> http://www.wired.com/wired/archive/3.06/teilhard_pr.html

<comment><USER2> Sounds like James Cameron has another influence to thank for his Pandora.

</comment><comment><AUTHOR> A strange eddy in the history of ideas.</comment>


What are the most surprisingly useful PHP functions? - <AUTHOR> http://www.quora.com/What-are-the-most-surprisingly-useful-PHP-functions

<comment><USER2> > extract() - dynamically destroy local scope

Very nice.

> class_alias() - dependency injection the PHP way

Didn't know about that one. That's messed up.

> __halt_compiler()

What?</comment>


San Francisco's Bureaucracy, Unions Stifle Modular Housing for Homeless (2016) - <AUTHOR> https://www.forbes.com/sites/scottbeyer/2016/09/23/san-franciscos-bureaucracy-unions-stifle-modular-housing-for-homeless/ <comment><AUTHOR> The more I think about it, the only way to resolve homelessness is by charity and charitable organizations that will donate private land and find subsidized ways to build for the homeless. Anything the govt gets involved with gets lost down the red tape lined rabbit hole.

Patrick Kennedy btw did go ahead and build those dorms for UC Berkeley and they are renting 300 sq ft studio dorms for $2265/month to students.

There is a lot of derision towards NIMBYs but why doesn't anyone consider unions? Thoughts?</comment>


Ask HN: What's a good introduction to the actor model? - <AUTHOR> I'd like to learn about the actor model, and especially about its use in real world asynchronous (networking) applications.
What I've seen so far are contrived examples that demonstrate the similarity (or differences) to OOP or just showcase message passing, but that's barely enough to get a feel for how a real program would be structured around the actor model.
Books, tutorials, and videos are welcome.  It would also be nice to study the code for a small (<10k LoC) application that does more than contrived examples. <comment><USER2> Late to the party, but...

The Distributed Job System I've written for .NET [1] uses Akka.NET under the hood.

Mind you, this runs 'distributed' in the sense that you can have multiple servers; the actors in the Execution System itself don't do anything outside of process. However, they are very much written in the actor spirit.

There's only 3 Actors, but that might fit the bill for being something that's not 'hello world' but not huge.

The entire system is about 4KLOC, And the Actor part of the system is under 1KLOC including the Class that manages the ActorSystem at runtime.

The unmerged branch has some even more advanced examples around recovery as well as a more 'pure' model around the Coordinator (In /master/ there is still a blocking call to the Queue.)

[1] [https://github.com/to11mtm/oddjob](https://github.com/to11mtm/oddjob)

</comment><comment><USER3> I'd explore the Erlang language and, specifically, building OTP applications. Anything in this [0] list should be good. I'd recommend starting with Joe Armstrong's writings. If Erlang doesn't work for you, check out Elixir. I haven't, so I can't recommend any particular books or resources on it, but I have read and heard great things about it.

Joe Armstrong did an excellent job explaining the motivation behind Erlang's concurrency model (which is based on the actor model), and the language is _very_ practical. From there, move to other texts or resources on the OTP portions of Erlang to understand how to develop actor-based applications at scale.

[0] [https://erlangcentral.org/books/](https://erlangcentral.org/books/)

</comment><comment><USER4> Just learn Akka, it's the most complete Actor framework out there.</comment>


Evolutionary Neural Networks - <AUTHOR> https://github.com/thepropterhoc/TF-Genetic <comment><AUTHOR> TensorFlow-backed Evolutionary Neural Networks.

Simulate a population of neural networks as individuals and evolve their structure to find your optimal mix.

Written in pure Python and TensorFlow. Also buggy as hell considering I wrote it in an afternoon.

Enjoy

<comment><USER2> This seems really cool. It can be pretty unintuitive sometimes which activation function(s) to choose.

<comment><AUTHOR> Exactly. This lets you test it out for you and actually can pick up on some highly nonlinear behavior. Actually applied this myself to housing price prediction with better results than single functions on a layer.

<comment><USER2> Makes sense. Out of all of the permutations of functions, I see no reason why using the same function at every node would lead to an optimized network.

I really commend the ML people out there that have developed an intuition for network architecture and which functions to use... If there's a Feynman for machine-learning, I wanna listen to some of his lectures.</comment></comment></comment></comment>


Jepsen: Aerospike 126.96.36.199 - <AUTHOR> https://jepsen.io/analyses/aerospike-3-99-0-3 <comment><USER3> I highly recommend people watch Kyle Kingsbury's talks on YouTube. They're extremely interesting, especially to see the differences in issues that affect each database. He's also a great speaker, and makes it entertaining.

</comment><comment><USER4> Anyone using AeroSpike in production? It's not a project you hear about much these days.

<comment><USER5> MySQL Cluster (NDB) is a better choice for many looking for a near real-time key-value store. It support cross-partition transactions, partition-pruned index scans, partition-aware transactions, multi-master asynchronous geographic replication between clusters, an Event API, on-disk columns, on- line add node, and has been around the block a lot longer. We benchmarked AeroSpike vs NDB (a key-value store on top of NDB), and NDB out-performed Aerospike for most workloads with equivalent hardware: [http://www.diva- portal.org/smash/get/diva2:849736/FULLTEXT01...](http://www.diva- portal.org/smash/get/diva2:849736/FULLTEXT01.pdf)

<comment><USER6> Is this tested with Jepsen?

<comment><USER5> I don't think so. I believe it would fail quite a few of Jespen's tests, as it is a real-time DB. Typically, you set the transaction deadlock detection timeout to be a couple of seconds. Jepsen thinks transactions can take minutes to complete, and it's ok - the messages are just late. In fact, in NDB, most people think 2PC is a blocking protocol, but it isn't in NDB. If the Transaction Coordinator (TC) fails, there is a quick leader election protocol and a new one takes over failed TCs. Participant failures cause blocking - <USER2> only a few seconds (the configured deadlock detection timeout). So, it's behaviour is similar to abortable consensus (aka Paxos), if automatically retry failed transactions.

<comment><AUTHOR> > Jepsen thinks transactions can take minutes to complete, and it's ok

Jepsen takes great pains to make no assumptions about the time it takes for transactions to execute. We test systems with millisecond-level timeouts, or multi-minute timeouts, and both work fine.

<comment><USER5> Ok, my mistake. I saw you tried to install NDB several years ago, without any luck. Nowadays, there's mysql cluster mgr, severalnines installer, and we wrote some chef code to automate it. So it's pretty straightforward.

</comment></comment></comment></comment></comment></comment><comment><USER7> Aerospike Founder here. Does anyone care that, after working with Kyle and generating since new versions through the development process, Jepsen found no real consistancy problems?

ie, "In hundreds of tests of SC mode through network partitions, 18.104.22.168 and higher versions have not shown any sign of nonlinearizable histories, lost increments to counters, or lost updates to sets." – Kyle Kingsbury, Aerospike 220.127.116.11, 12-27-2017

Curious.

<comment><USER8> As someone who follows Kyle's work I can say that with Jepsen it's only impressive and shows how serious you are about consistency and distributed systems if there are no real consistency problems on the first try.

</comment></comment><comment><USER9> Be aware it's nothing to do with rocket science.

<comment><USER10> There are so many software libraries named after normal concepts from physics and engineering. While I see it as dreamfull flattery, it is really damned annoying when you travel in all of these worlds, especially here on HN where people post all of it.

(Best Regards, Grumpy software-, (defense-)systems engineer and former particle physicist).

<comment><USER11> I find this one especially egregious because Jepsen looks a lot like Jeppesen the aerospace products company. They mostly do charts and stuff though so I was pretty confused that they were mentioned alongside the term 'aerospike'

<comment><USER12> IIRC, Jepsen here is named after Carly Rae Jepsen, the pop singer.

<comment><USER13> As in "call me maybe", both a reference to the song lyrics and what might happen in a distributed system when a network partition occurs.

</comment></comment></comment></comment></comment><comment><USER14> Community edition is not a production ready product.

Use Aerospike only if you are ready to buy enterprise edition. Jepsen test results are applicable to enterprise edition only.

<comment><USER15> Can you elaborate on that? What's missing from CE that makes it not suitable for production? Thanks for your input.

<comment><USER16> One of the biggest issues is that deletes aren't durable in the CE edition. The index entry will be removed but the key and data remains, and may or may not be reclaimed by a background compaction process. If the server restarts before the data is compacted, the deleted data is revived because the index is recreated.

</comment></comment></comment><comment><USER17> I would like to run the jepsen tests on my own with the current versions of Mongo and Redis, does anyone know how to do that?

<comment><AUTHOR> The Redis test was quite early in Jepsen's history, and I totally rewrote the library after that--you can still find the original code in the `old` branch, if you'd like to experiment. The Mongo tests are reasonably current; you'll just need to pass a new --package-url on the CLI, I thiiiink. No promises, of course, compatibility is always a moving target. ;-)

</comment></comment><comment><USER18> I am asking this because I think it's funny (so don't be mad) but does anyone know what this means?

 _" AEROSPIKE 4.0 NOW GA

Strong Consistency with High Performance for Systems of Record and Systems of Engagement"_

It seems like Marketing 3.0 speak, maybe their target market would understand it intuitively?

<comment><USER19> > Systems of Record

OLTP

> Systems of Engagement

OLAP

<comment><USER18> ty, knowing they actually mean something is reassuring. I've been seeing too much of this kind of buzzword-mashups lately. Glad to see I am wrong. :)

</comment></comment></comment><comment><USER20> This kind of analysis is awesome. In a past job I might have deployed Aerospike, then eventually realised it sometimes lost my data. But that would be years later, after much midnight oil had been burned.

</comment><comment><USER21> This reminds me of the turbo encabulator.

<comment><USER22> [https://www.youtube.com/watch?v=rLDgQg6bq7o](https://www.youtube.com/watch?v=rLDgQg6bq7o)

</comment></comment><comment><USER23> My take - if you didn't test your app in nodr loss scenario, you are likely to lose data when that happens, even if your DB won't.</comment>


Engineyard SHA Contest Leaderboard I threw together - <AUTHOR> http://jazzychad.com/engineyard/

<comment><USER3> It looks like "hashbreaker" and "CodingCrypto" are both running through nearly the exact same word sets. Either they are from to different people using the exact same algorithm or (what I suspect is more likely) they have created multiple twitter accounts to get around the submission limit.

      hashbreaker
      bUGs BUgs bUGS bUgS bLAnk BlAnK BlANK BLank buGS bugS bUGS bUGs dCao2

      CodingCrypto
      BugS BUgs buGs BUgs bLANK Blank bLanK BlAnk BuGs BuGs BUGS bUGs FcBkw

      CodingCrypto
      cOWs cOwS cOwS cowS DIRtY Dirty DiRTY DIrtY cOWS Cows CoWs Cows owjo0

      hashbreaker
      CoWS coWS CowS cOWS DiRTY DIrtY DirTy DIRtY COWS COWS cOwS cOWS bvpDq

Edit: Also, accounts had their first tweets on the 19th.

<comment><USER4> I didn't scrutinize the sha-1 algorithm, but is it possible to optimize its computation (e.g., precompute intermediates) when using a small number of repeated words as input? This might explain why a group of people all arrived at similar solutions, since it only matters how quickly you can guess if all guesses are pretty much equal.

I lucked out and got a solution at 35. Coded something up in python and only got about 85k hashes/sec on one core. The CUDA program, though, was about 100x faster on my low-end Nvidia 8400 GS.

<comment><USER5> Yes: very roughly, you could precompute the internal state after 'abcdefg' -- essentially checkpointing the algorithm -- and then compute SHA1('abcdefgX'), SHA1('abcdefgY'), etc. more easily than other strings. (In fact, you'd want your checkpoint aligned at some multiple -- maybe 512 bits? -- but that's the general idea.)

I figured the 'final 5 arbitrary characters' was a nod to this approach. So the best strategy I could think of -- and I didn't try coding anything up for the contest -- was to (1) guess the word dictionary; (2) precompute a whole bunch of checkpointed-except-for-last-five-characters function states; (3) at the moment the real dictionary and target was announced, try many different last-five-character extensions against any valid precomputed states.

</comment></comment></comment><comment> brown9-2 Kind of interesting or funny that these entries are pretty close to the top of the standings:

      31  	d241c4b4dd38aacb197c8e48e9f8488264e39a83
      bUGs BUgs bUGS bUgS bLAnk BlAnK BlANK BLank buGS bugS bUGS bUGs dCao2

      32  	334945bffdd1000f1d198e02497188a2b17f1991
      cOWs cOwS cOwS cowS DIRtY Dirty DiRTY DIrtY cOWS Cows CoWs Cows owjo0

(assuming the score are correct)

</comment><comment><USER6> Would the contest have been any easier had one made the (dubious) assumption that the "passphrase" encoded would _likely_ be a valid sentence?

I've not tried any crypto work before but this has really grabbed my interest. It just occurred to me however, that discarding lines of inquiry leading down the "BuGS bugs bUgs BUGs BLAnK BlAnK....." route in favour of something that made more sense grammatically could be a good tactic to save time?.... or it could just increase the overhead in generating the next hash?...

<comment><USER7> Unless you found the exact sequence, the difference between hashes won't tell you how close you are.

<comment><USER6> I don't see what you mean by "the difference between hashes won't tell you how close you are" .... isn't that the whole point of this competition?

Say for instance you coded four stages:

1/ Generating a list of strings for analysis 2/ Parsing that list for sentences that made sense 3/ Hashing these sentences 4/ Comparing the hashes generated with the one being sought

to me it just looked as though a lot of gibberish was analysed because there is a lot more entropy than order to sift through and nobody tried step 2... does my logic make sense or am I getting mixed up and not really seeing the bigger picture?

<comment><USER8> There's no benefit in trying step 2 because the gibberish input is as likely to be a close match as a sentence that makes sense.

A sentence that starts 'Be' (001000010...) and the same sentence changed to 'He' (001001000...) would produce very different hash results by the Avalanche Effect ( <http://en.wikipedia.org/wiki/Avalanche_effect> ). So having similar letters in a similar order or in similar positions, or using similar length words, wont help.

 _Isn't that the whole point of this competition?_

The point was to get a hash with a similar quantity of 0 and 1 bits in the output as their hash, not to guess their sentence (they said what it was).

<comment><USER6> _The point was ... not to guess their sentence (they said what it was)._

Ahh, I missed that! That makes sense in terms of the Avalanche Effect - <USER2> for the knowledge

</comment></comment></comment></comment></comment><comment><USER9> Twitter accounts with "private" updates do not show up on the public timeline... my three submissions are nowhere to be found and they do validate.

</comment><comment><USER10> Good job! Could you add a timestamp and option to sort by it? (I know, feature creep!)

ETA: I also find it interesting that the lowest score from those not using the optional 5 random chars is currently 38. All the lower ones use random chars.

<comment><USER11> Right now you're showing duplicate entries (the same entry of value 31 is showing from two people)

<comment><AUTHOR> That's because somebody just copied and pasted somebody else's entry. The system is working as intended. I'm sure Engineyard will catch it in their judging (or so I hope).

</comment></comment></comment><comment><USER12> the rules says you can "append up to 5 characters" but you consider entries with less than 5 optional characters invalid.

<comment><AUTHOR> No, there are several listed with less than 5 extra characters. It's just that most people are using all 5.

<comment><USER13> My entry (rkneufeld) is cut off at the end. Actual suffix is: "3:OID". This moved me from 34 to 68 :( . Just thought you might like to know

<comment><AUTHOR> Apologies. I have corrected it.</comment></comment></comment></comment>


Show HN: I built this in 5 minutes, and this is why Ratchet is awesome - <AUTHOR> https://github.com/zafriedman/ratchet-signup

<comment><USER2> I give up. I failed to find the awesomeness. I see a login form (view). Ratchet is an MVC framework, your page does nothing. :(

<comment><AUTHOR> The awesomeness in my opinion is that a) this took me literally five minutes which means that I had to write absolutely none of my own CSS classes for mobile prototyping, the wonderful guys who developed Ratchet did that for me already! Also, Ratchet is not an MVC framework, it's a front-end iPhone targeted mobile prototyping framework. It allows you to prototype a mobile UI with HTML/CSS/JS faster than prototyping in native Objective-C. At first glance, I honestly think these guys are being humble by calling it a prototyping framework. I could see myself using this for mobile web versions of my apps, although that does presuppose that everything "works" pretty closely to the level one would expect it to reading the docs on their homepage. Check out the link I posted in another comment, you might be confusing Ratchet with something else.

<comment><USER3> Interesting, so how is it different from something like jQuery mobile?

<comment><AUTHOR> I think it really comes down to a matter of preference. This looks interesting to me, and it caught my eye when I saw it on Hacker News much in the same way Meteor did when it came out. For me personally, it just passed the initial smell test really well.

In terms of some concrete attributes which I like, I think it probably is the best framework I've seen in terms of the general aesthetic. I have high hopes that it will allow me to create most of the view elements I'd need to prototype in an incredibly short period of time. I haven't used jQuery mobile for some time now (> 1 year) yet it seems like there are more CSS classes in Ratchet that mimic the iOS look and feel well enough that one could develop a full iOS app prototype in a day or less. If I remember correctly from using jQuery mobile, there was a bit more code to write, and the general look just didn't feel as good to me.

</comment></comment></comment></comment><comment><AUTHOR> UPDATE: This was probably really stupid of me to do right now, but I've been wanting to change my GitHub name for awhile... and just did... so if anyone would like to checkout the repo you can find it now at <https://github.com/kulte/ratchet-signup>

</comment><comment><USER4> You forgot the sign up button! :-)

It has a form (missing a submit button) and then an "OR, sign up with Facebook".

</comment><comment><USER5> Looks great, is there a hosted link somewhere for people to play with without having to download?

<comment><AUTHOR> If what you mean is there a CDN for the Ratchet javascript, I don't think so. If you go to the website <http://maker.github.com/ratchet/> there is a download link, but my repo just augments what you get in the download folder... like I said, 5 minutes :)

EDIT: I see what you mean now, no I didn't put up a hosted link since I just developed the one screen during a 5 min break. That's a decent idea though, maybe I'll make a full demo application (maybe I'll make it a Todo list, no one has ever used that for a demo app before) and put it up on Heroku.

</comment></comment><comment><USER6> Hey, that's great, but you could already do rapid app prototyping all the way to full app development with Kendo Mobile. Paired with PhoneGap it's a highly productive way to build an app.

<comment><USER7> Can you give me some examples of apps built with Kendo Mobile so I can see how it actually performs?

<comment><USER6> I built the ReMeme app for iOS and Android using Kendo Mobile.</comment></comment></comment>