OpenBSD, C, httpd and SQLite – Web App stack - <AUTHOR> https://learnbchs.org/index.html <comment><USER2> I find the site a little tongue-in-cheek, but I genuinely like the ideas behind it. I honest-to-$DEITY prefer programming in C to any sort of Javascript. C is kinda hellish for large, complex projects, which is good. The world needs fewer large and complex websites. OpenBSD also provides very good security features that mitigate the inherent security challenges of writing C. pledge(2) is just a very good idea.

I generally think websites should do their processing in the backend, whenever possible. Running code to generate the website should not be the user's problem. This also comes with the advantage of not being constrained to *Script languages. C is alright, but if you're of the defeatist camp who thinks writing C safely is impossible, you can adapt the BCHS philosophy just as well to C++, Rust, Go, D, Python, AWK, Common Lisp…

<comment><USER3> C is also hellish for string processing.

From memory management to buffer overflows to understanding the encoding types.

<comment><USER4> Objective-C with a reasonable Foundation is such a great language for this...

It has essentially the predictability and simplicity of C, just with a minimal amount of dynamic binding to make it comfortable. And NSString variants tend to take care of string handling and encodings.

<comment><USER5> At that level, you can just switch to the clang compiler based Swift language, which is much nicer/modern. And look, there are already Swift web frameworks like [https://vapor.codes](https://vapor.codes)

<comment><USER6> I really dislike the Swift/Scala/Rust/Dart/Kotlin etc. style, heavy and centralised syntax. ObjC is not very beautiful, but it has an elegant way of combining composable primitives (functions and objects). The same kinda sorta goes for Perl too, which I think is a nicer alternative to C in this stack.

</comment></comment></comment></comment></comment><comment><USER7> If you want simplicity as your #1 objective then I would recommend Flask and Python. If speed is your #1 objective then you can try Go.

Maintaining a C webapp is terrible.

You don't have any primitives, you need to manage your own memory which is extremely dangerous, and the amount of best practices you need to learn so you won't shoot yourself in n the foot and create an exploitable/unstable application is immense.

<comment><USER8> Python is the opposite of simple.

If you want good performance, then, for non-trivial sites, you cannot afford to spawn the interpeter on each request, so you have to use some asynchronous tooling. Similarly, spawning connections is much heavier weight than it should be, so you end up with backend connection pools.

At work, we used fabric for this, and it blew up in our face at scale, and we had to rewrite.

In order to get off the ground, you also need to use pip, which causes an inordinate number of problems vs. the bsd ports tree or .deb files. Managing dependencies and versions between the host os and pip space still seems to be an open problem. You can use virtual environments for each script, but then patching zero days becomes infeasible.

If you want basic type safety, you need to rely on external tooling. In my experience, third party python libraries evolve types and api's without warning. This shows up at runtime, so it introduces security holes.

I could go on for hours.

<comment><USER9> > "you cannot afford to spawn the interpeter on each request"

You mention this as a case against using Python, but this is entirely unrelated to Python as a framework. In other words, had they used PHP or Node or %s, they'd still have to face the issue of optimizing the request-to- response path.

This relates much more to the issues that often come with the common practice of separating your webserver, from your business logic (Flask, in our case).

> "so you have to use some asynchronous tooling"

Solutions such as mod_cgi and wsgi have been the standard for years at this point. That's, in many ways, a solved problem.

> "In order to get off the ground, you also need to use pip, which causes an > inordinate number of problems vs. the bsd ports tree or .deb files."

Any specific issues you can expand on? Because this is a rather vague form of criticism.

Using pip is ridiculously easy. I've been using Python extensively in various production environments and rarely had to point a finger at pip. Package and dependency management in Python are solid (that isn't to say things can't improve).

> "You can use virtual environments for each script, but then patching zero > days becomes infeasible."

Again, this is an entirely subjective form of criticism. I haven't had any issues with keeping my virtualenvs up to date.

> "If you want basic type safety, you need to rely on external tooling."

Again, you just seem to favor strongly typed languages and that's fine, but that's hardly a point against using Python here. Type safety is simply not a Python thing, much like weak typing is not a C thing.

> "In my experience, third party python libraries evolve types and api's > without warning. This shows up at runtime, so it introduces security holes."

Developers will sometimes break APIs. How is this a Python-specific issue? These things are literally happening everywhere.

<comment><USER10> > Again, this is an entirely subjective form of criticism. I haven't had any > issues with keeping my virtualenvs up to date.

What tools do you use keep all of your virtualenvs up to date?

<comment><USER9> I can recommend Snyk[0] (I am not affiliated, have used their products with clients before). They've got a fully functional free tier, and they have a ton of relevant features. And yes, there are other, competing products out there you could use.

[0] [https://snyk.io/docs/snyk-for-python](https://snyk.io/docs/snyk-for- python)

</comment></comment></comment></comment></comment><comment><USER11> Important pieces of the puzzle are Kristaps' kcgi (perhaps kwebapp/ksql).

[https://kristaps.bsd.lv/kcgi/](https://kristaps.bsd.lv/kcgi/)

And also OpenBSD pledge / "sandboxing" on other OSs

[https://man.openbsd.org/pledge](https://man.openbsd.org/pledge)

[https://learnbchs.org/pledge.html](https://learnbchs.org/pledge.html)

[https://kristaps.bsd.lv/kcgi/tutorial6.html](https://kristaps.bsd.lv/kcgi/tutorial6.html)

<comment><USER12> I really hope other systems will adopt pledge. Of all the selective privilege dropping mechanisms I know of, pledge seems to be the least complicated to use.

<comment><USER13> Mojave has a new security aware runtime for its applications.

Bringing sandboxing to all applications and requiring listing of entitlements and digital signing.

Microsoft has decided if the apps don't come to the store, the store goes to the apps, and are merging Win32 with UWP sandboxing concepts via Windows containers and the new MSIX package format.

Android sandboxing is still the most complicated one.

</comment></comment></comment><comment><USER14> > because the open internet is damn inhospitable.

And the answer to this is writing text-processing functions that you expose to the world, in C... skeptical face

<comment><USER15> We could argue that modern static analysis tools and the compiler itself are good at catching a lot of errors, and that high level languages have bugs too. C is a dangerous beast, but with good practices it isn't that dangerous. Of course you have to be extra careful when dealing with user content, like JSON payload, but many decoders in other languages are written in C.

Rust is a good solution, but it isn't trivial to learn, and might not suite all situations. I think it's great to have a C solution like this.

<comment><USER16> I don't think any non-gc language is trivial to learn, C also. Manual memory management is hard, and either you learn from the system (C way) or learn it upfront from the language (Rust way), it's still the same amount of concept you have to digest to write practical software.

<comment><USER17> That's true, but the original idea of Unix is to have many small program invocations work together, not to erect a monolithical long running daemon that does it all in a single address space. Memory management for one-shot command line apps isn't hard at all and can often get away with static allocations. Even if you're screwing up your heap, process isolation will take care of recycling memory when your program terminates.

<comment><USER18> Many small programs simply doesn't scale to the modern web when just the baggage associated with spinning up processes will kill you at any reasonable load. UNIX has nice ideas but recognise they were founded in the shared computer between a few dozen users environment of the 70s, not a web server serving thousands of requests a second of today

<comment><USER2> What do you mean by modern web? I don't see how modernity implies more load, at least for fork() bottlenecked programs specifically.

If you're serving thousands of requests per second, you probably need a pretty beefy server anyway, regardless of the stack you're running. Forking makes good use of multiprocessor capabilities if nothing else.

<comment><USER18> Modern language runtimes used in the server space like Go basically multiplex green threads across processes to minimize context switches between concurrent execution paths. By leveraging the fact that I/O is much slower than processor ops and switching green threads on I/O without necessarily always switching OS process, they easily scale to hundreds of thousands of concurrent requests. I don't really know about Erlang's model but it also has green threads. There's a reason why high traffic sites use languages like these.

[https://talks.golang.org/2012/concurrency.slide#1](https://talks.golang.org/2012/concurrency.slide#1)

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> As much as I align myself philosophically to most things this project stands for, I wouldn't trust myself to write public facing web applications in C. The same philosophy could still be held up while replacing the last components with a less dangerous language like Rust(with the trade off of developing complexity, and possibly giving up a bit of control).

OpenBSD as a base operating system, running relayd and httpd to face the hostile web proxying for efficient services written in Rust would be my ideal choice. SQLite for most simple data store needs, and Postgres if the data management gets big enough that it might spill to another machine.

</comment><comment><USER20> Two points. Would be nice if there was an AMI on AWS for this stack.

Second point. We do a lot of RESTful microservices. Which might be good for this in the sense that with C you would want to keep your code small (cause no classes and manual mem-management). But what people do not always appreciate at first about microservices is that you need to rely heavily on libraries (either custom or external like Spring Security) to handle cross cutting concerns, else you are writing the same code multiple times. Essentially if you write a peice of code that you expect will go on more than one microservice, then it should go into library. So my concern is how easily would it be to create custom libraries to deal with cross cutting concerns like logging, and security in this stack, AND does C ecosystem offer external libraries for dealing with these common concerns that a modern RESTful microservice would encounter?

<comment><USER13> And most important how sure can one be that those libraries have been propely sanitized.

</comment></comment><comment><USER21> If you want to do web backend in C, this approach is far from the only game in town. For instance, there is this: [https://kore.io/](https://kore.io/)

I like the idea of a web framework having a C API. We can make bindings to it in different languages and have a kind of semantic standard across the board, like is done with numerous other things: numeric libraries, crypto, GUI, ...

</comment><comment><USER8> I suspect "unsafe" C leads to safer server side code when combined with a few, simple libraries.

One issue that seems to fly over the head of "safe" high level language proponents are that you need to know what data types you are manipulating in order to write safe code.

Also, language-level isolation is much harder to implement correctly than process-level isolation (and OpenBSD has spent tons of time hardening process- level isolation).

I'd love to see a security bake off of a few mature applications built with this and random-language-de-jour.

<comment><USER5> > I suspect "unsafe" C leads to safer server side code when combined with a > few, simple libraries.

As many mentioned this might be true, but only if you don't have to deal with user input, strings, unicode and anything but extremely simple manual memory management.

The essence is, C and similar languages like Rust are great for writing an OS or a browser, but you really, really, really do not want to write complex server side web apps in those languages.

Sure, you can force your way through and if you have 30+ years of C in your head, it might even be safe. But it is not reasonable nor advisable for the general case. IMHO.

<comment><USER22> Rust brigade engage!

Not sure why you're lumping Rust in with C here. Rust has automatic memory management as part of the language, _without_ a runtime, and it has excellent libraries for dealing with Strings, unicode, and concurrency and asynchronous functions. It's a perfect language for writing a complex server side web app.

<comment><USER5> Sorry, my ignorance. I've had only a brief look at Rust, it did't 'click' in my head. On a second thought, it seems much more viable than this plain C mockery. Especially with the right support e.g. [https://rocket.rs](https://rocket.rs)

</comment></comment></comment></comment><comment><USER23> I am seeing SQLite pop up more and more in articles that tout its effectiveness and simplicity for many use cases over popular alternatives like PostgreSQL. It has definitely persuaded me to consider it on future projects.

<comment><USER24> Oh yeah. SQLite is incredibly versatile and for most web applications it's more than enough. The only pain point I have with it is that all data is stored as a string no matter the table schema types specified, this can lead to some bugs, but if you have a good library in your language for SQLite then it's not a problem.

<comment><USER23> I am currently working with a data set that is billions of rows and tens of GB. Will be interesting to see how SQLite handles it, which I plan to try. All of that in one DB file!

<comment><USER25> The main place you might run into problems is if doing lots of simultaneous writes. As long you're mostly dealing with reads then it should work.

</comment></comment></comment></comment><comment><USER26> The example says no mysticism but is using at least 4 undeclared variables. I'm not a C developer so I'm sure I'm completely wrong, but where do pledge, puts, EXIT SUCCESS/FAILURE come from?

I assume from importing stdlib etc, which probably give you additional variables to use, that you would need to know about explicitly. One thing I like about zeit.co's micro server (js) was the argument about not using bodyparser and magically having res.body available, but using async/await and assigning it to a variable. This is popping up more with things like es module imports, render props in react, etc. They make it clear where variables come from and allow you to avoid stepping on existing variables. What other 10-100 variables can I step on potentially with global patterns as a new user?

<comment><USER27> It's surprising to me that someone could be a programmer without knowing anything about C.

<comment><USER28> The best programmer in the world may never have even seen a computer for all we know.

<comment><USER29> Dijkstra famously didn't use one until his friends and colleagues made him get a mac so they could email him.

</comment></comment></comment></comment><comment><USER30> I really don't like OpenBSD's httpd. Last I checked (three months ago?), it couldn't do something as simple as adding a custom header to a static file. Why you'd use it over Apache or nginx is beyond me.

<comment><USER31> I'd love to know what header you desperately need just for a static file, that would necessitate a behemoth such as nginx, or apache and all it's CVEs?

Simply having all the bells, whistles and free candy, doesn't determine 'toy' status

<comment><USER30> X-some-signal-to-tell-aws-that-its-okay-to-use-this-file: true

"Why would anyone need this" was also the typical dismissive response I got when looking around for help.

<comment><USER31> It's definitely a legitimate question - not everybody knows that AWS needs some silly header, nor why. OpenBSD people will be the first to call out unnecessary/outlandish requests unless you can back it up with a good reason outright.

I appreciate the fact that httpd doesn't attempt to be the end-all server for everybody - its primarily a lightweight way to run things like the BGPd looking glass, other tools, a simple website, or something via fastcgi. The term they use often when denying pull requests is 'Featuritis', which Apache & Nginx suffer from.

<comment><USER32> what would you recommend for someone needing to add headers?

<comment><USER33> If you're using httpd you'd probably pair it with relayd.

With relayd you can add custom headers, as this random example shows:

[https://github.com/reyk/httpd/wiki/Using-relayd-to-add- Cache...](https://github.com/reyk/httpd/wiki/Using-relayd-to-add-Cache- Control-headers-to-httpd-traffic)

However that's probably not great still, because it's a global solution to a per-file problem.

</comment></comment></comment></comment></comment></comment><comment><USER34> I may be young and dumb, but C programming seems very simple and straightforward to me. With abstraction comes lack of knowledge of implementation in favor of ease of use; but in C you can build your own abstractions quickly and reach a level close to even JS (opaque types, function pointers). At least in C you know exactly what is going on.

Can someone briefly summarize the security issues in C? If you manage memory properly and take a conservative approach to handling input, where is the risk?

Like I said, I'm young and have only be programming in C for ~3 years.

<comment><USER13> In C you think you know what is going on, specially bad when developers mix the compiler specific behavior with the standard and then try to write portable code.

\- Decays of arrays into pointers.

\- Decays of enumeration into numeric types

\- Default signess is implementation defined

\- Implicit conversions

\- Currently ISO C11 lists about 200 UB cases, C2X plans to list even more

\- Strings are a pointer to somewhere in memory that you hope the caller actually terminated them.

\- While the pre-processor seems basic, compared with real macros, you can still be very creative with it

\- No way to validate security issues in binary libraries

The LLVM and PVS Studio blogs have quite a few examples of C gotchas.

<comment><USER35> > While the pre-processor seems basic, compared with real macros, you can > still be very creative with it

That sounds like a benefit, not a downside. See, for instance, [http://libcello.org/](http://libcello.org/)

<comment><USER13> Try to be do maintenance work on a foreign code base and you quickly will change your mind regarding it being a benefit.

Doing code fixes on a server code which was using clever tricks to convert between memory handles and the real memory addresses teached me that.

</comment></comment></comment></comment><comment><USER36> Lol, about 10 years ago I wrote a CGI framework in C. It was fun and the minimalist in me loves the lark but for productive work especially working with other people, opting for a stack with next to no abstraction is crazy.

Put it this way; Would you invest in s startup intending to use this stack ? Can this stack ever be the result of pragmatic decision making?

If you can genuinely answer yes to both in your use case , then why not ...

<comment><USER33> People generally invest in startups because they believe the sales/customer numbers. Not because they pay attention to the implementation of every single component which is used by the company.

That said OkCupid famously used a custom HTTP-server to power their site:

[https://github.com/OkCupid/okws](https://github.com/OkCupid/okws)

</comment></comment><comment><USER37> > C is a straightforward, non-mustachioed language

Maybe non-mustachioed out of the box but I've done the mustachio thing to generate code for my ASDL parser backend.

Well...I guess it was a C++ library since it was "header only" but I'm sure I could have found a C library if part of the project wasn't to learn boost::spirit.

</comment><comment><USER38> Please...I'm convinced this "BCHS Stack" is just an elaborate joke.

</comment><comment><USER39> I can imagine an even more weird one: DOS, ASM, dBase... (;,;)

> C is a straightforward

Indeed, what can be more straightforward than manual memory management and pointer arithmetic...

</comment><comment><USER40> > "man pages and "-Wall -Wextra" are your new best friends" These guys are > more likely friends with "-pedantic"

<comment><USER2> And nothing wrong with that. Stricter compiler is a good thing if you don't need it to be laxer.

My favourite flag in clang is -weverything. Everything I write in C I compile with it and fix whatever I reasonably can.

</comment></comment><comment><USER41> I love this; we are definitely in the territory of Poe's law here.

I strangely really like the css formatting used in the website's source code (e.g. [https://github.com/kristapsdz/bchs/blob/master/json.css](https://github.com/kristapsdz/bchs/blob/master/json.css))

<comment><USER42> That formatting is cool. Why would they not want radio buttons to display? `input[type=radio] { display: none; }`

</comment></comment><comment><USER43> Doing anything on the web with a language that doesn't have a string type seems like masochism.

<comment><USER34> A character array seems like a natural way to me to represent a string of characters. But Unicode though...

<comment><USER44> As C functions can't take arrays as arguments, you don't get very far this way.

<comment><USER34> ...you pass in a pointer to the first character. Arrays in C are simply contiguous spans of memory.

<comment><USER44> Thid then has the problem that thr length is unknown, leading to well known problems.

<comment><USER31> Well if you wrote the array, or your program did, then you do know the length. Not a problem. Would it be nice if an array internally stored that? Sure. Does it? No. Is that a real problem? No, not really - its been solved and worked around for decades

</comment></comment></comment></comment></comment></comment><comment><USER45> If you're looking to develop a website exclusively with the tools that ship with OpenBSD, Perl would probably be a much saner choice than C, not to mention one with a long history of use in web development.

</comment><comment><USER17> I'm loving the spirit if not the snark, but it's still odd OpenBSD's httpd (not to be confused with Apache httpd) calls it's CGI gateway "slowcgi" (and kindof derides CGI programming). I mean, it being slow is entirely the fault of the O/S isn't it ?: And using C gets a whole level more challenging with long-running processes and async I/O because of manual memory management. Native CGI programming IMHO is best enjoyed with a reverse cache if you can help it, such as with Apache's mod_cache.

<comment><USER11> It's slowcgi.. you know, as opposed to fastcgi. It's just a name. It was implemented so that httpd(8) didn't have to support executing CGI programs itself and could speak only FastCGI.

You don't /have/ to use it, but it's safer.

[http://man.openbsd.org/slowcgi](http://man.openbsd.org/slowcgi)

<comment><USER17> I was referring to the release announcement back in 2014 or so which I'm unfortunately not able to find (there was more than a bit snarkiness towards CGI programming).

I'm aware what FastCGI is ;) Btw. if you're interested in native HTTP I'd be looking into nghttp2 rather than FastCGI and OBSD's httpd.

Edit: see also [https://ef.gy/fastcgi-is-pointless](https://ef.gy/fastcgi-is- pointless) and [https://news.ycombinator.com/item?id=9202039](https://news.ycombinator.com/item?id=9202039) (previous discussion of OBSD's httpd)

<comment><USER46> how can i trust a lib which still uses openssl in this day and age?

</comment></comment></comment></comment><comment><USER47> How will does it run on a raspberry pi?

<comment><USER48> It should run just fine, but I haven't tested it myself. OpenBSD has support for the Raspberry Pi (BCM2837): [https://www.openbsd.org/arm64.html](https://www.openbsd.org/arm64.html)

</comment></comment><comment><USER18> Expose a C program with all the language's subtle undefined states and manual memory management to the big bad web! What could go wrong...

<comment><USER31> Well untrained programmers who have no business writing C apps in the first place, for starters.

Everyone here is completely glossing over this as if it were suggested that this is intended for everybody, for every solution. If you're not extremely confident in your C skills, or have the experience to back it up, _This is NOT for you!_ If you think you can only do it with another language, then _This is NOT for you!_ If you think you're gonna write the next Amazon, _This is NOT for you!_

And lets not forgot just how many things are written in C and still _running the internet_. It's just another tool in your toolbox.

<comment><USER49> Linus Torvalds is "extremely confident in his C skills". Doesn't stop hundreds of CVEs per year targeted at the kernel.

Sick of the C Apologism Task Force's idea that "if only rockstar programmers wrote code we wouldn't have any problems", completely ignoring the reality of software development and the evidence of 50 years of exploits.

<comment><USER31> 50 years of the same bugs over and over - the language might let you shoot yourself, but that doesn't abdicate the programmer from continuing education, code review, or testing. Seacord has a whole book on avoiding these; arguably anyone without it isn't doing all they can to write safe code, and might as well be considered uneducated. You can write unsafe code in any language if you don't know what's going on, or not a 'rockstar'

As for Linus, the kernel was written when he was a student, and I doubt he personally still makes the same low hanging fruit vulnerabilities that are the big issue.

</comment></comment></comment></comment><comment><USER50> Any example site with huge traffic, or is it just a "joke" project?</comment>


To protect itself from attack, Estonia is finding ways to back up its data - <AUTHOR> http://www.economist.com/news/technology-quarterly/21645505-protect-itself-attack-estonia-finding-ways-back-up-its-data-how?fsrc=scn/tw/te/pe/ed/howtobackupacountry

<comment><USER2> Does anyone have more knowledge about how they use the digital identities in Estonia? It sounds like everybody has a personal key that they use for signing stuff. Is that correct? How is it distributed? If so, that's the most fascinating part of the article to me.

<comment><USER3> Please see: [http://www.id.ee/index.php?id=30470](http://www.id.ee/index.php?id=30470)

Here's the technical architecture doc: [http://open- eid.github.io/](http://open-eid.github.io/)

We all have either a credit-card sized chip card with our keys or a similar mobile SIM card which uses SIM applications. Using those we can do basically everything online, be it authentication to private or public services (using PIN1), or digital signatures (using PIN2).

This is also available to anyone in the world, so feel free to apply for your e-resident ID card here: [https://e-estonia.com/e-residents/about/](https://e-estonia.com/e-residents/about/)

<comment><USER4> Is there much concern in Estonia about the potential privacy leaks? We also now have digital ID cards here in Portugal, but its use is for now fairly limited - you can use them to log in to some governmental sites (like the IRS), but no banks or other sites have adopted them.

I read on Wikipedia that some of your newspapers have started requiring it to comment on their articles, and I'd be wary of such developments, since they're so easy to justify (even to oneself) and to accept (since it makes authentication even easier than remembering password).

How do you deal with the fact that most sites would like to have your unique, cross-site identifiable key fingerprint?

<comment><USER5> [http://en.wikipedia.org/wiki/Estonian_ID_card#Uses_for_ident...](http://en.wikipedia.org/wiki/Estonian_ID_card#Uses_for_identification)

Apparently it is not used anymore?

Web commentary columns of some Estonian newspapers, most notably Eesti Päevaleht, used to support ID-card based authentication for comments. This approach caused some controversy in the internet community.[3]

<comment><USER3> People are too lazy to use ID cards for something like commenting. More importantly though, most Estonian comments are anonymous with people attacking each other or politicians or whoever they like. Nobody wants to comment using their real identity.

</comment></comment></comment></comment></comment><comment><USER6> This is a fascinating article. Are there any other countries that are so open with their failover strategies for critical national infrastructure?

</comment><comment><USER7> Instead of government-in-exile, we can now have roaming governments, with no physical land, but with laws and services operating within a framework. It is government without a country and it puts everyone on equal footing when trading and building a reputation. Do we even need nation states any more?

Perhaps we will see the first virtual governments soon, existing solely as an interface to facilitate identification, arbitration and trust. It could be done over the internet and be a real example of virtual democracy. Looking forward to it.

<comment><USER8> I have always thought that we'll have some internet courts of arbitration that would cut through the red tape of dealing with many jurisdictions. But can we have sovereign states that exist only in the internet? I doubt - maybe there will be some ingenious crypto systems that would make the systems more and more independent form the physical servers (in a way like the bitcoin system) - but eventually still the physical states will always be able to switch off the virtual states if they wanted to and if they cooperated.

<comment><USER9> Here you go: [https://en.wikipedia.org/wiki/Wirtland_%28micronation%29](https://en.wikipedia.org/wiki/Wirtland_%28micronation%29) [http://wirtland.com/](http://wirtland.com/)

<comment><USER8> The crucial question here is if the virtual state can do anything that would be in conflict with the state that has jurisdiction over its servers. I think that with some advanced cryptography this can work to some extent (with servers behind tor and distributed in a manner that there is no one god-admin) - but in the end [http://aeon.co/magazine/technology/on-the-high-seas-of- the-h...](http://aeon.co/magazine/technology/on-the-high-seas-of-the-hidden- internet/)

</comment></comment></comment></comment><comment><USER10> "We will start issuing the cards by the end of 2014." \- well, they're late.

Also, why can't we use an Estonian embassy anywhere in the world?

<comment><USER11> Probably because the embassies are not yet equipped to issue them yet.

Here's how you apply: [https://e-estonia.com/e-residents/apply-2/](https://e-estonia.com/e-residents/apply-2/)</comment></comment>


Editing my blog's HTTP headers with Cloudflare Workers - <AUTHOR> https://jvns.ca/blog/2018/09/03/editing-my-blog-s-http-headers-with-cloudflare-workers/ <comment><USER2> Under "things I tried", she lists:

> clearing my Cloudflare cache lots of times (this would temporarily fix the > problem, but it would just crop up again later)

> upgrading to a new 'realm' on my webhost, in the hopes that there was a bad > Apache server or something that I could move away from

> Making sure <!DOCTYPE html> was at the beginning of all my HTML in case that > helped browsers figure it out it was HTML (it didn't)

> Switching away from nearlyfreespeech's "free beta bandwidth" program

> emailing Cloudflare's support to see if they knew anything about this

> making a lot of curl requests to my webhost directly to see if I could > reproduce it (I couldn't)

The last point in this list makes it seem like the issue could probably be with Cloudflare. Considering that she spends an additional $5 per month for the Cloudflare Workers solution now, a similar thing I would've tried would be to turn off Cloudflare DNS/caching on the site for a few weeks (or longer) and observe (this can be done by turning off caching on the Cloudflare console or by changing the DNS servers on NearlyFreeSpeech.net from Cloudflare to what NFSN provides). This would add a minor cost every month, though a lot lesser than $5, IMO. If the problem shows up, then it's certainly something on NFSN. If not, then it's certainly Cloudflare.

<comment><USER3> It could be that some particular request header (maybe an unusual user-agent) causes the web server to fail to send a content-type, for some reason. But if the response was nevertheless marked cacheable, then Cloudflare may have cached it and served it to other users who didn't send the unusual header. This problem would seem to go away if you disabled Cloudflare caching, but would still technically be a problem with the web server, not Cloudflare...

When thinking about cost, don't forget to factor in developer time. As a rule of thumb, your time as a developer is worth $1 per minute.

(Disclosure: I'm the tech lead of Cloudflare Workers.)

<comment><USER4> You need to set "Vary: User-Agent" when the response varies. It's very important.

<comment><USER5> Cloudflare CDN does not use the `Vary` header and only caches by the URL, unless you have an enterprise plan.

<comment><USER4> Good to know. That breaks a lot of stuff. Good thing we always had the enterprise plan at work.

<comment><USER3> `Vary` is tricky because it potentially forces the cache to do many more disk reads per lookup, hurting performance. Without `Vary`, there's at most one cache entry per cache key, and logic is simple: load that entry, verify it hasn't expired, and then return. With `Vary`, there could be many entries matching any cache key, and you can't even compute a hash bucket upfront because you have to examine the `Vary` header on the cached response to find out which headers matter. You either have to do a linear scan of all entries for that URL, or you need to maintain some sort of fancy index, and even then it's going to be slower than without `Vary`.

FWIW, with Cloudflare Workers you can implement logic that is (almost) strictly superior to Vary, and doesn't have these performance challenges. Instead of varying on a header, you can write code to vary on arbitrary properties of the request. For example, `Vary: User-Agent` would normally partition the cache not only for every browser, but for every minor version of every browser and every OS version, ruining your cache hit rate. With a Worker, you can parse the User-Agent and decide the buckets in your own way in code, then compute a custom cache key (or just add a query parameter to the URL) based on that. (Workers run "in front of" cache, so you get to modify the request before cache lookup occurs.)

The only down side compared to the header is that your code has to know what to vary on at request time, whereas the `Vary` header can be determined at response time. But if you're writing code specific to some application or web site, usually this is not that hard to determine based on URL alone.

(Disclosure: I'm the tech lead of Cloudflare Workers.)

<comment><USER4> It doesn't affect performance of the browser or the web server, but I understand that it takes some work to support for a CDN. Too bad since you run a CDN ;)

I had to debug some pretty nasty issues caused by missing Vary. I would certainly do without it if it were possible, but it's not an option.

Vary is strictly required for anything that renders content per browser (User- Agent), compression (Content-Type Content-Encoding) or CORS policies (Origin).

<comment><USER5> Cloudflare is/was built on nginx so Vary is already supported and available for enterprise plans as stated. They just avoid it since they have such a large free tier of users and vary headers can easily eat up cache space for most scenarios.

It seems Workers is their standard answer for more flexibility now and it works well (if you're ok with the pricing) since you can create your own cache key easily by combining and hashing the different headers you're interested in and just turning them into a querystring param in the origin request.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I totally understand fixing it this way, but it's of course more satisfying to fix the underlying issue.

I couldn't (easily) find documentation of the webhosts' webserver setup, but I did find that Apache 2.4 deprecated DefaultType and will return pages without a Content-Type header if there are no matching rules. It seems possible that some portion on the original hosts may not be configured the same as everything else, leading to this problem.

Detailed access logs (if available) might help show where the problem request hit and help track it down?

It's probably worth poking at the host's customer support too. They seem pretty competent, and worst case, they say they're not going to look at it, because there's not enough information.

</comment><comment><USER7> This feels like an expensive fix to a problem that seems worth digging into. Is the hosting provider the one not sending the header? Or is it Cloudflare or the web server? Saving 60 dollars a year to find that out seems worth it to me.

<comment><USER4> $5 a month is rather cheap for web hosting.

<comment><USER7> It's $5/month just for the HTTP header interception/injection. The hosting cost isn't listed.

</comment></comment></comment><comment><USER8> That's the most expensive HTTP header I've ever seen.

</comment><comment><USER5> Why not also use the Workers to now inspect the requests and log the ones that are missing the content-type header to see what's happening?

<comment><USER9> She does that, according to the second to last paragraph.

</comment></comment><comment><USER10> Seems like she's spending $60/yr to fix a broken hosting company's problem?

</comment><comment><USER11> You can also customize request or response headers with AWS Lambda @ Edge and it only costs $0.60 per million requests: [https://nvisium.com/resources/blog/2017/08/10/lambda-edge- cl...](https://nvisium.com/resources/blog/2017/08/10/lambda-edge-cloudfront- custom-headers.html)

<comment><USER3> I try to avoid commenting on competitors... but I really have to point out that Amazon's pricing is misleading here.

To use Lambda@Edge, as far as I can tell, you need to pay for:

\- Lambda@Edge base cost: $0.60/M

\- Lambda@Edge CPU/RAM cost: minimum $0.31/M (every request is rounded up to at least 128MB+50ms)

\- CloudFront requests: minimum $1.00/M (assuming HTTPS in US+Canada)

\- CloudFront egress bandwidth: minimum $1.70/M (assuming average 20KB responses in US+Canada)

\- Probably other things, too?

So... $3.61 per million... and probably more due to bandwidth and region.

Cloudflare Workers charges $0.50 per million requests, with a $5 monthly minimum (covering your first 10M requests). There are no other costs: You can use it on top of Cloudflare's free plan, which gives you unlimited bandwidth. And... it performs better: [https://blog.cloudflare.com/serverless- performance-compariso...](https://blog.cloudflare.com/serverless-performance- comparison-workers-lambda/)

(Disclosure: Again, I'm the tech lead of Cloudflare Workers, so take my comments on competitors with appropriate grains of salt...)

</comment></comment><comment><USER12> Wow, this is a bug in Cloudflare that you are paying to work around?

<comment><USER13> I understand she doesn't know for sure, that's just her best guess.

</comment></comment><comment><USER14> I wanted to test if an old school META "http-equiv" directive would be a temporary fix for this issue.. so I tried to recreate the problem by making a one-shot HTTP server with netcat that only returned Content-Length and no Content-Type header:

        { printf 'HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n' "$(wc -c < a.html)"; cat a.html; } | nc -l 8080

Yet Chrome, Firefox and Safari automatically detect the payload is HTML and renders it properly _without_ any Content-Type header, unlike with her pages. What am I missing?

<comment><USER15> Perhaps a X-Content-Type-Options: nosniff header?

<comment><USER14> Aha, that was indeed the one - thanks :) And.. I can confirm meta http-equiv does not help in this scenario at all.

</comment></comment></comment><comment><USER16> If you're serving any user-created content (such as comments or user uploaded images) I would think this would be a bad idea security-wise, because it might lead to XSS.

But if you're just doing author-created content, it should be ok security- wise.

</comment><comment><USER17> I think my light bulb was flickering when she published this.

</comment><comment><USER18> Beware anyone dealing with Cloudflare; If you're locked out of your account for any reason (lose your mobile phone/2FA, or any other reason), you lose your account forever.

There is no way to contact Cloudflare _without_ the account you just lost access to.

<comment><USER19> That's a really good thing, assuming they are upfront with warnings and advice. I wish more service providers behaved similarly, especially amazon, google, and others.

<comment><USER18> It's a good thing to not provide a support contact without a login? What planet are you living on? And you want other tech providers to follow suite and hide their support channels?

<comment><USER20> I specifically use the email host I use because there is no way to recover a lost account. If you lose the password - too bad! The owner does not process password resets and there is no way to recover your account. The host makes this very clear that you are in charge of never misplacing or forgetting your password. It removes an entire attack vector - social engineering a password reset.

It is very much something I wish more companies would do, or at least let users opt into. "Never under any circumstances reset my password no matter how much I might beg you to." Only tangentially related to _support_ as a whole but more directly related to "being able to recover an account".

<comment><USER18> You're missing the point here. Whether or not I can recover my account is tangential to the real problem of them not allowing me to contact Support.

You might be happy with that, but most people aren't. The "contract" is: so long as I can prove who I am to a reasonable degree, give me access to my account.

</comment></comment></comment></comment></comment><comment><USER21> As others have pointed out, this is an odd issue that really shouldn't be happening. Hopefully, since posting, the author has investigated alternative solutions for website hosting as there are many (which are very nice and very free).

I sort of got the impression that this was just a paid promotion for CloudFlare workers. If it wasn't, maybe they could do you a solid and help you identify the actual issue. :)

<comment><AUTHOR> It's not a paid promotion. [https://blog.cloudflare.com/no-payola- here/](https://blog.cloudflare.com/no-payola-here/)</comment></comment>


Show HN: Competitive Crowdfund, Hillary V. Trump for Sky Advertising Over Denver - <AUTHOR> https://www.winulu.com/ <comment><AUTHOR> This one had been percolating in my head for quite a while.</comment>


Nokia Caught Wiretapping Encrypted Traffic From Its Handsets - <AUTHOR> http://falkvinge.net/2013/01/11/death-twitches-nokia-caught-wiretapping-encrypted-traffic-from-its-handsets/

<comment><USER2> Repost: <http://news.ycombinator.com/item?id=5040209> <http://news.ycombinator.com/item?id=5035907></comment>


React Mental Models: Working with Input - <AUTHOR> https://learnreact.design/2020/03/31/react-mental-models-working-with-input <comment><USER2> I've always pointed folks to Gosha Arinich's posts on forms in React [0], as the best resource out there, particularly the post on "controlled vs uncontrolled inputs" [1]. (And if you really want to go in depth, he wrote a book on the topic as well. [2])

If you're looking for libs to help abstract working with more complex forms, Formik [3] and React Final Form [4] are probably your two best options.

[0] [https://goshakkk.name/on-forms-react/](https://goshakkk.name/on-forms- react/)

[1] [https://goshakkk.name/controlled-vs-uncontrolled-inputs- reac...](https://goshakkk.name/controlled-vs-uncontrolled-inputs-react/)

[2] [https://goshakkk.name/the-missing-forms-handbook-of- react/](https://goshakkk.name/the-missing-forms-handbook-of-react/)

[3] [https://jaredpalmer.com/formik/](https://jaredpalmer.com/formik/)

[4] [https://final-form.org/react](https://final-form.org/react)

<comment><USER3> Throwing my thin wrapper around Material-UI and Final Form into the mix. I personally prefer RFF over Formik because of RFF's ability to control renders.

[https://github.com/lookfirst/mui-rff](https://github.com/lookfirst/mui-rff)

</comment></comment><comment><USER4> Why does a site like this need annoying notifications in the bottom left telling me that some random person enrolled in some random thing? Jesus, the modern web is a mess.

<comment><USER5> Conversion. Social proof gets more people to join.

</comment></comment><comment><USER6> might be good if you know jquery and trying to learn react.

<comment><USER7> I know JQuery, love Vue and liked angularjs when it came out. I've played with react application using local state/drilling down only. I found this to be the best tuturial for me introducing setState.. after reading this I'm starting to understand/accept putting the event code directly in the template rather than it belongs somewhere else.

<comment><USER8> There's no templates in React...

<comment><USER9> Blocks of JSX are basically templates, even if they're not string-based like in frameworks.

<comment><USER10> Speaking of mental models, "basically templates" is the wrong mental model for JSX. JSX is syntactic sugar for the React.createElement() function. Writing some components using this function directly will lead to a much better understanding than treating it as templating.

<comment><USER9> I'd argue that not only is JSX "like templates", it actually is a template. I know what the syntax sugar compiles down to, and I'm very appreciative of the benefits over traditional string-based templates. But for me the fundamental property of template is an easy way to interpolate data into a rendering structure, and JSX definitely is that. Perhaps you have a more restrictive idea of what constitutes a template?</comment></comment></comment></comment></comment></comment>


Mock Mock – Who's There? - <AUTHOR> https://dev.to/meeshkan/unmock-vs-nock-comparing-two-js-mocking-frameworks-ii0 <comment><AUTHOR> My team built a new JS framework for mocking REST APIs called "unmock" that focuses on fuzz testing integrations. We hope to move it closer to property- based testing libraries like fast-check. The hard part is building a shrinker - shrinkers ([https://medium.com/criteo-labs/introduction-to-property- base...](https://medium.com/criteo-labs/introduction-to-property-based- testing-f5236229d237)) are tough to get right when testing side effects. Please check out the article, leave your feedback if any, try out unmock ([https://github.com/Meeshkan/unmock-js](https://github.com/Meeshkan/unmock- js)) and let me know what you think!</comment>


I Built a CSS Gradient Animator Tool Using VueJs - <AUTHOR> https://fuze.8bit.codes <comment><USER2> Very aesthetically pleasing, nice job! Hitting 'c' for copy didn't work for me on Chrome@Windows.

<comment><AUTHOR> Thank you and thanks for reporting the issue. I've sentry plugged in, will fix it.</comment></comment>


DeSoto County School District - Gang Policy - <AUTHOR> http://images.pcmac.org/Uploads/DesotoCSD/DesotoCSD/Departments/DocumentsCategories/Documents/jdca-gangpolicy.pdf

<comment><AUTHOR> The back story [https://www.aclu.org/racial-justice/mississippi-school- distr...](https://www.aclu.org/racial-justice/mississippi-school-district- clarify-gang-policy-part-settlement-aclu-lawsuit)</comment>


Using Induction to Design Algorithms (1988) [pdf] - <AUTHOR> http://akira.ruc.dk/~keld/teaching/algoritmedesign_f05/Artikler/05/Manber88.pdf <comment><USER2> Surprised I didn't know of this paper, this is the paper that led to the book:

Introduction to Algorithms: A Creative Approach by Udi Manber (1989).

This book was awesome, a reviewer on Amazon explains its unique style of teaching better than me:

"By Vinhthuy Phan on July 2, 2003 : Most books I have read on the subject seem to be simply a collection of algorithmic tricks and techniques. There's no single universal idea, except for this book by Udi Manber. The cover picture says it all. In my opinion, the most fundamental and universal concept in algorithmic design is the idea of induction, recursion, and building bigger desired solutions from smaller already-constructed ones. The experts establish this as their intuition and perhaps take it for granted. But as a beginning algorist, when you have to ask yourself how do I solve/optimize/approximate this problem, I think you'll find this idea so important. This book attempts to make this concept your algorithmic intuition, and that's I think is a good thing. That said, it should be read along with other books in algorithms design and analysis."</comment>


Ask HN: Do you have any kind of "strategy" for StackOverflow questions? - <AUTHOR> I wanted to play a bit with SO. Seems like helping people could be a nice way to learn something new about tools I'm using as well as doing something good for the world.
Anyway there are a few problems. Sometimes when I write an answer, with a proper explanation, a solution and usually some links to docs, there suddenly appears a message that there is a new answer to the question. And usually it's as good as mine, so I don't even bother with sending it. It's a bit demotivating.
When I tried to browse questions without answers that were asked longer than 30 minutes ago there are usually just a bunch of badly formulated questions.
Do you have any tips how to effectievely browse StackOverflow? <comment><USER2> I no longer contribute, but when I did my most upvoted answers were the ones I wrote on old and very popular questions. So one strategy is to order questions by the number of votes, go through them and see if you have an alternative answer that is not yet posted on the question. Doing this you will not have to rush writing your answer. Your reputation will start increasing at a slower pace, but the increase will be more stable as there are lots of people that get referred to the most upvoted questions over and over again.

For new questions there are two strategies. First is to post a sketch of an answer first, and then edit and edit and edit to add more content to it. This way OP will see the core of it quickly and can upvote and nobody will "scoop" you on it. Second strategy is to write detailed and elaborate answer so that when your answer appears it will outshine the rivals by being the most comprehensive and most deep.

But frankly, nowadays, I feel like 95% of newly posted questions are duplicates and should be closed, yet aren't because even high rep users prefer to post a quick answer for some points, instead of going through the (as of yet unrewarded) effort to close the answer properly.

Also a lot depends on the tag.

<comment><AUTHOR> Thanks.

Fun fact: I've just typed a random problem into Duck and selected the first result from StackOverflow. The top answer is at 238 right now. Then I checked it in archive.org at random points at time and it seems that reputation growth was slow, but almost linear. Amazing. And I was wondering why SO posts I stumbled upon had so many upvotes, when the fresh questions and answers rarely have more than just couple of upvotes :)</comment></comment>


Ask YC: What options are available for sharing ad revenue with users? - <AUTHOR>

Is anyone here aware of a way to share a site's ad revenue with a site's users? <comment><USER2> I think for a smaller shop / startup the easiest way to go would be to have users enter their own google adsense client number and use that a certain portion of the time.

By making them sign up for Google you get G to do the footwork of verifying they are eligible, and managing payments...etc

</comment><comment><USER3> Take a look at what Revver has done with user accounts. Or Jellyfish.

There's usually a float involved which can be fun for cash flow (time between affiliate payout >= time your affiliates need to be paid).

Some companies won't let you / have rules about compensating users. For ex: WalMart doesn't like it, on the grounds that they feel their brand is strong enough to attract people without affiliates having to offer a monetary incentive.

<comment><USER4> Float is good. Also usually the deal is "Minimum payout $20" for example.

Reward sites actually make most of their money by relying on the fact that most people earn a few rewards, give up, and never claim because they don't get to the minimum payout threshold.

Not a great business, but that's how that sort of thing works...

</comment></comment><comment><USER5> Well, just give it to them.

Cut checks, use PayPal, or Amazon FPS.

Accounting would be your biggest hassle. You need to file a separate 1099 with IRS for each person you pay more than $650.

<comment><USER6> But how do you easily measure what pages the clicks are coming from?

Oh and if a ton of people were making me 1200 each (@50%) then I wouldn't mind filling out 1099s all day:)

</comment></comment><comment><AUTHOR> Thanks for the feedback everyone - some nice leads.</comment>


Bitfinex's get out of jail token is stuck in limbo - <AUTHOR> https://decrypt.co/7897/a-look-at-leo-a-week-after-the-nyag-allegations <comment><USER2> Prediction: LEO collapses, Tether and Bitfinex follow suit, and the entire Bitcoin market is revealed to have been a thin Ponzi scheme built on flimsy lies and bad debt. The price goes to ZERO, and no amount of unassailable cryptographic security will protect Bitcoiners' assets as the entirety of BTC's value flees the market.</comment>


New Snowden revelation shows Skype may be privacy's biggest enemy (2013) - <AUTHOR> http://www.computerworld.com/article/2474090/data-privacy/new-snowden-revelation-shows-skype-may-be-privacy-s-biggest-enemy.html <comment><USER2> relevant: [http://www.zdnet.com/article/skype-ditched-peer-to-peer- supe...](http://www.zdnet.com/article/skype-ditched-peer-to-peer-supernodes- for-scalability-not-surveillance/)

Skype ditched peer-to-peer supernodes for scalability, __not surveillance __

</comment><comment><USER3> Article is over 3 years old</comment>


Tell HN: Android Developer Console now equipped with analytics - <AUTHOR>

Today when I logged into my android developer console under the download statistics for each of my apps I had a Statistics link.
The following statistics are available for an app: App install breakdown by:
* OS version
* Device
* Country
* Language
On the side it also shows similar global market statistics.
Besides it also shows how the number of downloads increased over time.
All of this is shown using Google analytics style pretty graphs and charts. <comment><USER2> This is a welcome improvement! Much more detailed than the spreadsheet I've been keeping of downloads over time.

I did notice that the data they're providing is only going back until December. Are other people noticing this too?

</comment><comment><USER3> Finally! Very welcome and highly wished for.

Interestingly there seems to be an error: a spike on Jan 1st to almost double the regular installs, getting back to the normal number on Jan 2nd (confirmed on a number of apps).</comment>


Ask HN: When will YC publicly announce which companies got in Summer '16 class? - <AUTHOR> <comment><AUTHOR> @justinkan?</comment>


As Robots Grow Smarter, American Workers Struggle to Keep Up - <AUTHOR> http://www.nytimes.com/2014/12/16/upshot/as-robots-grow-smarter-american-workers-struggle-to-keep-up.html

<comment><USER2> There's a strong possibility that technology in the future could create a massive wave of unemployment - as people are replaced by smart machines.

Of course, life could still be better overall as these machines could provide for all the necessities for the unemployed at a low cost. So the unemployed would be more like retired people.

The big question is whether people will merge with machines or whether they will be distinct. If they don't merge, the replacement issue is much more likely (with high unemployment). If there's a singularity a la Kurzweil, then the problem is moot.</comment>


A Cheap, Thin Film Gives Portable Night Vision to Cell Phones and Eyeglasses - <AUTHOR> http://www.popsci.com/technology/article/2010-04/tapping-oled-tech-cheap-thin-film-gives-night-vision-cell-phones-eyeglasses

<comment><USER2> 'sucks a lot electricity, thousand of volts'... And that magazine called __* Science??? Electricity, like any energy, is measured in Watts. Volts are used to measure voltage, not energy.</comment>


Are there any location based communities for programmers? - <AUTHOR>

For example I'm planning to move to a new city and I want to be able to find answers to my questions, learn what is going on there by browsing on regular basis, get a feed of what software engineers are talking about in that city. Is there such a website?<p>Basically, I'm looking a site to learn what's software engineers in [San Francisco] talking about. Significant news, tips, advice, discussions on anything. <comment><USER2> Anytime I move I check out meetup.com You can find a community for nearly any interest under the Sun 5 to x miles from your locale. And, if there isn't, you can create one. In my experience, there is typically a Code for America division or some other general meetup group (Python enthusiasts, Node.js beer night, etc). Your mileage may differ based on the size of the city you're in.

</comment><comment><USER3> Tech is tech. Code is code.

I'm sure if we have people on Mars they'll be talking about the same news/tips/advice/discussions we are.</comment>


This $1,500 Toaster Oven Is Everything That's Wrong with Silicon Valley Design - <AUTHOR> https://www.fastcodesign.com/3065667/this-1500-toaster-oven-is-everything-thats-wrong-with-silicon-valley-design <comment><USER2> The $1500 price tag alone makes it difficult for anyone to really be affected by their objective of 'getting more people to cook.'

Who doesn't cook? Millennials. [1]

Who doesn't have money for a toaster oven that costs twice their monthly rent? You guessed it.

The author nailed it here - '...yet, June is taking something important away from the cooking process: the home cook's ability to observe and learn.

The sizzle of a steak on a pan will tell you if it's hot enough.

The smell will tell you when it starts to brown.

These are soft skills that we gain through practice over time. June eliminates this self-education.'

If we want more people to cook, we should give them solid reasons to. Automating the process doesn't teach people to 'cook', it teaches them to be yet more reliant on a piece of technology.

[1] [http://www.theatlantic.com/business/archive/2016/11/millenni...](http://www.theatlantic.com/business/archive/2016/11/millennials- groceries/506180/?utm_source=quartzfb&single_page=true)

<comment><USER3> _Automating the process doesn 't teach people to 'cook', it teaches them to be yet more reliant on a piece of technology._

Precisely the direction SV wants us to keep marching blindly in.

</comment></comment><comment><USER4> I kind of agree with the author, but that ship sailed a long time ago. It's not coming back.

This reminds me of the (Hitachi?) thousand dollar rice cooker.

I guess you can argue for balance between craft and automation, but it's rather futile and more a nice mental exercise.

It's like trying to convince most kids to learn how to derive square roots longhand when they have a computing device in their pockets.

<comment><USER5> Which ship sailed? The one with all the people cooking? or another one.

I wouldn't count cooking as 'out' just yet. Many people of all ages love to cook, but many of them love the creation of it. An oven that cooks for you is like paint by numbers, it 'might' get people interested in cooking to the point where they want to learn more.

<comment><USER4> The ship that sailed is the one that says we must keep traditional systems alive and well in the modern world.

Yes, some people live cooking --as do I when I have the spare time, on the other hand, often I don't have the time to futz and just need a pragmatic meal, rather than experience based cooking.

</comment></comment></comment><comment><USER6> The first thing I thought of when I saw the headline: [http://www.ee.ryerson.ca/~elf/hack/ktoast.html](http://www.ee.ryerson.ca/~elf/hack/ktoast.html)

<comment><USER7> It seems like everyone's switched to 8-bit microcontrollers now, but pretty much accurate.

That being said, there is room for a smartphone-controlled toaster / oven; but it should give more control (customizable & shareable heating profiles, as part of a recipe database) rather than less.

</comment></comment><comment><USER8> Still 1800W, so it'll still take 5min to make a piece of toast.

"Carbon fiber heating elements ... preheats faster" Total B.S.

Much better to install a real outlet and get something like [https://www.katom.com/569-FC33.html](https://www.katom.com/569-FC33.html)

</comment><comment><USER9> The only lasting complaint here is "the oven does things for you, which means you won't learn to do things yourself." Which is ridiculous, because that's the point of automation. Every other issue (bugs, price) can be chalked up to the early adopter principle.

The other fallacy is that everyone wants or needs to learn how to cook. I enjoy cooking, but sometimes I also enjoy dumping ingredients into a rice cooker and firing and forgetting.

I think the thermal sensor is a very clever bit of tech. I'm guessing the estimates won't be perfect because the oven can't gauge thickness, and it refines estimates once it measures the rate of heating.

> The salmon's done at 6:52 p.m.

Which is meaningless, because this is the first time the article's mentioned the time.

</comment><comment> alex- I do LOVE that tag line

> Automated yet distracting. Boastful yet mediocre. Confident yet wrong.</comment>


Tugboat: A command line client for DigitalOcean - <AUTHOR> https://github.com/pearkes/tugboat

<comment><USER2> This looks like another great tool.

Like many people considering using DigitalOcean, I did some research over the weekend on "droplet" management. My results came up with the following resources (many already shared on hacker news):

1) Tugboat (today)

2) Fog - <https://github.com/fog/fog> \- digital ocean support just recently added (<https://github.com/fog/fog/pull/1525>)

3) Vagrant DigitalOcean - <https://github.com/smdahlen/vagrant-digitalocean>

Both #2 and #3 are well established technologies that are allowing DigitalOcean plugins whereas TugBoat seems to be a specific tool for DigitalOcean.

Could someone with more extensive sysops experience comment on when/why you'd use one of these tools over another? (particularly relating to system provisioning options/interplay after droplet management...)

<comment><USER3> There is also a Knife plugin with Solo support: <https://github.com/rmoriz/knife-digital_ocean>

</comment></comment><comment><USER4> Awesome. Does DO not yet have a comprehensive command line tool?

I'm curious if any Hacker News readers have successfully migrated from Engineyard/Heroku to Digital Ocean. If so, I'd love to hear about your experiences. I am specifically curious about:

1) How complex/big was your Heroku/EY deployment 2) Time invested for migration 3) Stability since migration 4) Ongoing maintenance commitment 5) If you could turn back time and invest this investment elsewhere, would you?

I love Heroku, but I'm a bit concerned that we're becoming too interwoven/dependent on them. We currently have an issue with the stability of one of the Heroku 3rd-party redis tools timing out and feel a bit trapped... I would love to hear an analysis of others' migration experiences.

<comment><USER5> There was write-up from another customer who migrated from Heroku to DigitalOcean

[http://matteodepalo.github.io/blog/2013/03/07/how-i- migrated...](http://matteodepalo.github.io/blog/2013/03/07/how-i-migrated- from-heroku-to-digital-ocean-with-chef-and-capistrano/)

<comment><USER4> This is very helpful from a technical angle. I'm interested in the full cost- benefit analysis. Any thoughts there?

</comment></comment></comment><comment><USER6> Very cool. It would be super useful if you could disable fuzzy matching, disable confirmations, and return proper exit codes like a standard CLI utility.

<comment><AUTHOR> (author here)

Thank you. Yes, agree on all points. In the issue tracker.

</comment></comment><comment><USER7> Nice simple toolset. It's nice to see a script that knows what it is and does that well. Great name for the tool as well.

</comment><comment><USER8> This is really great! However, I really wish it had support for allowing you to specify arbitrary droplet sizes and custom images without having to somehow look up the numeric ID of both.

<comment><AUTHOR> I agree. I think that's an improvement that needs to be made on DO's side, though. I could certainly "map" them to human names on my side, but that's not going to be consistent, IMO.

I've sent them this feedback. :)

</comment></comment><comment><USER9> Simple Ruby onefile Digital Ocean CLI for ~/bin dir <https://gist.github.com/shtirlic/5390788></comment>


Why I'm not going back to A16Z - <AUTHOR> https://www.linkedin.com/pulse/why-im-going-back-a16z-haiqun-léo-wang <comment><USER2> It appears I have to be logged into linkedin to view this.

<comment><USER3> Don't worry. Articles people post on LinkedIn rarely are worth reading.

<comment><USER2> Yeah, figures. If it walls itself off, it is likely rubbish.</comment></comment></comment>


Chemistry of Cast Iron Seasoning: A Science-Based How-To - <AUTHOR> http://sherylcanter.com/wordpress/2010/01/a-science-based-technique-for-seasoning-cast-iron/ <comment><USER2> I actually like using the same oil that I cook with because sometimes I can taste the seasoning and I don't want the taste to differ.

<comment><AUTHOR> That's the pragmatic answer for sure</comment></comment>


Show HN: Open up Twitter Lists to self-enrollment with Crowdlistr - <AUTHOR> http://crowdlistr.com/

<comment><AUTHOR> This app was inspired by some work I did for a non-profit conference that was looking to let its attendees add themselves to Twitter lists, rather than rely on them to remember to use the correct hashtag. It was a long weekend of work, but it's gotten some decent use.

The premise is simple: The app allows you to create a simple portal where users can add themselves to Twitter lists that you designate 'Open'.

So, for the account @Crowdlistr and the twitter list @sample, people can add themselves from here: <http://crowdlistr.com/crowdlistr/sample>

Let me know if you have any questions or suggestions (beyond telling me that the design sucks, because, well, the design sucks.)</comment>


Large-scale cluster management at Google with Borg - <AUTHOR> http://research.google.com/pubs/pub43438.html <comment><USER2> Oh, fun, they finally published this. How many times I've had to catch myself from saying the word "Borg" I don't even know.

The performance isolation is good, but I wouldn't really seek after high utilization unless compute costs are significant to your business. We've seen some crazy things where nominally non-interfering jobs cause significant performance degradation to other jobs on the same node. There's work yet to do here.

<comment><USER3> Why is everyone acting like the name was some big secret? Everyone who has been using Mesos knows Borg, for crying out loud, big writeups were published in Wired and The Verge over 2 years ago!

<comment><USER4> It was a secret. You weren't allowed to mention borg outside Google. Note John Wilkes was generally careful about not saying the name was 'Borg' or acknowledging it as such and instead referred to 'Omega'.

<comment><USER3> Perhaps you were told it was secret. It was not secret. Here's one of the articles I was referring to. [http://www.wired.com/2013/03/google-borg- twitter-mesos/](http://www.wired.com/2013/03/google-borg-twitter-mesos/) John Wilkes mentions Borg, acts coy about the name for some reason (seems like a pattern), and mentions Omega, its nascent replacement.

<comment><USER4> John does not mention Borg by name publicly (well, until the paper was published). That's what I meant (consistent with what I was replying to). The article you linked to confirms that.

"According to Wilkes, Google plans to publish a research paper on Borg (though he still won't use the name). "

Wilkes won't even call it Borg. "I prefer to call it the system that will not be named,"

(I work with John and have contributed to Borg)

</comment></comment></comment></comment></comment><comment><USER5> All of us who have used Borg over the years are very appreciative of the technology and its capabilities. Congrats to all the Googlers, current and ex, that contributed to it. This paper should be toward the top of the reading list for anyone working on the topic.

Nevertheless, there are many open questions for large-scale cluster management for researchers and developer to address. Here are some of my favorite: \- The curse of overprovisioning: Borg and many other systems rely on reservation which are systematically exaggerated by users. Right sizing these reservations is one way to go beyond the 40-50% usage shown in the Borg paper (see fig 12). A promising way of doing this is Christina Delimitrou's work using classification techniques (see [http://goo.gl/vFf8oN](http://goo.gl/vFf8oN)) \- Oversubscription using better isolation mechanisms): this is what the Borg paper calls resource reclamation. Take unused (but reserved) resources from priority jobs and use them for best effort analytics. David Lo ([http://web.stanford.edu/~davidlo/](http://web.stanford.edu/~davidlo/)) has a very interesting paper coming up on how to coordinate cpu sets, cache partitioning, Linux TC, RAPL/DVFS (power management) to run websearch clusters at >90% by packing them with analytics without causing ANY glitch on search. And that is Google search.

There are definitely more interesting. Exciting times.

<comment><USER6> > [http://goo.gl/vFf8oN](http://goo.gl/vFf8oN)

goes to: [http://web.stanford.edu/~cdel/2014.asplos.quasar.pdf](http://web.stanford.edu/~cdel/2014.asplos.quasar.pdf)

It's my understanding that URL shorteners are frowned upon in HN posts or comments.

<comment><USER5> I learn something every day :)

</comment></comment></comment><comment><USER7> There was a great talk by John Wilkes (Google Cluster Management) re: Omega in 2011 at Google Faculty Summit [1]. Absolutely fascinating to see the scope of the problems they are dealing with.

[1] [https://www.youtube.com/watch?v=0ZFMlO98Jkc](https://www.youtube.com/watch?v=0ZFMlO98Jkc)

Edit: remove error in my comment re: borg/omega order.

<comment><USER8> Omega actually comes after Borg (hence the name). It's just that Borg was quite confidential until now. I'm pretty surprised they let the name slip.

<comment><USER9> The name was public since Omega was publicized. The paper itself is great, there's everything about Borg in one place (scheduling, quota, isolation, etc).

</comment></comment></comment><comment><USER10> Was always in awe of Borg and Omega while at Google. Really nice to see them finally publish a paper on this. Guess they've more far enough along now that it makes sense to do so. Omega will be a far superior beast to Borg and the open source Kubernetes but I have high hopes for the future of Kubernetes.

</comment><comment><USER11> Apache Yarn[1] looks like same thing as borg\omega. A plus point with Yarn is we can get our hands on it.

[1] [http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop- yar...](http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn- site/YARN.html)

<comment><USER12> Yarn tries to do the same thing, but is quite different. It's skewed toward running short running jobs, like batch jobs, it was meant for hadoop and co after all. Borg/Omega seems more like a combination of Mesos at the scheduler layer and Marathon/Kubernetes on top. It's funny to see how many services they run on top of it though.

<comment><USER13> This is exactly right. Marathon or Kubernetes running on Mesos or Mesosphere's DCOS is functionally similar to Borg.

</comment></comment></comment><comment><USER14> > cc would be reachable via 50.jfoo.ubar.cc.borg.google.com.

I've implemented similar at my current job, as that sort of naming is very convenient. [http://www.boxever.com/using-google-apps-openid-connect- with...](http://www.boxever.com/using-google-apps-openid-connect-with-apache) has a sketch of how to do this with Apache as a reverse proxy with Google Auth, though we're using a PAC file now going to a HTTPS forward proxy to avoid limitations of SSL wildcard certs.

<comment><USER4> Keep it up, Brian :-)

<comment><USER14> If that was the craziest thing I was doing, I'd be doing well :)

</comment></comment></comment><comment><USER15> Nice writeup - I just saved the PDF in my keep for ever PDF repo.

When I contracted at Google at 2013 I loved their infrastructure. For my task I had to run huge Borg jobs and the job submission, monitoring and logging system were very easy to use. I really liked the summary of hardware failures that occurred - hardware really is not very reliable when running at scale.

After not using AppEngine for a few years I have started using it recently for two personal projects. Using AppEngine's logging and scaling features is a tiny bit like using Google's internal infrastructure - makes me a little nostalgic.

</comment><comment><USER16> The Register [1] for years has referred to Cisco as the Borg because of their acquisition strategy.

The title implied to me that Google and Cisco were working together.

[1] [http://www.theregister.co.uk/2015/04/16/borg_routers_open_to...](http://www.theregister.co.uk/2015/04/16/borg_routers_open_to_repeat_remote_dos_attack/)

</comment><comment><USER17> How does Borg compare with Google's Kubernetes?

<comment><USER9> There's a whole section about it in the paper (lessons learned and future work).

</comment></comment><comment><USER18> Is this similar in concept to Oracle Grid Engine (SGE)? How is it different, superior?

<comment><USER19> I started doing SGE stuff in ~2001...It still amazes me how well that thing worked and how often a lot of it's features are reinvented.

<comment><USER18> I agree. It's really simple to get started (short learning curve) and it's completely language agnostic.

I'm amazed how often it gets ignored. There's even StarCluster so you can automatically set up a cluster on EC2.

</comment></comment></comment><comment><USER20> I wonder if disruptive jobs are named Hugh?

</comment><comment> mkr-hn You will be scalesimilated.</comment>


Conrad Barski's Haskell Tutorial - <AUTHOR> http://lisperati.com/haskell/ <comment><USER2> Wow, how did my Haskell tutorial get to the HN front page out of nowhere? :-)

If you like my stuff, keep an eye out for my new Bitcoin book, which will be coming out this fall. I've been slaving over the artwork for months, it should be a beautiful book! The working title is "Bitcoins for the Befuddled".

<comment><USER3> Thanks for the wonderful Land of Lisp book!

I only wish someone would do a similar "Land of C" book some day :-)

<comment><USER4> I haven't read that one, and I am looking for an into to Lisp book. What did you like about it?

(and I think it would be a missed opportunity to not title it "Sea of C")

<comment><USER3> To be clear, I haven't finished the book yet, but I made it through the first eleven chapters (~230 pages) so I think I've gone through enough material to render a reasonable opinion. With that disclaimer out of the way...

I love that the author chose to expose readers to core, foundational concepts through example programs that were continually improved instead of using lots of throwaway one-off examples.

Put differently, the author managed to ensure that readers that diligently followed through from beginning to end would gain a real sense of progression as they went through the material.

The only complaint I really have so far is Chapter 11 -- but that's not really the author's fault; that's just the terrifyingly mind-numbing formatting functionality (Common?) Lisp offers.

</comment></comment></comment></comment><comment><USER5> > _In Haskell, this type of integer can 't get bigger than 2^31..._

It's platform-dependent

      h> finiteBitSize (0 :: Int)
      64

> _We could have used the type Integer instead- This type of integer grows > "magically" and can hold any sized integer, no matter how big... but it's > slower, so we used Int instead, since our numbers will never get that big._

Premature optimisation at its best.

<comment><USER6> In Haskell you can also use the Integral typeclass to make functions polymorphic to Int and Integer.

<comment><USER5> And why would you do this in simple tutorial program? Anyway, you'd need Num.

<comment><USER7> I don't know why anyone would use the Integral typeclass over just Int or Integer in a simple program like this, but you wouldn't need Num in addition to Integral, every instance of Integral must also be an instance of Real, and instances of Real must be instances of Num.

</comment></comment></comment></comment><comment><USER8> Picnicmob was a really clever idea. I think I remember that it got cancelled though, which was unfortunate.

I like the tutorial though. I'd be willing to bet it's his style and work that inspired the "Learn you a..." books. The Haskell one is great, and so is the Erlang one.

<comment><USER9> don't forget _why's (poignant) guide to ruby [http://mislav.uniqpath.com/poignant- guide/](http://mislav.uniqpath.com/poignant-guide/)

<comment><USER4> I hated that book, the signal to goofy joke ratio was awful. Compare with, say, "Learn you an Erlang", which was goofy while still informative.

</comment></comment></comment><comment><USER10> My local Barnes and Noble has this book next to Felleisen et al's "Realm of Racket". These will be hours and cups of coffee well spent!

[http://realmofracket.com/](http://realmofracket.com/)

</comment><comment><USER11> Simple and refined, guaranteed to blow your mind...

The tao of regular expressions link is broken in the first chapter. Does anyone have a link?

</comment><comment><USER12> I like the tutorial very much! Thank you! And I like the final words, which I think brings to the point the whole discussion about Haskell vs. Imperative languages:

      [..] Because the physical world changes only slightly
      from moment to moment, it means that languages that
      can comfortably mutate large data structures in targeted
      ways will always have a role to play in real-world
      software- The "real world" usually just doesn't work the
      way Haskell, and other functional languages would prefer
      it did: Haskell preferred that at every moment in time, a
      "new universe" would look at the "old universe" and would
      rebuild itself, from scratch, from what it saw in the
      past, with radical changes happening all the time.

      Despite its many advantages, I humbly suggest, therefore,
      that in the future there will continue to be a rift
      between the "imperative" and "functional" camps of
      programming, until someone comes up with a truly robust
      way of uniting these two camps- And I think that some
      profound programming discoveries still need to be made in
      the future before this problem is really resolved- I get
      the feeling it's just not good enough to wave at the
      problem and say "Monads".

</comment><comment><USER13> ghc 7.6.3 (brew) installed on OS X doesn't like the first code presented (System.Random, Text.Regex and StdGen aren't found).

<comment><USER14> `brew install haskell-platform` should help.

</comment></comment><comment><USER15> I'm unsure which regex package flavor to use... regex-base? regex-compat? regex-pcre? regex-???

<comment><USER16> This confused me when I was learning Haskell as well. The way I understand it is that regex-base defines a base-line interface -- such as the context-aware =~ function -- that all of the other packages implement, which is why they're known as 'backends' (e.g. regex-tdfa, regex-pcre, etc.).

In newer versions of GHC (> 6.6), the backends re-export this interface so that you just need to import the backend. regex-tdfa is the backend that I've used myself, mainly because I saw it used in other packages (Hakyll).

[http://www.haskell.org/haskellwiki/Regular_expressions#regex...](http://www.haskell.org/haskellwiki/Regular_expressions#regex- base)

</comment></comment><comment><USER17> This website was written on a planet where 244 lines of Haskell is less than 100 lines of Haskell.

<comment><USER2> That's why there's a footnote on the front page:

[1] - Lines of active code only, not counting optional function signatures. Intermediate debug output lines excluded. May cause eye irritation if viewed for prolonged periods of time. Speak to your physician to find out if this tutorial is the best treatment option for you.</comment></comment>


Show HN: Extra Strength Responsive Grids - <AUTHOR> http://dfcb.github.com/extra-strength-responsive-grids/

<comment><AUTHOR> I know. Just what the world needs, another responsive grid system. Still, I think it is a little different, and I had a lot of fun doing the project page. Hope at the very least it gives you a light chuckle.

<comment><USER2> Do you have a solution for setting a footer down and getting it to stay down in desktop mode?

<comment><USER3> You mean like the Sticky Footer example from bootstrap ([http://twitter.github.com/bootstrap/examples/sticky- footer.h...](http://twitter.github.com/bootstrap/examples/sticky- footer.html))?

It's actually not part of boostrap.

</comment></comment></comment><comment><USER4> There's some nice effects in there. It would probably help on the uptake if you showed how to integrate it with Bootstrap (E.g. replace the Bootstrap grid with yours)

</comment><comment><USER5> I really like equalize.js. In the past I've used min-height in CSS to make the heights of containers equal.

<comment><USER2> Interesting. Not to go completely off-topic here, but does equalize.js work equally well with zepto in your experience?

<comment><USER5> I'm sure it would.</comment></comment></comment>


Are / how much are unregulated bitcoins susceptible to tulip mania? - <AUTHOR>

<comment><USER2> <http://www.google.com/search?q=is+bitcoin+a+bubble></comment>


Heroku for Science - <AUTHOR> http://dennyluan.tumblr.com/ <comment><USER2> I studied biomedical engineering at Hopkins. Before I started there, research was the promised land. I dreamt of spending my time thinking about how to solve critical problems and testing solutions.

What I saw instead were people spending the vast majority of their time pipetting. All the way up the ladder, upto and including postdocs. I sometimes thought our PI had it worse for having to spend most of her time applying for grants.

The AWSification of synbio research would be a game changer. Some labs at Hopkins have tried to build robots but with limited success. Given how cheap labor is at research institutions competing on price will be incredibly difficult.

<comment><USER3> I also thought research was the promised land. Went to Cornell for undergrad studying biological sciences and was amazed at the research opportunities... but then worked in a lab studying type II diabetes. I pipetted, cleaned beakers, measured out chemicals to prepare solutions, sucked up cell cultures and extracted DNA all day making $8/hour in extreme boredom. There were postdocs with Ph.Ds and loads of experience from prestigious schools doing the same grunt work alongside me, often confessing they were completely miserable and wishing they could start their life over again. They worked 7 days a week and long, long hours each day. I got the hell out of research and am enjoying my life much better now in tech. I also feel like the work I do is a lot more impactful. All of my peers did the same and went to consulting, finance or tech. In my view the basic research field is in total crisis...

<comment><USER4> So depressing. This (and the parent comment) is the reason I didn't go into research -- it's a life of pipetting and manual labour that no-one's interested in automating, either because it's too complex or because labour is so cheap that there's no financial incentive to do so.

I'm happy to leave someone else to do that. I'd rather be in a job I actually enjoy the day-to-day of.

And that's to say nothing of the problems of PhDs: namely that there are ten times more PhD positions than there are postdoc positions. That ten-to-one crunch when it comes to finding a job sure does sound fun...

<comment><USER5> With the utmost respect to you and the post you responded to, research is about answering questions to things you find interesting; for a biologist, pipetting is simply the means you take to get there.

If you want to contribute to Firefox or any other non-trivial open source project, you need to spend time creating a development environment and it likely will take weeks to months before you can make a substantive contribution.

If anyone is reading the comment I'm responding to or its parent comment, keep in mind that the manual labor is in pursuit of a goal.

<comment><USER6> I think your parent is alluding to the fact things could be automated in biological research but aren't because of the disincentives; and that at the same time things in tech are more amenable to automation and often parts of it are indeed automated.

Yes, it's all a means to an end, but how much time one wants to spend in the "means" (which can get extremely repetitive, apparently) is what counts for the parent (I'm supposing).

</comment></comment></comment></comment></comment><comment><USER7> Hi! This is the Yelling Math Fairy and THE WORD EXPONENTIAL DOES NOT MEAN MORE. IT IT IS A MATH WORD. IT MEANS e^kx. IT MEANS THE SOLUTION OF y' = ky. DOES EACH NEW ASSAY DOUBLE THE TOTAL AMOUNT OF WORK? NO IT DOES NOT. THE ADDED OVERHEAD FOR EACH ASSAY IS LINEAR GROWTH. NOT EXPONENTIAL GROWTH. I KNOW IT SEEMS LIKE A LOT TO YOU BUT THAT DOES NOT MAKE IT EXPONENTIAL. EXPONENTIAL IS NOT A SYNONYM FOR BIG AND EXPONENTIAL GROWTH IS NOT A SYNONYM FOR FAST. THANK YOU.

<comment><USER8> Keep fighting the good fight, Eliezer.

To add on: Some words have valuable and specific meanings! We don't have a good substitute word for "exponential" that means the same thing. Please make an effort not to do this!

<comment><AUTHOR> Sorry, but in this case, it's actually N^x. Even running PCR and gel analysis requires steps where the time is dependent on the number of samples (e.g. pipetting, walking to the centrifuge, making multiple gels)

<comment><USER8> Generally speaking, running double the number of samples requires approximately double the amount of work, maybe a little less since you're doing it in bulk.

Are you really saying that going from 10 samples to 11 samples causes a doubling (or 1.2x-ing or tripling or whatever) of the time/work required? That's what exponential means.

</comment></comment></comment></comment><comment><USER9> I believe this startup is as close as it gets (for now!) to what you're describing, [https://www.transcriptic.com/](https://www.transcriptic.com/).

(I don't work there or anything)

<comment><AUTHOR> There's been quite a few efforts in this. See Transcriptic and Emerald Therapeutics ([http://www.emeraldcloudlab.com/](http://www.emeraldcloudlab.com/)), while there's the more traditional suppliers for things like short oligos or expression vectors ([https://www.dna20.com/](https://www.dna20.com/) and [http://www.idt.com/](http://www.idt.com/)).

I think there's also been a lot of independent academic attempts at this (see: [http://klavinslab.org/](http://klavinslab.org/) which is CS/BioE at UWash), but all kind of waded around in the shallow water.

The reason why I think this is compelling because I think almost every synthetic biologist has an _existing_ workflow. It's basically design using some sort of CAD software, order from IDT, receive materials next day, run test by hand, ship to Genewiz for sequencing, etc. That's just one example of a workflow involving 4-5 specialized 'steps'. As the steps get cheaper/faster/better, consolidating and automating this is just a no brainer.

<comment><USER10> Emerald doesn't really exist yet, and I believe they misrepresent their automation. They've posted a very pretty site with a bunch of mockups. They have a set of workflows that work for their internal antiviral research, but "Heroku for Science" is a completely different game.

Transcriptic, on the other hand, started taking orders six months ago and has customers at Stanford, Caltech, Harvard, and more.

<comment><AUTHOR> There are probably others much more familiar with cloud infrastructure who can chime in, but the AWS of science and the Heroku of science are two very different challenges, and I feel the analogies probably cross over pretty well.

Definitely having the infrastructure 'warehouse' layer that Transcriptic is building (with a real API! wow!) will be valuable. And like you hint at, power users won't need hand-holding, but 99% of the market of users will. That's where packaging, ease of use, and limited configuration seem to be the difference maker (Heroku starting exclusively with Rails).

</comment></comment></comment></comment><comment><USER11> The problem with this fantasy is that easily automated and distributed tasks are not the rate-limiting steps in most biomedical research. The hard parts (in addition to designing the right experiments and analyzing data..) are in constructing and validating relevant model systems and doing the specific experiments to address questions of interest.

These are extremely dependent on the question being studied and often are not amenable to automation, and may require very rare, expensive, and difficult- to-handle samples. For example, my collaborators work with transgenic mice that are a model for a particular disease, and these mice have to be bred then aged to 12 weeks until they exhibit the phenotype before we can even start doing an experiment. In another model, they have to do brain surgery on each mouse and then wait several weeks for the phenotype.

The 'easy' parts, such as DNA synthesis and sequencing, are already highly standardized and automated, and there is fierce competition to improve the technology and bring costs down.

<comment><USER12> This is certainly the bottleneck in my research. I ran thousands of core years of computer simulation in the first year of my PhD. I have all the data I need to write a PhD thesis but I'm still years from graduating due to the aforementioned bottlenecks. An arsenal of software I've written in numpy/scipy/pandas saves time, but only goes so far when you're trying to carve out stories from your data to write papers.

</comment></comment><comment><USER13> A big problem is that scientists are traditionally very secretive. This would increase the possibility of leaks, and there'd need to be some way of assuring that the experiment was conducted correctly. Good idea, though.

<comment><USER14> If this was available convenience would trump security for many people.

<comment><USER15> I just have experience working in 3 different labs in Germany and I can tell you: secrecy is the holy grail and just using privat repos (for code not data) on github/sourcefore is forbidden. But this is perhaps just small sample size and a german habit ;)

</comment></comment></comment><comment><USER16> Most worthwhile research is about the mundane. One of the first research projects I did required painstakingly adjusting and modifying conditions to the point that I could actually start collecting data. That process took weeks, but the day it worked was insanely satisfying. In the process I became a master at making small incremental changes, recording them, and learning exactly what didn't work. Years later, as a computational scientist, the process was much the same, except that there were no pipettes and beakers involved.

Any worthwhile work I have ever done has mostly been about grunt work. Along the way there have been cool things (after all Leno made fun of our research [1] once) and insanely fun times. I may not be in research now, but every day I apply the lessons learned from patiently repeating and iterating.

1\. [http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=723226...](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=723226&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel4%2F5858%2F15610%2F00723226.pdf%3Farnumber%3D723226)

</comment><comment><USER17> I've taken a _very tiny_ first step towards something like this for computer graphics and vision: trying to make user studies as easy as possible: [http://www.imcompadre.com](http://www.imcompadre.com) It's not a ready product by any means, but two paper submissions have been made with it so far.

It's a difficult problem to solve, because these pesky researchers are always trying out new things that you didn't anticipate - who would've thought! But still, for the mundane things that can be automated, something like this is definitely the way to go. Of course, as other people here point out, figuring out what to actually test is always the hardest part.

</comment><comment><USER18> In our lab today we are consistently dealing with the opposite problem. The experiments themselves are easy in comparison with the design and analysis.

<comment><AUTHOR> The startup equivalent would be "it's easy to build the app, it's hard to get users". Not all apps can yield users, but lowering the bar to launch will certainly yield more opportunities for successful apps.

<comment><USER18> You are right. I think we would all appreciate making more stuff that we do automatable and outsourceable. I guess where I was trying to go with the comment is that it would be nice if there were similar tools for design and analysis. Oh automated paper writing would also be much appreciated.

<comment><AUTHOR> Experiment is working on that. :)

</comment></comment></comment></comment><comment><USER19> The Center for Open Science poses itself as something similar to what you describe. [http://centerforopenscience.org/](http://centerforopenscience.org/)

I believe they call themselves more of the Github of Science for scientific collaboration. Adding hooks to 'push' the tasks and 'checkout' the findings could be maybe extensible on their platform.

</comment><comment><USER20> Experimenting with a simulation is a great time saver as far as it goes, however, all models are just models, subject to the assumptions that went into them.

There is a great deal we do not know about cellular biology. Any simulation would be a fairly gross approximation. The point of many experiments is to further our understanding of the model of cellular mechanics.

</comment><comment><USER21> From my experience (having done a similar project as an undergrad) is that the first problem is convincing people to switch and take the risk/time to use your new workflow, even if your workflow allows them to continue to use their existing infrastructure / machines.

</comment><comment><USER22> DIYbio has already started on this: Experimental Robot for $4k: [http://www.opentrons.com/](http://www.opentrons.com/)

</comment><comment><USER23> [http://www.synthego.com/](http://www.synthego.com/)

</comment><comment><USER24> There could be some synergies with (yc12) science exchange</comment>


Show HN: A simple Unix cat implementation in various languages - <AUTHOR> https://github.com/rspivak/2x25/blob/master/simplecat.md <comment><USER2> Here is another implementation using Node.js (although it does not explicitly check for EOF):

      require("fs").createReadStream(process.argv[2]).pipe(process.stdout)

Here is a implementation in PostScript (it does explicitly check for EOF, and specifically deals with one byte at a time):

      /o (%stdout) (w) file def
      ARGUMENTS 0 get (r) file
      {
        dup read {
          o exch write
        } {
          quit
        } ifelse
      } bind loop

(I have no GitHub account. But if the owner of that repository want to add, they can add what I have; I post this message (and all of my other messages on Hacker News) to public domain.)

<comment><AUTHOR> Thank you for the implementations! Do you have any suggestions how to test the postscript code on mac/linux? I tried on macos using ghostscript, but I get the following error:

$ gs cat.ps helloworld.txt

Error: /undefined in ARGUMENTS ...

Thanks!

<comment><USER2> To give command-line arguments to a PostScript program in Ghostscript, you must prefix the name of the PostScript file with two minus signs, such as:

      gs -dNODISPLAY -q -- cat.ps helloworld.txt

(The -dNODISPLAY and -q are not needed to get this PostScript program to work, but -dNODISPLAY disables graphical output, and -q avoids Ghostscript's messages getting mixed in with the PostScript program's output. You might also need -dNOSAFER; if you get a invalidaccess error, try that. As far as I know, ARGUMENTS and -- are a feature specific to Ghostscript.)

</comment></comment></comment><comment><AUTHOR> Feel free to add your own implementation. :)

<comment><USER3> FWIW, iter() handles the logic of testing for an explicit EOF marker, so another way to write it is:

        import sys

        with open(sys.argv[1]) as fin:
            for c in iter((lambda: fin.read(1)), ""):
                sys.stdout.write(c)

<comment><AUTHOR> That's a good point. Thanks!</comment></comment></comment>