Unikernels: No Longer an Academic Exercise? - <AUTHOR> http://250bpm.com/blog:138 <comment><USER2> I'd like to point out that there is a unikernel called IBM's TPF which is something everybody is interacting with daily. It's used in the payment card network by Visa, as well as the airline and hotel industry for reservations. This is typically run under the hypervisors z/VM or PRSM. [https://en.wikipedia.org/wiki/Transaction_Processing_Facilit...](https://en.wikipedia.org/wiki/Transaction_Processing_Facility)

A few years ago someone trying to spin container technology did a lot of damage to other attempts at unikernels with marketing dogma, not reality. Claims of non-debugability and other FUD. It's long been standard operating procedure to trace and debug systems from outside the system's view. This is how people do bringup of new chips as well as OS ports. On hardware there is usually a dedicated debug and trace facility as part of the CPU or board support package or a firmware monitor. In a virtualized environment like a unikernel this is way easier because you can run code above the guest's ring 0 supervisor privilege against its RAM/pagetable root. Modern systems like POWER even allow debugging from the BMC, with a plan to allow full gdb sessions over that out of band interface [https://github.com/open- power/pdbg](https://github.com/open-power/pdbg).

There's nothing implicitly wrong with unikernels, or other systems software ideas like microkernels, just because they are less popular technology at the moment. I'd encourage people to continue exploring this space.

<comment><USER3> That FUD was from Bryan Cantrill ([https://news.ycombinator.com/item?id=10953766](https://news.ycombinator.com/item?id=10953766)) making aggressively resolute, but also easily falsifiable claims about the production-worthiness of unikernels.

From the perspective of someone who'd debugged and traced unikernels from both inside the runtime (LING) and outside the runtime (xentrace), and who worked in the domain of the very z/TPF system you referenced above, the indictment seemed at-best strangely misguided and at-worst intentionally duplicitous.

As you can imagine the indictment wasn't at all persuasive to me, and thus I keep exploring the space in bits and pieces where applicable.

<comment><USER4> While acknowledging that your claims about his claims may well be 100% correct, I find it hard to believe that Bryan Cantrill would be "intentionally duplicitous". People, even very smart people, can occasionally be very wrong (btw, I'm not making a claim either way here), and that doesn't mean there is an agenda/intent that is malicious in nature.

<comment><USER5> Cantrill is a smart guy. But he has always had a big mouth. And the fact is his post was written with ZERO experience with Unikernels. Zero. He was just assuming they act the way he imagined and attacked that image.

Which is fine if you're upfront about it. He wasn't.

<comment><USER6> I appreciate the kind words, but let's give me a tad more credit in the experience department, please: I have been doing production OS kernel development for over two decades, and have done non-trivial work in essentially every privileged subsystem across several different microprocessor and OS architectures. If you want to say that I have gobs of experience in kernel development (and more generally at the hardware/software interface), but no experience with unikernels per se, then fine, I guess -- but at the same time, let's acknowledge that you are the CEO of a unikernel company who very much has a dog in the fight?

<comment><USER5> Absolutely. I would rank you in the very top of developers with experience in development of traditional operating systems. In particular DTrace stands out as an excellent piece of work. It's one of those fundamental advances that serves as inspiration for others.

Now, I'm not sure I agree that I have a horse in the race. I don't necessary believe that there is a race. I've never really been a proponent of the schism between Unikernels and Containers. I struggle to see how Unikernels can offer the same flexibility and ease of deployment as containers. We're likely won't be able to support the vast amounts of runtimes and infrastructure needed to replace something like Docker. Perhaps there could be very specific uses where something like the paper described could be used, but I'm not betting on it.

As a software project IncludeOS has a much narrower target than what people traditionally have thought when thinking of Unikernels. And as a result of of this we're not in the business of replacing neither containers not general purpose operating systems(GPOS). We're aiming to carve out a few niches where we are confident that a GPOS isn't the answer. We're only going to address those needs where we're pretty certain we can actually add some value. Basically we're think we can improve on security in addition to adding real time capability whilst still remaining source-code compatible with Linux (mostly thanks to musl).

My grief is singularly with the myths you helped create that Unikernels are something where you are forces to work with stone age tools and hardly without any tools, except printf, for debugging. We've had to spend a lot of time dispelling these. There are a few other things I believe you where wrong about at the time but I'll spare you the details. Better suited discussions over a beer of coffee.

</comment></comment></comment></comment></comment></comment><comment><USER7> It's a nice exercise, but I don't see t going anywhere.

1st. I don't see that many bugs in the TCP/IP, filesystems, etc. specially those exploitable by black hat hackers, specially remote exploits. We usually see wide-spread vulnerabilities in the middleware like OpenSSH, which is in the userspace, putting everything in the app is not going to solve that, quite the contrary, we need to wait for the library maintainer to fix the bug, for the applications to integrate with the newer version of the library, and then, update all the applications.

2nd. Side-channel attacks. I'm not really confident in letting applications have that low level of hardware access. I mean, it's really easy for one application to steal the resources from another. We need authority if we are sharing peripherals. Well, we have one peripheral already that is shared by processes, that's memory and look at the mess that is a modern MMU and all the attacks on those that we have seen throughout the decades.

3rd. If we are going to implement that as a shared library, the result is exactly the same as today's micro and hybrid kernels, if we are not going to save memory and implement that as a static library, it's exactly the save as virtualization.

Correct me if I'm wrong.

<comment><USER8> Co-author of the paper referenced in the blog post here.

1\. Regarding bugs: There is a lot of code in your monolithic OS kernel. There have been, and will be, a lot of vulnerabilities in that code. Various sandboxing mechanisms notwithstanding, the vast majority of processes running on your system can potentially use all of that code as an attack vector. Unikernels let you switch off access to most of that "host" code, and, through the use of a library OS, contain only the minimal set of libraries needed to run your application in the "guest". Regarding updates: Valid point, but no different from any modern application of substantial complexity, except that now you also have to update (e.g.) the library providing its TCP stack.

2\. Unikernels don't make that problem any worse. In fact, if you run on something like Muen ([https://muen.sk/](https://muen.sk/)), it'll mitigate a bunch of these attacks by giving you a less precise RDTSC at the subject ("VM") level.

3\. I don't follow. Implement what?

<comment><USER9> If the unikernel is running as a normal process, all that potentially vulnerable code is still on the box, and still potentially executable in the event of a vulnerability. The security selling point of a standard unikernel is that the all the potentially-vulnerable code _doesn 't even exist_, because only the libraries that are actually used get compiled and linked into the app, and it has no code relevant to functionality that it's not actually using.

The paper suggests using system-call filtering via seccomp or similar to block off this attack surface, but if you trust seccomp to have zero bugs, you might as well do traditional process sandboxing a la Google Chrome.

The problem unikernels solve is that traditionally, a vulnerability in C code means "game over" and the box is completely pwned with all memory and storage directly accessible, while with a unikernel, a vulnerability means only that an attacker can access other functionality within the app, and can't rely on eg. a shell or a debugger being available to arbitrarily move bytes around.

<comment><USER8> > If the unikernel is running as a normal process, all that potentially > vulnerable code is still on the box [...]

Correct. Conceptually the same applies to all Type 2 hypervisors. Type 1 less so, but you could still potentially exploit Xen and you have all of the dom0 to play around in.

> The paper suggests using system-call filtering via seccomp or similar to > block off this attack surface, but if you trust seccomp to have zero bugs, > you might as well do traditional process sandboxing a la Google Chrome.

If you do that, you will have to do a line-by-line analysis of the code you want to sandbox in order to determine exactly which syscalls it's using. With the approach presented in our paper the developer does not need to care about this. For example, she can develop her MirageOS unikernel as a normal UNIX process and switch to a guaranteed-to-work-minimal-seccomp sandbox with a simple change of target (build-time configuration option).

But yes, you are now trusting seccomp instead of KVM. I believe in giving people the ability to easily make that choice.

> The problem unikernels solve is that traditionally [...] can't rely on eg. a > shell or a debugger being available to arbitrarily move bytes around.

That part does not change with the sandboxing mechanism changing. The stuff that's inside is still only your unikernel.

<comment><USER10> > you are now trusting seccomp instead of KVM.

Which do you trust more?

<comment><USER8> That's a very good question. The answer is, "it depends".

For the 80% case, on x86_64, I consider them more or less equivalent. KVM is used daily in anger to provide isolation (e.g. GCE, and now ChromeOS) and has been around much longer _but_ you need to trust hardware virtualization which is a large attack surface on the CPU itself. Given what we've learned about CPU vulnerabilities over the last year, I wouldn't be surprised to find some lurking in the VT-x/SVM implementations.

Seccomp OTOH is difficult to use correctly for arbitrary/existing applications but exposes less of the kernel (depending on your metric, see our paper) and does not need hardware virtualization.

For the 20% case, where the stakes are higher (e.g. High Assurance), I would use something like Muen or SeL4 and run a disaggregated system on top of that.

</comment></comment></comment></comment></comment></comment><comment><USER11> I don't quite understand the title of this article. Where is the part that explains why it's no longer an academic exercise? Nothing has been shown to now exist at a scale that makes it any more than another toy (in the academic computer science meaning of the word).

<comment><USER12> I was personally an unikernel conference in Beijing in May. Baidu, Alibaba, VMWare, ARM and many other multi-billion dollar companies were talking about how they are rolling in support into things like their serverless infrastructure.

Also - there _are_ production deployments out there.

<comment><USER11> VM support for unikernels is literally a prerequisite for unikernel uptake, so that's hardly surprising, but coming from academia: if a highly niche market using a thing still keeps it a mostly academic exercise, and this article does not contain anything that talks about how it has finally escaped the "mostly interesting to academics, and that handful of people who use it in their highly specialized or possibly instead just plain old 'someone once set it up and it's too critical to change now' systems".

<comment><USER13> Microsoft used their unikernel research (Drawbridge) to bring SQL Server into Linux.

[https://arstechnica.com/information- technology/2016/12/how-a...](https://arstechnica.com/information- technology/2016/12/how-an-old-drawbridge-helped-microsoft-bring-sql-server-to- linux/)

It is also how they kind of implement secure kernel in the recent versions of Windows 10.

</comment></comment></comment></comment><comment><USER14> I don't see the value in unikernel-as-a-process. Most of the syscalls save time by e.g. sending a bulk payload and having the kernel space TCP/IP implementation break that up into fragments and handle sending them to the interface. Moving the barrier halfway down now means you did all of the work and got negative performance benefits - seemingly under the guise of security... but isn't the whole point of unikernels to run hardware assisted VMs as your segmentation barrier instead of relying on software ferrying calls and data between hardware trust levels?

<comment><USER5> The value lies in security in this context. You also get a bit of performance as the "unikernels" are self-contained. So you can do most things, including the IP stack without switching into kernel mode.

I'm guessing this is mostly interesting for FaaS platforms or similar. You get isolation similar to hardware-assisted vms but with a lot less overhead and with phenomenal boot-times.

</comment></comment><comment><USER15> While it is possible to do, there is an important cost benefit analysis to make. All of the benefits the article lists can be a drawback as well. Say your specific network needs tweaking to get good performance, with a unikernel you must do this with all applications, while for a "traditional" monokernel you can do it once and it works for all applications.

That is not to say the approach is not interesting, but it'll probably be for in some kind virtualization, where you abstract away the OS entirely, and run applications directly on the hypervisor.

<comment><USER16> I am not sure the example makes sense at all.

Default settings always not working for all apps.

If you need to tune network, if that's a tuning for all apps, development can be done inside net lib, all apps rebuilds and release, that's not more expensive than tuning a kernel config.

If you need to tune on per app basis, then unikernel is wildly safer because of the isolation.

<comment><USER15> Not everything is open source.

<comment><USER16> Isn't that make my statement more true?

</comment></comment></comment></comment><comment><USER17> Single Root IO Virtualization (SR-IOV) has the potential to enable this. If the number of virtual functions climbs high enough for the number of process, or at least the number of critical processes, then you can just map a virtual function into all of the necessary processes. DPDK and SPDK already partially enable this but the number of virtual functions or complete lack there of for NVME devices is still a limiting factor.

</comment><comment><USER18> What are the best tools for experimenting with unikernels?

<comment><USER5> I believe it comes down to your programing language preference. If you are into OCaml the Mirage Unikernel is rather complete. If C or C++ is your thing there is IncludeOS (potentially adding other language runtimes next year) and for Haskell there is HalVM.

There are others as well, but AFAIK these are the ones with active development happening. Please correct me if there are active ones I've forgotten.

All three support compiling your application as a Linux binary, meaning you can do most of the development and debugging using the tools you're used to, IDE with visual debuggers and the whole shebang.

Once you want to run in a separate VM you can still debug, but it becomes a tad bit harder. I know how to debug IncludeOS application when they are running under Qemu or KVM. Qemu can act as a gdb remote, you just need to get it working, which is a bit of pain.

And now you have the option of running it the way it is describes in the paper. I've never touched this so I don't have a feeling for how hard it is getting it to run.

<comment><USER19> Rumprun[0] is another big one in the C camp. Though I think you can use basically any software that conforms to posix.

[0] [https://github.com/rumpkernel/rumprun/](https://github.com/rumpkernel/rumprun/)

<comment><USER5> No commits since April. I think Antti has left the project and started a brewery or something.

</comment></comment></comment></comment><comment><USER20> For the counter argument, see "Unikernels are unfit for production" January 22, 2016 - by Bryan Cantrill

[https://www.joyent.com/blog/unikernels-are-unfit-for- product...](https://www.joyent.com/blog/unikernels-are-unfit-for-production)

<comment><USER21> I like this essay, but I think it's really pointing to a narrow space, rather than a lack of space.

Bryan points out many weaknesses of unikernels, but assumes everybody needs multi-tenancy and the ability to spawn processes. And debugability is weak, but that probably depends on the application / environment you run. A lot of people want to run unikernels in a VM environment, but for me, it seems like the right application of unikernels is where you have one application that you want to expand to fill a single machine -- bare metal, boot to the application, save all the layers; there's no multi-tenancy, but if I'm running on hundreds/thousands of machines, I don't need multi-tenancy.

Debugability is important, but lots of people run without a kernel debugger, so it might not be that important to everyone. If you want it, you'll have to build it, but DTrace and friends had to be built too -- and it's easier to build it a second time, since you know it's possible and what it should look like.

<comment><USER8> And indeed, we have built some tooling for debugging, since we wanted it: [https://github.com/Solo5/solo5/blob/master/docs/debugging.md](https://github.com/Solo5/solo5/blob/master/docs/debugging.md). It wasn't that hard.

</comment></comment></comment><comment><USER22> Can a in kernel Webserver be understood as unikernel?

I.e. [https://en.m.wikipedia.org/wiki/TUX_web_server](https://en.m.wikipedia.org/wiki/TUX_web_server)

<comment><USER5> Sure. As long as you disable init on the server you'll have a unikernel setup. Round hole, square peg, though. Linux isn't meant to work like that.

</comment></comment><comment><USER23> I still think it is. There's still been no push in the open source community to make tooling around Unikernels to make it user to a broader audience and a broader set of use cases.

</comment><comment><USER24> [https://pdos.csail.mit.edu/archive/exo/](https://pdos.csail.mit.edu/archive/exo/)

</comment><comment><USER25> What advantages do unikernels have over microkernels?

<comment><USER26> Unikernels are older than microkernels, in terms of design philosophy, and owe more to VM (the IBM project from the 1960s) than later OS design.

Microkernels are about splitting a single OS into what are sometimes called 'servers', whereas unikernels are individual systems running on a hypervisor; ultimately, the biggest advantage is that, in a microkernel, if one 'server' goes down (the disk server, for example) the whole system is down, whereas a single unikernel can fail without impacting any other unikernel.

[https://jdebp.eu/FGA/microkernel-conceptual- problems.html](https://jdebp.eu/FGA/microkernel-conceptual-problems.html)

[https://utcc.utoronto.ca/~cks/space/blog/tech/HypervisorVsMi...](https://utcc.utoronto.ca/~cks/space/blog/tech/HypervisorVsMicrokernel)

<comment><USER27> > the biggest advantage is that, in a microkernel, if one 'server' goes down > (the disk server, for example) the whole system is down

Rather, if the disk server goes down, the whole system _might_ go down. It doesn't necessarily _will_ go down. You could, for example, simply restart it. If your Graphics server goes down, you might still be able to SSH to the machine, and in the example of yours which I quote you could run a remote emergency SSH server in full RAM.

Unikernel and microkernel are good examples of (fine-grained) principle of least privilege. Doing that right is _difficult_. Just ask the NSA or Intel.

<comment><USER26> Realistically, if a disk server goes down, there are two possible reasons:

1\. The disk hardware is bad. By all means, take everything down until you replace the disk so you don't get garbage in important files!

2\. The disk driver software is bad. By all means, take everything down until you replace the software so you don't get garbage in important files!

In neither case is simply bouncing the disk server an acceptable answer. Debugging and fixing the underlying problem is essential, and that requires taking the whole system down.

<comment><USER21> The disk server could conceivably fail because a disk took an absurdly long time to respond to a command --- by all means, update the firmware, but in the mean time, restarting the disk server (and replaying any pending writes, somehow) will get you back on track. At least restarting the disk server in read only mode allows reads to continue to function -- maybe even allows for an orderly evacuation of the data.

<comment><USER27> Good thinking. Read-only mode is an interesting suggestion. Filesystems allow this as well, and it can allow for the live extraction of data having to resort to professional recovery means (ie. physical and expensive).

I've had times with bad sectors (Deathstar at the very least) where reads would result in complete OS lock up or very laggy situation. That can be solved by only trying so many times to read (though that was on Windows 9x with PATA).

</comment></comment></comment></comment></comment></comment><comment><USER28> We're talking about pretty significant improvements in network I/O performance (among other things) with a unikernel, correct ? My understanding is that there have been several projects that have revolved around putting Linux network drivers into userspace so as to improve the throughput and avoid the kernel mode switch.

<comment><USER29> Here are some presentations from Google touching on reducing kernel overhead for high performance networking: [https://netdevconf.org/1.2/slides/oct6/03_jerry_chu_usTCP_LK...](https://netdevconf.org/1.2/slides/oct6/03_jerry_chu_usTCP_LKL.pdf) [https://www.usenix.org/sites/default/files/conference/protec...](https://www.usenix.org/sites/default/files/conference/protected- files/nsdi16_slides_eisenbud.pdf)

<comment><USER28> Beautiful, thanks for the links.

</comment></comment></comment><comment><USER30> It might be interesting, at least as an exercise, to combine unikernels with the split kernels from distributed "lego os". Having a slimmed down only what is used footprint spread out across the network could lead to multiplied advantages.

</comment><comment><USER31> After a presentation I saw at ToorCon earlier this year, I don't trust unikernels in general. Several of them are beyond bad from a security standpoint with little to no protections from malicious software.</comment>


Google's new look for mobile - <AUTHOR> http://googlemobile.blogspot.com/2010/05/googles-new-look-for-mobile.html

<comment><AUTHOR> The button to the left of the search results has an arrow on it.

In most cases (including Google's new web interface) the button is an action button with the arrow pointing in the direction of future action (expansion or contraction). In this case the button looks like a status button with the arrow pointing in the direction of the past action (expanded or contracted). It feels awkward.</comment>


25 Days of ReasonML - <AUTHOR> https://medium.com/@lilactown/25-days-of-reasonml-a4b20a7e8c6 <comment><USER2> I've been learning ReasonML for React Native these past few weeks. I love the static typing. I can work on my editor for minutes at a time, and when things compile I see it "just work" on my phone. This is important since reloading a native app is not nearly as fast as reloading a web page.

The best part is how FAST the compiler is! It blows me away how fast the reason/ocaml compiler is, considering it spits out JavaScript AND type checks at the same time. Babel, TypeScript, Flow, etc. are all slow as dogs by comparison :)

<comment><USER3> How are you handling navigation?

<comment><USER2> I haven't gotten that far yet, but I'm planning to use their minimal built-in one: [https://reasonml.github.io/reason- react/docs/en/router.html](https://reasonml.github.io/reason- react/docs/en/router.html)

<comment><USER4> ReasonReact currently only handle browser routing (it hooks directly into window.location API. So for ReactNative, it could do it the way you used to do with Javascript.

</comment></comment></comment></comment><comment><USER5> I find ReasonML a VERY HARD sell when F# is sitting there waiting. If I could get people onboard with an ML, why wouldn't I use F# with it's massive backing ecosystem and fantastic server side runtime(node has come a long way, but it's no CLR)?

<comment><USER6> I tried getting into F# and for someone not running on Windows without knowledge of the CLR and .NET ecosystem its a hard sell.

<comment><USER5> Was this recently? Things change really fast, within the last year or so we got ionide and core clr support. I've read of people preferring the experience of F# in VSCode these days.

Just a few weeks ago I got an F# project with TensorFlowSharp up and running on OSX in VSCode with the ionide extension. It JustWorked™.

In any case, I'd suggest you give it a crack again if you are interested. The experience has improved greatly and the tooling seems pretty great now on Linux/OSX. I've been following it for 3-4 years though, so I certainly understand how it would seem like a non-starter in the recent past.

</comment></comment></comment><comment><USER7> I'm interested in Reason/Buckle, coming myself from the ClojureScript.

The seemingly tinier minified compilation size would be a huge plus. But what are the actual sizes emitted for large scale projects (arguably the ones that matter most for static typing)? Anyone could share real-world experiences?

For reference: I know of cases where 60K lines of clojurescript (arguably equivalent to 150K lines JS) compile down to 5MB under advanced optimization mode.

(in case that scares you: well it's an actually large app. We should do route- based code splitting soon - easy/robust with clojurescript apparently)

<comment><USER8> That's a very interesting question. We could ask some Messenger and Instagram people, since they are heavily using BuckleScript. In general output JS is already fairly spartan, because BuckleScript uses a few tricks here and there, like encoding modules and records as arrays instead of objects; and completely inlining functions whenever it can.

BuckleScript also emits ES6 modules, with imports and exports, and it marks its output as either pure or impure at the module level. So potentially there's a lot of room for bundlers to do tree-shaking.

Bob Zhang, the BuckleScript team lead, is working on a new typesafe standard library for BuckleScript, which takes advantage of compile-time checks to compile to extremely minimal and idiomatic JS. Exciting times up ahead!

<comment><USER9> I have heard about the Messenger where they have 25-50% of the web client codebase in Reason, but I have not heard about the Instagram story. Can you share more info or some links about that?

<comment><USER8> Hi, about Instagram, I was mistaken about 'heavy' use. As far as I know there is one guy using Reason in Insta and from what he's said, he's found it pretty straightforward. I don't have a direct link, but if you search for 'instagram' in the ReasonML Discord chat you will find the discussion.

</comment></comment></comment></comment><comment><USER10> A weak point of ReasonML for me, last time I tried it, were the Promises; you have to remember to resolve them, with the compiler not providing any help. It's very easy to miss/forget to resolve them.

While it is definitely improvement over JS, it's a step back from the sense of safety you get with TS.

<comment><USER8> I would say it's the other way round. In ReasonML, if you forget to resolve a promise, the compiler tells you the types don't match up. This is better and more reliable than the best linting rules you can get today for JavaScript.

With TS, to be honest I'm not sure how safe it can be since it must play nice with the JavaScript rules that promises can be automagically resolved.

</comment></comment><comment><USER11> Nice article! I was also doing Advent Of Code[0] with ReasonML but sadly due to time constraints I didn't get any far. Still learned a lot about ReasonML, even if the solutions weren't really idiosyncratic! Definitely going to dive into ReasonML later again.

[0] [https://github.com/petetnt/advent-of- code-2017-reasonml](https://github.com/petetnt/advent-of-code-2017-reasonml)

</comment><comment><USER12> How does ReasonML/Bucklescript stack up against F#/Fable. Is the only difference that F# has the .NET environment?

<comment><USER8> Good question. I would say the differences are:

\- BuckleScript targets ES5 (+ES6 modules) directly, while Fable targets ES2015 (and beyond) and relies on Babel to transpile its result to ES5

\- BuckleScript/ReasonML needs only NodeJS/npm to work for JavaScript compiles (but you can get the opam compiler/package manager if you want to do native development)

\- BuckleScript/ReasonML has a more powerful traditional OCaml type system with higher-level abstraction features like functors and generalised algebraic data types, while Fable relies more on OOP for abstraction

\- At the moment at least, BuckleScript's JS output is far more minimal than Fable's–try defining a simple record type like `type person = {id: int}` in both their online playgrounds to see the difference–it's pretty striking

\- But Fable does have an open GitHub issue talking about minimising their output to be more BuckleScript-like, so it might catch up

\- BuckleScript's JavaScript interop is just as powerful if not more, but does require a shift in thinking because its premise is to convert JavaScript idioms into OCaml idioms

<comment><USER12> Thanks for this, great explanation(and as a F# looks fair). I'll try out ReasonML one of these weekends, thank you.

</comment></comment></comment><comment><USER13> I like the idea of Reason, but I currently do much more server coding than front end. The native compiler is currently classified as work-in-progress ([https://reasonml.github.io/docs/en/native.html](https://reasonml.github.io/docs/en/native.html)). I also wonder about how the community evolves if javascript-targeted reason libraries aren't compatible with native reason libraries. For instance, would a wrapper for a js promise library be able to work in server code? I'm interested in the language, but currently not investing time in it.

<comment><USER14> It's fully compatible with nodejs, so super feasible to do backend dev

<comment><USER15> Has anyone actually done this in a real app? The last time I looked for nodejs examples, I could only find small POCs. The vast majority of ReasonML material seems focused on the front-end.

<comment><USER16> I ran all of my Advent of Code[0] solutions in Node.js - there's nothing specific really about running a ReasonML app in Node.js vs the browser. Except you don't need a module bundler for the former. :)

I've also been slowly working on a small CLI app using ReasonML, ReasonReact and react-blessed.

There definitely is a dearth of ReasonML server-side examples, libraries and frameworks, though - especially compared to all the front-end chatter. bs- express[1] is actually quite nice, but hasn't seen much attention from the larger community. I would highly recommend it if it's something you're interested in pursuing.

[0]: [https://github.com/Lokeh/advent-2017](https://github.com/Lokeh/advent-2017)

[1]: [https://github.com/reasonml-community/bs- express](https://github.com/reasonml-community/bs-express)

</comment></comment></comment></comment><comment><USER17> Whoa, I was just looking at some examples of this library and it seriously looks just like JS, only give away is the patten matching(which is a feature I wish JS had)</comment>


Show HN: Beauty Contest for Virtual Life - Digital Art Project (lurepet.com) - <AUTHOR>

Hi, looking for your feedback on my first public project.
The Idea:
Every lurepet (big picture) that you see on the site has some "genes" and is drawn according to them.
After the best lurepets are selected they can be bred to make children looking like their parents. This is going to be repeated to create more and more beautiful lurepets.
And of course the reason to vote is to save the future humanity that is attacked by evil robots which has the only flaw - insane obsession with lurepets :)
Implementation:
The drawing logic which is 99% of all the logic, is written in Clojure. I'm very glad that I chose it, because I can not imagine beeing so productive in developing and maintaining (!) complex domain logic in a more verbose language like Java.
The web part currently is a set of static files: html, js and images. Cloudflare is used to CDN the static except the index.html. I hope that will help if there is a massive spike in traffic. The votes are processed from nginx logs in offline mode.
The development took about a year. Hope you will like the lurepet drawings and help to select the best genes. <comment><USER2> I like the idea of generating an image and selling it as a t-shirt but you have to do better than this.

Website design is not very appealing. Have a look what other people do: <http://dribbble.com/>

Your LurePets are not a new idea. Zazzle is selling them already: <http://www.zazzle.com/lurepets+gifts> . Personally I like fractal t-shirts more: <http://www.zazzle.com/fractal+tshirts>

But don't give up. It's a good idea that needs better front-end and more advanced math and pattern generation at the back-end. If you auto-generate images that no one else has and are appealing then you might have a sustainable business.

<comment><AUTHOR> Actually, Zazzle ones are mine :) But I see that it's not clear enough.

Thanks for the feedback.

<comment><USER2> That's great news. Keep doing it. I was thinking about using finite state machines to generate eye-catching structures. I'm guessing that you are using genetic algorithms to create populations of lurepets and improve them using users' input.

You could build a google app engine web app to do that. Users that will train your genetic algorithms will get discount for their t-shirts. You would use google "in-app" purchases API to sell your t-shirts. If you get enough data from users you can try to personalize, e.g. association rules, if someone bought t-shirt A then he is likely to by t-shirt B.

Note to self: web app that uses genetic algorithms to improve its product and association rules to sell more is also a data mining web app.

</comment></comment></comment><comment><USER3> The idea reminds me of electric sheep:

<http://electricsheep.org/>

</comment><comment><USER4> It think this is a really cool idea. I just think the design need a bit of work, especially the 'thumbs up' thumbnails.

Do you use any libraries for the genetic algorithm or was that all custom code?

<comment><AUTHOR> It's all custom code.

The fitness function is the human's taste for beauty which cannot be formalized in a direct way. As I suppose genetic libraries are only solving the cases when the fitness function can be expressed formally.

</comment></comment><comment><USER5> Couldn't you periodically poll the Clojure runtime for votes instead of processing logs? I thought Clojure made stuff like reporting easy. Just curious as someone new (1 day in) to Clojure.

<comment><AUTHOR> Yes, I could. I just overestimated the load a lot. That is why I went that "static" route.

</comment></comment><comment><USER5> This is pretty interesting. How many original lurepets were there initially, and how many are there now? Do they die?

<comment><AUTHOR> About 20000.

Currently on the site there are less than 100.

No, they did not die. They are still sleeping on my hard drive :)

</comment></comment><comment><USER6> Clickable: <http://lurepet.com></comment>


Contributing to Go with go-critic - <AUTHOR> https://medium.com/@olegkovalov/contributing-to-go-with-go-critic-88a25e162bf6 <comment><USER2> Planning to use this secret weapon on the next Go contributors workshop in Russia. :)</comment>


An introduction to Stoic philosophy and resilience-building [video] - <AUTHOR> http://learn.donaldrobertson.name/p/stoicism-in-five-minutes/ <comment><USER2> This book by Ryan Holiday: "The Daily Stoic: 366 Meditations on Wisdom, Perseverance, and the Art of Living: Featuring new translations of Seneca, Epictetus, and Marcus Aurelius" is a good intro to the topic. [https://www.amazon.in/Daily-Stoic-Meditations- Perseverance-t...](https://www.amazon.in/Daily-Stoic-Meditations-Perseverance- translations-ebook/dp/B01KAFIQE6?_encoding=UTF8&btkr=1&ref_=dp-kindle- redirect)

<comment><USER3> I double this recommendation - I read a chapter each morning on my commute to work.

</comment></comment><comment><USER4> A great book on the subject is called "A Guide to the Good Life", by William Irvine. It's a great read, and a good introduction to stoicism.

I read it while recovering from surgery when I was unable to walk for a few weeks, and found it to be very helpful in keeping sane.

<comment><USER5> I second this. "A Guide to the Good Life" is a fantastic introduction to Stoic principles.

I also read "Stoicism and the Art of Happiness", but did not enjoy it as much. I think Irvine's book is more to the point.

I've found that stoic practices help me in things I would not have thought of: job interviews, for example. Doing negative visualization, imagining interviewers asking me stuff I don't have complete knowledge of, and imagining how I'd respond to it was _extremely_ helpful recently.

</comment></comment><comment><USER6> Best intro to Stoicism I've seen: [https://www.youtube.com/watch?v=seLLJP3H1FU](https://www.youtube.com/watch?v=seLLJP3H1FU)

Massimo Pigliucci also have a great web site, how to be stoic [https://howtobeastoic.wordpress.com](https://howtobeastoic.wordpress.com)

<comment><USER7> Do you know where that video was recorded? I'd swear that's Bobby Knight at 12:36.

</comment></comment><comment><USER8> I maintain a small page about Stoicism here: [http://pa- mar.net/Main/Lifestyle/Stoicism.html](http://pa- mar.net/Main/Lifestyle/Stoicism.html)

I believe that the Stoicism's appeal (for modern tech people) is that it is basically a very rational approach to life and its problems. It is also very adaptable to modern day culture: it downplays Religion but respects Society, for example.

Last but not least, the image of dour, ascetical, judgemental party poopers is a bit off mark, I think. The ideal Stoic is of course quite restrained in everything, but feeling somehow superior, a part of some elite is a mistake in itself.

</comment><comment><USER9> Why is Stoicism the philosophical flavor of the month amongst tech people?

<comment><USER10> Because being in tech is life-ruining and only can only be propped up by ascetic myth-making, lol.

I suspect stoicism also appeals strongly to a personality that enters tech at a disproportionate rate -- like bodybuilding, nootropics, hackathons and life- hacks, it presents rigid, rationalized mastery of the self as both desirable and achievable. I'm hesitant to attempt to characterize the personality order to which this so strongly appeals, but it seems observable.

<comment><USER11> >like bodybuilding, nootropics, hackathons and life-hacks, it presents rigid, rationalized mastery of the self as both desirable and achievable.

This is a good observation. It reminds me of some kind of digital protestantism on steroids. The same kind of displayed humility, work obsession and so forth.

I think it's very self-aggrandising in a way.

<comment><USER10> So you want to live "according to nature?" Oh, you noble Stoics, what a fraud is in this phrase! Imagine something like nature, profligate without measure, indifferent without measure, without purpose and regard, without mercy and justice, fertile and barren and uncertain at the same time, think of indifference itself as power – how could you live according to this indifference? Living – isn't that wanting specifically to be something other than this nature? Isn't living assessing, preferring, being unfair, being limited, wanting to be different? And assuming your imperative to "live according to nature" basically amounts to "living according to life" – well how could you not? Why make a principle out of what you yourselves are and must be? – But in fact, something quite different is going on: while pretending with delight to read the canon of your law in nature, you want the opposite, you strange actors and self-deceivers! Your pride wants to dictate and annex your morals and ideals onto nature – yes, nature itself –, you demand that it be nature "according to Stoa" and you want to make all existence exist in your own image alone – as a huge eternal glorification and universalization of Stoicism! For all your love of truth, you have forced yourselves so long, so persistently, and with such hypnotic rigidity to have a false, namely Stoic, view of nature, that you can no longer see it any other way, – and some abysmal piece of arrogance finally gives you the madhouse hope that because you know how to tyrannize yourselves – Stoicism is self-tyranny –, nature lets itself be tyrannized as well: because isn't the Stoic a piece of nature? . . . But this is an old, eternal story: what happened back then with the Stoics still happens today, just as soon as a philosophy begins believing in itself. It always creates the world in its own image, it cannot do otherwise; philosophy is this tyrannical drive itself, the most spiritual will to power, to the "creation of the world," to the causa prima.

<comment><USER12> Ok, who wrote that?

<comment><USER10> Me.

It's actually Nietzsche -- "Beyond Good and Evil," Part 1, §9. I have very serious reservations about Nietzsche, but he doesn't pull his punches.

</comment></comment></comment></comment></comment></comment><comment><USER13> My favorite Stoic book is called A Guide to the Good Life: The Ancient Art of Stoic Joy. Extremely accessible compared to original Stoic texts and honestly changed my outlook on life for the better, especially how I view expectation and disappointment.

<comment><USER4> I came here to mention this. You beat me to it.

<comment><USER14> This is the third time I've seen this book recommended this week, time to download I think!

</comment></comment></comment><comment><USER15> One of the best books I ever read was from a dude named Seneca (a stoic Roman philosopher).

The book is called On the Shortness of Life: Life is Long if You Know How to Use it[0].

[0]: [http://amzn.to/2klxZky](http://amzn.to/2klxZky)

<comment><USER16> Written in 49 AD, so out of copyright, and available free online. [https://en.wikipedia.org/wiki/De_Brevitate_Vitae_(Seneca)](https://en.wikipedia.org/wiki/De_Brevitate_Vitae_\(Seneca\))

<comment><USER15> Even better, although I like having a physical copy.

</comment></comment></comment><comment><USER17> This makes me remember this TED talk, that was the first time I heard of Stoicism. [https://www.ted.com/talks/tim_ferriss_why_you_should_define_...](https://www.ted.com/talks/tim_ferriss_why_you_should_define_your_fears_instead_of_your_goals)

<comment><USER18> I really enjoyed this talk.

</comment></comment><comment><USER19> >Title: Stoicism in Five Minutes.

>Duration: Approx. 10 min.

Is it some meta stoicism to react stoically to that?

</comment><comment><USER20> I like the principle of Stoicism. But I don't see how can that help with the suffering of the people I care about. I am fine with losing my job, my home or my social position (or at least so I think), but what about my spouse? my children? How can I still feel calm when I fail to support them?

<comment><USER5> I don't think the point is feeling completely calm when things are going bad.

For me, at least, one of the points of stoicism is learning how to take a step back and look at the situation through more rational lenses, and then figure out how to deal with it.

Take the situation you mentioned: failing to support for one's family. I think someone who strives to practice stoicism would do something like this:

\- What are the reasons I'm failing to provide for my family? Am I spending too much? Is my salary too low? Is the place I live too expensive? Does my family spend too much money?

\- Of those reasons, which ones do I have full control of? (This is were you can actually act on).

\- Which ones do I have _some_ control of? If your family is the cause of financial distress, there are some actions you can take (like talking to them and explaining things have to change), but you can't fully control their actions and thoughts.

\- Which ones do I have no control of? Don't worry about these, there is nothing you can do anyways. But here is the catch, you are eventually going to worry about them. This is what people fail to get about stoicism. It is not a silver bullet that is going to take away all your worries. It is an instrument to help you overcome them, with reason. It takes some practice to get good at it.

<comment><USER21> Overspending is the first class citizen problem, there are many other problems can happened.

Billions of people live surrounded by crimes, wars and poverty.

</comment></comment></comment><comment><USER22> I enjoyed this podcast about stoicism [http://www.stuffyoushouldknow.com/podcasts/stoicism.htm](http://www.stuffyoushouldknow.com/podcasts/stoicism.htm)

</comment><comment><USER23> stoicism++ but I did a double-take at the site: it's a personal vanity site but styled as a MOOC? Ah, it's a hosted Teachable site on a custom CNAME.

</comment><comment><USER24> I wish I could be truly stoic. Or a Vulcan. Instead, I constantly walk on the edge of a panic attack.</comment>


Hi,thinking of creating an app to limit AWS account spend, need honest opinions - <AUTHOR> I am new here so please forgive me if I'm stepping out of line here.
Long story short, I recently got an aws account for side projects but I am scared that someday someone's going to go crazy with one of my public websites and I'd end up with a large $XXX bill. I have also heard horror stories from this sub about people trying something or forgetting to turn resources off and ending up with a large budget. I know we can set budgets and all but it doesn't work for people who want to cap the total $ spent.
I am a little surprised aws hasen't introduced this feature despite many requests (although when I think from a business pov, it makes sense).
Basically I am want to create an app where you could say something like
"I ONLY want to spend $XX per month, if my bill is more than that, shutdown my resources"
My idea is to create a website where you can set a budget amount (monthly?) for your account and setup rules for different resources ( like shutdown EC2, make S3 bucket private etc)
I am pretty good with creating backend tools and worked a lot with aws lambdas and s3.
I'd love to get some ideas from you guys.
Also if you don't think its gonna be helpful, let me know that too.
thanks <comment><USER2> Very helpful. Pretty hard to do comprehensively without (but less comprehensive solutions could still be useful).

The biggest question is "where are you going to get your data about current spend?. AFAIK the budget data lags behind usage (which is what I assume is the reason people haven't done this before). You can do this at a per-service level (e.g. EC2) by periodically polling EC2 for active resources and keeping a running tally of cost by yourself. This works well for some services like EC2, RDS (although serverless Aurora gets tricky), etc. However for other managed services like S3 or Lambda, it's harder to keep track of how much you have spent.

Cloudtrail could be useful here, but it probably doesn't have enough information to calculate actual cost (e.g. Lambda charging based on invocation time or Athena charging based on the volume of data scanned).

But even something as simple as a tool that keeps track of just your EC2 instance spend and shuts them down if it gets too high would be useful to developers who have AWS accounts for their side projects.

<comment><AUTHOR> Yes I think less comprehensive solution could cover major services (EC2, lambda etc)

> The biggest question is "where are you going to get your data about current > spend?. AFAIK the budget data lags behind usage

interesting, I was thinking I could poll billing metrics (total charges etc) to figure out account level charges.

In your experience, do you care more about EC2 costs or that was just an example?

<comment><USER2> > interesting, I was thinking I could poll billing metrics (total charges etc) > to figure out account level charges.

If it were that easy, this would already exist ¯\\_(ツ)_/¯

I think many people use AWS primarily for s3 and EC2 (and s3 costs are negligible for most side projects) so if you were picking a single service, EC2 would probably be the most broadly applicable (and IMO the most realistic one to actually build). On the other hand I would imagine that for many side project developers, their EC2 costs are the easiest to control - even if you do autoscaling, you can cap your max nodes.

What is most useful is tracking the total AWS spend because what really gets you is the service you weren't expecting to cost as much as it did, but that is a very very very hard problem.

<comment><AUTHOR> I was looking at AWS Cost Explorer API, looks promising but don't know if it's granular enough.

after thinking about your comment on EC2, I think this tool could help smaller teams where you don't know who's doing what with the resources and you just need an upper limit of your monthly/daily spend.

about the last point, yes it's very hard to figure out. Also I thought most people check out the pricing page before trying out any aws services, wouldn't that give an estimate of what they are getting into (i.e. RDS, API gateway) ?

But again I always forget about the data transfer charges so you might be right about this.

<comment><USER2> I've used cost explorer enough to know that it lags meaningfully behind current costs, but you would probably have to do some measurements to know how much (I would guess several days). If you do that, remember that individual services (or even different components of a single service) might have different lag times. The cost explorer data seems pretty good though.

> I think this tool could help smaller teams where you don't know who's doing > what with the resources and you just need an upper limit of your > monthly/daily spend.

Yeah, definitely!

> But again I always forget about the data transfer charges

Yeah. AWS can have some pretty complex cost models since they charge based on the amount of resources you actually used - there are way more axes of cost than most people have time to think through.

If you end up building this, I suggest considering that many people will not want their resources automatically shut down, either because they want to serve the spike in traffic or because they are paranoid that there could be a mistake and have production infrastructure accidentally shut down. I would consider designing it so that users can choose to get a Slack message or something instead of having the resources shut down.

</comment></comment></comment></comment></comment><comment><USER3> Unless I misunderstand your concept, using your website would entail granting it rights to my AWS account to access billing info and shutdown pretty much any service. So, something close to the Administrator policy. I guess you could do something more granular e.g. stop but not start, specify resources, etc but in any case I need to grant you enough rights to shut down my account, which is the entire premise of your service. What are the chances that I'll grant this type of access to a 3rd party I don't know about? Snowball, meet Hell.

Also, I don't want my production account to be shut down. Notified, sure. But if my spending takes off because Elon Musk tweeted about my website and I'm getting a lot of hits (and hopefully a lot of new paying customers) I definitely do not want my service to go down.

In other words, your idea isn't for me, but it might well work for side projects where you don't want to spend too much and don't care about the service going down. In any case the security aspect is going to be tough to manage properly, and is going to make your service a juicy target for hackers.

<comment><USER4> > Also, I don't want my production account to be shut down. Notified, sure.

This is already an AWS feature: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/monitor_estimated_charges_with_cloudwatch.html)

</comment></comment><comment><USER5> I'd advise against trying to turn this into a product. Companies will be hesitant to grant a third-party that level of privilege on their infrastructure. And sophisticated companies already have the means to manage their costs. For sophisticated deployments a third-party is unlikely to be in a good position to make those decisions. It's not as simple as "just turn it off". Unsophisticated companies tend to have a simple and predictable architecture with a (largely) fixed monthly cost. The product will be too complex for those simple companies. So I suspect a product like that will be "not good enough" for the serious companies, and too dangerous/difficult for simple companies.

</comment><comment><USER6> Since the available billing information lags actual usage, track the cost data over time and interpolate it forward in time to be a projected / estimated real-time figure, and act according to that projection. Be sure to let users know how your system works and that it's a calculation you're running based on a smoothed cost (you can make that as rigid or not as makes sense, you could look for variances in times of day to the cost for example and adjust your real-time estimates accordingly, and so on).

It's entirely unnecessary to be exact, you need to be close enough. This is especially true since it's a feature a lot of people want, they'll accept some weakness in the system within reason.

You could offer a flex range the customer can set, a tolerance. If you can't reliably get near that (eg if you can't get within N% when back-checking your estimates versus what ends up being the actual billing), then you let the customer know with a status update and the customer can make a decision as to what to do at that point (they could stop using your service, they could adjust the tolerance, and so on).

Require a minimum number of polled data points before the system comes alive and becomes capable of shutting down services. You need a certain number of records before you can reasonably extrapolate anything.

This system won't work well in cases of normal usage with a sudden extreme burst that blows out the customer's budget (where the customer is hoping to prevent that type of unexpected one-off surge). You could do an accelerated estimate guess if you see a very large, very unusual spike, and jump ahead as a means to over-protect (another setting the customer could choose, they could set a caution level that guides how much you should accelerate ahead in cases of huge surges).

If you're successful, Amazon will kill you or buy you (possibly ask to buy you for cheap, then kill you if you refuse). As with similar hyper platforms, the platform eats its own ecosystem inhabitants in the name of growth. And this is a blatantly obvious feature that is widely demanded, so if you figure out a way to do it even modestly well, they'll knock you out eventually. Print money for a while if you can, charge as much as you can, prepare for the inevitable; consider selling to a cloud service company if you can find the right offer.

<comment><AUTHOR> Thanks for the detailed writeup. I think projecting future costs would be hard but someone pointed out in previous comments that service specific rules ( EC2, S3) etc could be more realistic.

Also what do you think about AWS Cost Explorer API? It looks like it can provide monthly/daily charges and we can build rules for customer tolerance, like you mentioned.

</comment></comment><comment><USER7> I think it's useful. Im not sure if any third party tools has a similar feature but definitely would be helpful specifically to those who are new to the product.

So you will have the option to auto stop services to minimize spend or terminate services if stopping will not reduce the bill (rds is an example).

User probably needs to be able to set priorities, locks, etc to prevent disruption of services in case they are using it for production.

<comment><AUTHOR> agree. I could not find any third party tool that does that. Can you please tell me more about setting priorities, locks?

</comment></comment><comment><USER8> I would love this! I agree it seems like a missing feature of AWS. However as someone else commented I'm not sure how you could practically go about implementing it.

<comment><AUTHOR> thanks, yea I'll try to figure that out</comment></comment>


Sify.com Architecture - A portal at 3900 requests/second - <AUTHOR> http://highscalability.com/blog/2010/5/10/sifycom-architecture-a-portal-at-3900-requests-per-second.html

<comment><USER2> Hands up if you thought that said syfy.com and you were surprised

</comment><comment><USER3> 150 million page views a month works out to 57.9 pageviews a second. So the vast majority of those requests must be JS, CSS, and static image requests, although 98.5% seems a bit off?

<comment><USER4> The 3900 requests per second are likely not constant. That number is probably during their busiest time during the day, so their infrastructure needs to support that kind of traffic. You're right, though, requests per second likely includes all file types.

</comment></comment><comment><AUTHOR> Read the comments section as well. The author (Ramki) has responded to queries raised in that comment section.

<comment><USER3> From the comments, those are 'backend requests' and not even static file requests.

</comment></comment><comment><USER5> A portal? More like spam to me.</comment>


Could Netflix circumvent streaming limitations by remote-mounting the DVDs? - <AUTHOR>

Growing a bit bored with the streaming selection at Netflix.
What if they offered to host a DVD/BlueRay player for you, have it plugged in at a Netflix fulfillment center, load whatever DVD you chose, and then let you remote-mount that sucker to your computer?
Sure, there are some obvious scalability and responsiveness issues here (... and here we store the 10 million DVD players, and over there is the army that loads all the little discs...), and not everyone has a computer hooked up to their TV, but shouldn't it in theory circumvent any streaming limitations?
(This is obviously a retarded solution from a technical perspective - I'm just curious if it would work as a legal hack) <comment><USER2> Remote DVD streaming was the idea behind Zediva: <http://arst.ch/qf9>

<comment><AUTHOR> Thanks! I guess that pretty much answers that question ;)</comment></comment>


Deep Image Analogy - <AUTHOR> https://arxiv.org/abs/1705.01088 <comment><USER2> Supplemental Material with bunch of other examples: [https://liaojing.github.io/html/data/analogy_supplemental.pd...](https://liaojing.github.io/html/data/analogy_supplemental.pdf)</comment>


Show HN: A threaded, TCP, key value store in Haskell - <AUTHOR> https://github.com/honza/redish

<comment><USER2> You can change

        appV fn x = atomically $ readTVar x >>= writeTVar x . fn

to

        appV fn x = atomically $ modifyTVar x fn

The function getValue need not be in IO monad. You can simplify it using fromMaybe in Data.Maybe.

Sometimes you check "head l" and "tail l"; it's more idiomatic to use pattern matching. For example, you can change:

        setCommand handle cmd db = do
            appV (conv k v) db
            hPutStrLn handle $ "OK"
            where k = (head cmd)
                  v = (unwords (tail cmd))
        ....
        case (head cmd) of
            ("get") -> getCommand handle (unwords (tail cmd)) db
            ("set") -> setCommand handle (tail cmd) db

to something like this:

        setCommand handle key val db = do
            appV (conv key val) db
            hPutStrLn handle "OK"
        ...
        case cmd of
          "get":key     -> getCommand handle (unwords key) db
          "set":key:val -> setCommand handle key (unwords val) db

Also consider using hlint, it often gives good advice.

<comment><AUTHOR> Excellent, thank you. This is really helpful.

</comment></comment><comment><USER3> Nice example of using Software Transactional Memory. Which is starting to be available in things like gcc, but is much much nicer in Haskell for reasons involving purity and monads.

Particularly this function, which changes a variable in the database, transactionally.

      appV :: (DB -> DB) -> TVar DB -> IO ()
      appV fn x = atomically $ readTVar x >>= writeTVar x . fn

It would be easy to use this to add commands that eg, increment counters in the database, and the STM would ensure that it works correctly with multiple concurrent writers. No locking needed. A quick example, which could be improved by changing the DB type to support Int as well as String values:

      incrCommand :: Handle -> [String] -> (TVar DB) -> IO ()
      incrCommand handle k db = do
          appV incvar db
          hPutStrLn handle $ "OK"
        where
          incvar k v = insert k $ show $ (fromMaybe 0 $ readMaybe v) + 1

I don't know how well Haskell's STM performs compared with eg, database transactions or locking. It's been more than fast enough for my own needs. Anybody know?

<comment><USER4> As far as I understand it, this simple example won't perform any better than if it were written using MVar. (i.e. it was explicitly locking reads and writes)

The difference is that `atomically` calls can be nested and still work. If you do an readMVar whilst that MVar is already locked in a function above you in your call stack you would have a deadlock.

So STM seems to be a smart way of keeping track which locks the current execution thread has so you never double lock.

At least in this case :)

</comment></comment><comment><USER4>
        conv k v db = insert k v db

You could rewrite that as:

        conv = insert

Probably some remnant of you figuring out how you should write appV, which is a proper helpful function, but probably should be called updateTVar or something :)

<comment><AUTHOR> LOL, thanks! You're absolutely right. You hack and hack until you just can't see what's right in front of you.

<comment><USER5> hslint will also find things like this.

</comment></comment></comment><comment><USER6> If you like this, check out <http://www.happstack.com/docs/crashcourse/AcidState.html> . It allows for arbitrary Haskell data structures to safely and automatically be marshaled in and out of the data store.

<comment><USER7> I really loved AcidState when I first learnt about it. Saves tons of headaches when converting Haskell's superb algebraic data types into dumb SQL types; but it also limits the points of entry into 1 (which means no outside queries/reports which SQL does very well).

Another feature it lacked is replication which I believe was on the roadmap so I'm not sure about its current state.

If you can live with the fact that your application is the only entry point for your data, definitely look into AcidState.

<comment><USER8> Replication and sharding (horizontal and vertical scaling) in acid-state are in development right now. The developer's page has the road-map as well as his contact details: <http://acid-state.seize.it>. There's another project for distributed storage and data processing in Haskell called Holumbus <http://holumbus.fh-wedel.de/trac> which powers one of the search engines for Hackage API's.

</comment></comment></comment><comment><USER9> How do I store something with a \n in it ;)

Edit: While I posted this as a joke, I do love the way that redis avoids any kind of injection attack via prepending the message length. Of course, I understand that this project is not at the stage where it is meant to be deployed in production/security become an issue; it's just interesting to see how the different communication methods have different consequences.

</comment><comment><USER10> No monads?!)

<comment><USER4> Every single function in that Haskell file returns a monad except for the version which just returns a constant string.</comment></comment>


Ask HN: Your favorite practical, underutilized programming tip? - <AUTHOR>

Example:
Tip: Bash reverse search, because manually scrolling through your history is a nuisance.
How (OS X): CTRL+R, start typing a word from your command history. Press CTRL+R to cycle through matches, ENTER to execute, TAB to edit. If you want to add forward search to bash, you can re-map your key bindings in your bash profile with the following (maps to CTRL+T): bind "\C-t":forward-search-history <comment><USER2> Just stumbled on this one recently, I forget where:

        foo | awk '{ print $NF }'

That prints the last column. $NF = number of fields. I use it a lot like this:

        git status | grep deleted: | awk '{ print $NF }' | xargs foo

I actually just realized, though, that I don't understand why this works. It seems like it should be '{ print ${$NF} }' or something, and the command as- given should just print "3" or "6" or whatever. I'm not an awk expert, so I'd love to understand what's going on here.

<comment><USER3> Bare NF is number of fields, so the $ must be pulling the value first. Also, awk is 1-based, not 0, so you get the last field :)

</comment></comment><comment><USER2>
        rails g migration add_foo_to_bar
        vi db/migrate/*!$*

or

        bundle exec !!

or

        rake db:migrate
        RAILS_ENV=test !!

<comment><USER4> It's shell syntax. !! is replaced by the whole previous command, !$ is replaced by the last argument in the previous command. There are tons of others, but these two I use the most.

Feels great to yell sudo !!.

</comment></comment><comment><USER5> I'll cheat and throw out a tip I haven't used yet: apparently if you read _man bash_ long enough, you get to the built-in TCP/UDP sockets! [1]

        bash$ cat </dev/tcp/time.nist.gov/13
        53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) *

[1] [http://hacktux.com/bash/socket](http://hacktux.com/bash/socket)

</comment><comment><USER6> 1\. Using a copy-paste manager to get more than 1 buffer (like Keyboard Maestro, but there are free ones)

2\. In a lot of editors (not vi), if you need to get back to the point of your last edit (because you scrolled away) use the keyboard shortcuts for UNDO, REDO. So in a lot of editors, CTRL-Z, SHIFT-CTRL-Z -- this jumps you right back to where you last changed something.

<comment><USER7> if by vi you mean vim, this works perfectly (and indeed i use it a lot; it's more reliable than hitting '').

if by vi you mean justplain vi, then why are you not using vim in the first place?!

<comment><USER8> You can use g; in vim to go back to the last edited place. It's using the changelist to navigate; see :changes for a list.

<comment><USER7> nice, thanks! it amazes me how much there is to learn about vim, and how useful some of the little random features are.

</comment></comment></comment></comment><comment><USER9> Define an old function in your .bashrc

        old() {
          mv "$1"{,.old}
        }

I use this as a cheap way to backup old files (alternatively use cp instead of mv).

</comment><comment><USER10> In Sublime Text, you can put bookmarks in code using _Cmd+F2_ and navigate between them using _F2_. It's really helpful if you work with big files.

</comment><comment><USER11>
      continue;</comment>


Documentation Sucks and You Probably Do It Too Much - <AUTHOR> https://qvault.io/2019/09/03/documentation-sucks-and-you-probably-do-it-too-much/ <comment><USER2> as a point of order documentation is different from commenting. commenting your code is essential.</comment>


How Would a 'Do Not Track' Mechanism for Online Privacy Work? - <AUTHOR> http://blogs.forbes.com/kashmirhill/2010/12/02/how-would-a-do-not-track-mechanism-for-online-privacy-work/

<comment><USER2> The issue of course is that at the moment, cookies are cookies. There's already the ability to disallow 3rd party cookies but you would still lose functionality.

My solution to this would be different cookie silos. There is already a "private browsing" feature in modern browsers - why not having multiple personas?

I use this silo when I'm working, this one when I'm shopping, this one when I'm looking up midget porn etc.

As well as "open in new tab" links would have "open with new persona" so that if you were on your facebook page and you wanted to go look at a site about testicular cancer you could do so with privacy.

You should then be able to reload the current page using a different persona so that you can decide how to engage with the page you're currently looking at.</comment>


Why women read more than men (2007) - <AUTHOR> http://www.npr.org/templates/story/story.php?storyId=14175229

<comment><USER2> "I've read at least 100 books in the past year. Seriously. Probably more like 150 to 200," a user named Phyllis wrote on the literary blog Trashionista.

At what level of attention can you read 100, let alone 200, books in a year, and what books? Not, I think, long books like _Man Without Qualities_, and not dense books like that or like _Portrait of a Lady_, to mention only novels. Could it be that at least some men are more realistic about their literary consumption--well, more realistic than Phyllis?</comment>


Ask HN: Should companies create an app that flags if employees comes within 6 ft - <AUTHOR> <comment><USER2> That would be phones within 6ft? I expect quite a lot of false-positive if pinging every second during a working day. And remember not everyone has a smart phone, or cares for one.

If people via a hardware widget strapped to them, I imagine that could create some regulatory hang-ups and class-action lawsuits via various state or national laws on information security, and that a lot would refuse to wear it anyway. Or leave it on their desk. Or find other creative ways. This would be true with phones too.

No. Wouldn't work.

</comment><comment><USER3> It might make sense but the problem is this "6 feet" thing is basically "BS."

Any virus that spreads thru moisture in the air that you breath and lives on surfaces for days can travel far more than 6 feet [1].

[1] [https://www.webmd.com/lung/news/20200414/cdc-covid-19-can- sp...](https://www.webmd.com/lung/news/20200414/cdc-covid-19-can- spread-13-feet-travel-on-shoes)</comment>


Use PUT for Partial Updates in your REST API - <AUTHOR> http://kun.io/blog/69723849234/Use-PUT-for-Partial-Updates-in-your-REST-API#!

<comment><AUTHOR> I've been unsuccessfully debating this with the author on LinkedIn... In case anyone is interested why I think this is wrong: [http://www.linkedin.com/groups/Use-PUT-Partial-Updates- in-29...](http://www.linkedin.com/groups/Use-PUT-Partial-Updates- in-2906459.S.5816696924446670850)

Am I wrong here? Maybe I'm just being too critical...

<comment><USER2> You're not wrong. I said the same thing in the comments.</comment></comment>


Building a single page application with vanilla js - <AUTHOR> https://dev.to/vinay20045/building-a-single-page-application-with-vanilla-js/ <comment><USER2> Bored with all the coughing people do (but otherwise a good article, except for omitting auth).</comment>


The American Anti-Corruption Act - <AUTHOR> https://act.represent.us/sign/the-solution/ <comment><AUTHOR> Really well done video.</comment>