The Future is Hypermedia APIs - <AUTHOR> http://www.emergentone.com/blog/the-future-is-hypermedia/ <comment><USER2> _> Hint: Your browser is an API client_

Apart from parsing, _I am_ the API client in this instance. I'm the one making decisions from understanding what services are about.

The universal API client seems to be a kind of silver-bullet-IA in many people's mind. I feel it's both impossible and not necessary to accomplish that. The complexity we have to handle to use APIs is already minimal.

What makes hypermedia interesting IMHO is (minor point) easier intra-linking within a webservice, and (major point) _the ability within a service to point to resources from another one_.

 _This_ will enable the distributed web we are all waiting for.

<comment><AUTHOR> > Apart from parsing, I am the API client in this instance. I'm the one making > decisions from understanding what services are about.

Your browser parses, yes, but then it goes and fetches images, css, javascript, iframes, etc. Following the analogy, these are 'api calls'. It knows to make these calls because the HTML defined the links, and for the HTML to be rendered the contents of the links are required.

I maintain the browser is the client - it responds to your actions, but it does so with a lot more activity than just parsing.

>What makes hypermedia interesting IMHO is (minor point) easier intra-linking within a webservice, and (major point) the ability within a service to point to resources from another one. > >This will enable the distributed web we are all waiting for.

Couldn't agree more. But without some standards, those links to other services may not be as useful as they could be.

<comment><USER2> Sure css, js etc are fetched, but this is hard-coded behaviour, there is nothing dynamic or discovered about it. Or I fail to understand it.

This may well be the problem then, as we can see here and elsewhere. Many people like me don't understand. The claims are too vague. We need real-world exemples of how this can be applied with clear benefits. Mere transposition of the hypertext qualities to a medium intended for machine consumption will not do it for most of us.

<comment><AUTHOR> > Sure css, js etc are fetched, but this is hard-coded behaviour, there is > nothing dynamic or discovered about it. Or I fail to understand it.

It grabs the links from the content of the document it retrieved, using the rules of HTMLs structure to find them. That's both dynamic and discovered.

</comment></comment></comment></comment><comment><USER3> _"Proper" use of a REST or hypermedia API generally requires you only ever initiate communications with the API from the root (<https://my.api.example/> versus <https://my.api.example/some/other/url>). From there, every response from the API will then contains links to other resources, often with a relation to explain how each resource relates to the next. In this way, you never guess a URL - you are always given it._

Speaking from practical experience using an API like this: it can result in a lot of extra round trips to get to your final destination. The indirection is nice purity-wise, but in practice do you really want to make 3-4 times the round trips to the server for every API interaction? Especially when you might be on a mobile network? All for the sake of flexibility you may never need?

I'd say supporting the discoverability for learning, and consistency etc is nice. But in practice concerns like this (and the DHH example, and others already in comments here) will crop up and clients will start building their own direct URLs. So the idea that you'll be able to change the URL structure without affecting clients is a pipe dream.

<comment><USER4> > it can result in a lot of extra round trips to get to your final > destination.

You can usually get around this. Many people make their resources have too much hierarchy; there's nothing inherently unRESTful about having a flat one.

You should _not_ be having 3-4 round trips for every API interaction. Each request is an interaction. Hypermedia APIs expose a workflow, not a data model.

<comment><AUTHOR> > Hypermedia APIs expose a workflow, not a data model.

I would say this is true of REST APIs. Not necessarily hypermedia APIs. An API that directly exposed a graph database as an HTTP-based API using hypertext/links within resources would still need to be classified as a hypermedia API, but would fail to adhere to the HATEOAS principle.

Edit: More specifically - a graph database that exposed itself with links for every edge and a resource for every node.

<comment><USER4> I personally draw a distinction between "Hypermedia APIs" and 'hypermedia APIs.' The former is 'orthodox REST' and the latter is 'an API that serves up hypermedia.'

</comment></comment></comment></comment><comment><USER5> I'm confused about something. The author's "simple and easy solution" to the problem of people saving API URLs is to have a "bookmark relation" link; but all that means is assigning a unique, permanent name to the resource so people can find it again. Why wouldn't you just design your API so the API URL _is_ that unique, permanent name? The "bookmark" link "identifier" ends up being a URL anyway; why not make _it_ the API URL to start with?

<comment><AUTHOR> This is a fair point!

Mostly, I think that would produce an ugly API.

The reason I've been leaning towards that solution is that it assigns the responsibility of locating any other resource, in one request, with full backwards compatibility, to exactly one location, freeing other locations to change in a more fluid and graceful way without the cruft.

Edit: Oh and because other URLs can have the context of their parent paths deliver some value or information. But then again URLs should be opaque =). Definitely worth thinking more about!

<comment><USER5> _Mostly, I think that would produce an ugly API._

I don't see why the API would have to be ugly. The examples of "bookmark" link URLs given in the article are perhaps ugly (using query strings, etc.), but there's no reason why such URLs couldn't be "nice" ones. The article talks about things like "what if I want a URL structure like /User/projects/5 instead of /projects/5?", but that's a red herring: you could just as easily say "what if I want a bookmark query string that looks like ?user=User&project=5 instead of ?project=5?".

The bottom line is that as soon as you have a public API, you have identifiers that can't be easily changed because people need to be able to bookmark them; and those identifiers can't be too ugly because they are visible to users. So they should work OK as API URLs.

</comment></comment></comment><comment><USER6> I am having deja-vu. When I was just starting, everyone screamed about pure "semantic" markup and how all css class names should describe _what_ was contained in a node, and have nothing to do with how it was presented. People bent over backwards to comply. I couldn't make it work, but assumed I was just far behind the curve. A few years later and that roar is gone. Bootstrap.css for everything, with class names that describe structure. I can't say I'm hurt by this as it is what I had resorted to all along. But the absolute 180 without even a hint of the old battle cry in the air is startling.

I feel like that is again happening here. I _can not_ image how fully compliant REST api's would work in the real world. And when I ask for an example? _Twitter, check the Twitter API!_ But no, that breaks the rules all over the place in the name of practicality. _Oh yeah? You want a real example? How about the whole web? The whole web is your example!_. That example is so far outside of instructive or helpful I don't even know where to start. I stand agape, unable to even speak. The way clients access my blog system should be based on the whole web? Even though it itself is on the web?

I'm done chasing prophets. In five years I do not believe this REST mania will exist anymore, and not because it is so ubiquitous as to go unstated. It will have died or changed drastically in the name of actually working. Bootstrap.rest. Either I am too dumb, or the movement is crazy, or both. But either way, I'm done trying to make things in a way I don't understand. The great and holy can thump their Fielding Bibles with faith that they will be taken home soon. Me, I've given up on salvation, and find the road to hell a much less exhausting one.

<comment><USER4> > I can not image how fully compliant REST api's would work in the real world.

Have you ever used GitHub's API?

        $ curl https://api.github.com/

<comment><USER7> Oh! If that's the sort of thing we're talking about, then I'm all for it. The proponents of Hypermedia APIs have been saying a lot of vague things about how it will enable automated clients, and I assumed that the clients they had in mind would be able to do something useful with arbitrary APIs.

But it's clear from GitHub's API that it will do nothing of the sort: those link relations (and even concepts) are very much GitHub specific, and no machine will ever be able to figure out what to do with the them, what methods or media types the endpoints support, and so on.

(I'm not sure that it adds very much over a text document that that gives the relations and how to generate endpoints, but at least it's cheap to produce, and not actively harmful.)

<comment><USER4> > The proponents of Hypermedia APIs have been saying a lot of vague things > about how it will enable automated clients, and I assumed that the clients > they had in mind would be able to do something useful with arbitrary APIs.

Consider a web browser, or an RSS reader. These are generic hypermedia clients that consumer standardized media types.

> But it's clear from GitHub's API that it will do nothing of the sort: those > link relations (and even concepts) are very much GitHub specific, and no > machine will ever be able to figure out what to do with the them, what > methods or media types the endpoints support, and so on.

Right. We're not talking about AI, we're talking about generic re-usable components.

> (I'm not sure that it adds very much over a text document that that gives > the relations and how to generate endpoints, but at least it's cheap to > produce, and not actively harmful.)

The point is that by forcing you to define the communications protocol up front, you de-couple clients and servers. This means that they can evolve separately. Think about the RSS example: new versions of RSS clients can be made, and the servers don't need to be updated, and servers can update themselves and their responses and you don't need a new version of the client to handle it.

The other advantage is that if there are multiple services in the same product domain, and they use the same type, you get generic re-use of clients across services. Everyone spits out RSS, everyone consumes RSS.

<comment><USER7> RSS is a great example, because about the most sophisticated link relation in RSS is a link. But there seems to be a big difference in kind between what's being suggested for hypermedia APIs and the example of RSS.

With RSS, the client doesn't need to know very much--only one method is supported (GET), it corresponds to a click, and you'll almost always get one media type back (text/html). The IANA define some other generic link relations, but there seems to be a very big gap between generic link relations ("self", "back", and so forth), and application-specific link relations that are necessary for non-trivial apps.

How is GutHub's API useful to hypermedia libraries? A "library" that could be shared between GitHub and a different endpoint would be maybe 10 lines of code? HAL has a bit more structure, but we're talking maybe 100 lines of code. (Versus URI templates, which must be hundreds of lines, and with much more complicated rules.)

<comment><USER4> > But there seems to be a big difference in kind between what's being > suggested for hypermedia APIs and the example of RSS.

Well, RSS is just a simple example: its domain is small. Which is why it's nice to talk about. If you need bigger and more powerful things, than stuff gets complicated.

> With RSS, the client doesn't need to know very much--only one method is > supported (GET),

I should have said ATOM and ATOMpub; it has a slightly more complex edit/delete workflow as well.

> The IANA define some other generic link relations, but there seems to be a > very big gap between generic link relations ("self", "back", and so forth), > and application-specific link relations that are necessary for non-trivial > apps.

Right. The idea is that you use app specific ones at first, then, as you find they're useful and stable, you make them a generic one.

> How is GutHub's API useful to hypermedia libraries?

This is backwards. It's more like "how are hypermedia libraries useful for implementing the GitHub API."

<comment><USER7> > Right. The idea is that you use app specific ones at first, then, as you > find they're useful and stable, you make them a generic one.

Ok, but do you think there will will ever be a generic definition of "user" or "product" or "photo" that can access the full experience of e.g. Amazon, Google, Facebook, Flickr and GitHub?

Regarding Atom, Google adopted a extended version of Atom in for some (but not all) of their products, but no-one else adopted it in almost a decade, and Google are now deprecating it. (I'm sort of blurring the difference between link relations and media types, but they seem entwined.)

URIs and shared media types are great for interoperability, but my guess (we'll see if it comes true) is that pretty much anything beyond that isn't going to get traction. Hypertext was around before the web, but I think a big reason it took off is because had very loose interoperability demands. (e.g. links can break.)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> Is there a list of Hypermedia-esque APIs in use today (apart from sitemap)? I can find lots of explanations of the concept but it's hard to find examples.

The closest I can think of is the way some APIs handle pagination (GitHub and Recurly for example) by using Link: rel=next/prev/start headers rather than through GET parameters, which in principle would allow a generic REST client to iterate through results on its own.

<comment><USER4> GitHub does it for more than pagination: <http://developer.github.com/v3/pulls/>

Please see my comment here: <http://news.ycombinator.com/item?id=4949311>

Also, Balanced (YC W11): [https://www.balancedpayments.com/docs/overview#storing- the-u...](https://www.balancedpayments.com/docs/overview#storing-the-uri-vs- id)

<comment><USER9> $ curl <https://api.github.com>

We're experimenting with more :)

<comment><USER4> <3 <3 <3.

I really need to buy you a beer sometime.

</comment></comment></comment></comment><comment><USER7> A few thoughts:

* Mike Kelly (the author of the HAL spec) is not aiming for a universal API client <http://news.ycombinator.com/item?id=4949357> (and also seems to be under the impression that no-one else is either).

* A universal client seems impossible because APIs will typically need to use service-specific link relations for all "interesting" relations (e.g. product, author). You might be able to share generic link relations (like those in IANA's table: [http://www.iana.org/assignments/link-relations/link-relation...](http://www.iana.org/assignments/link-relations/link-relations.xml)), but sharing only these doesn't seem to be worth the effort.

* Hypermedia-powered experimentation is kinda cool, but you're still going to need something to drive it (see e.g. Kelly's HAL browser <http://haltalk.herokuapp.com/explorer/hal_browser.html#/>), at which point you might as well write a full-fledged API dashboard.

<comment><USER4> We already have a 'universal API client' called the browser. Therefore, implementing a NEW one isn't very interesting.

<comment><USER7> Sorry, I don't get what you're responding to. Browsers don't understand hypermedia APIs (or even HAL)--they only grok HTML.

<comment><USER4> > Browsers don't understand hypermedia APIs (or even HAL)--they only grok > HTML.

HTML is a hypermedia media type. A server spits it out, the client (a browser) consumes it.

A browser is a generic hypermedia API client, for services that expose HTML.

<comment><USER10> please can we not stretch the definition of hypermedia "API" to include HTML. HTML is a hypermedia type, but it's primarily for representing information for human consumption, so it's not an API in the sense most people understand.. why confuse things?

<comment><USER4> It's not stretching the definition: HTML and the web are where RESTful principles are defined from. They're the OG hypermedia API.

> it's not an API in the sense most people understand

Neither are hypermedia APIs in general.

<comment><USER10> I know where REST came from, my point is that - as far as normal people are concerned - HTML web apps are.. apps.. not APIs.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> The HATEOAS wonks are crazy, IMO. Most people who consume your API will have no idea what HATEOAS is and they sure as hell won't care. What will they care about? The URL to hit to get the data they want. They will hardcode that URL in their app, too, so expect any changes to it to require a great deal of fanfare to re-educate your users.

In theory, the URN solution could work, but it's far more complex and requires more maintenance than simply having URLs that can be relied upon. His solution to the problem of breaking bookmarks is also a lot of work for consumers of his API--they have to two codepaths: one for when the URL works like it should, and another for when it changes and they have to use hypermedia to find it again. Again, most consumers of the API aren't going to give a shit. They are going to hardcode URLs and complain when you break them.

Not only that, but consider the implications of an API that requires you to start from the beginning each time for mobile traffic. Each request over a mobile connection takes an eternity due to high latency. You're seriously telling me that you want to force users of your API to make N * depth of resource requests, instead of just going straight there, when each request takes 100 ms or more? Okay, so maybe you can cache it, but that just means the user's first impression of the app is that it takes 500 ms just to do the first thing. Pretty shitty first impression.

The comparison to a browser doesn't hold much water for me either. Users (and search engines) will save your URLs, so if you break them, you will lose traffic. If I try to go to my bookmark and get a 404, you really think I'm going to take the time to find your page again? Maybe, but it's much more likely I'm going to shake my head and close the tab. Of course Google will eventually find your page again, but it will have lost the SEO points it gained, which means you'll have to start rebuilding your SEO from scratch.

The other issue with the browse as hypermedia engine that he leaves out is the human brain. The human brain is excellent at extracting information from text data, so there doesn't need to be any kind of common standard or format to browse a webpage. The brain just figures it out that when you say click: [here], [here] is a link that goes to whatever you were just talking about. That dramatically reduces the burden on webpage authors to conform to any kind format. His theoretical (well, implementations certainly exist) universal REST client only works when every API you want to consume standardizes to presenting the information in the same format.

I drank the HATEOAS Kool-Aid when I first learned about REST too, and then I built a beautiful API that used it. What did the developers ask for? A list of URLs.

The future is almost never $elegantly_designed_complex_system. It's almost always a pile of garbage, with a few humans sitting around sorting through the garbage to find the thing they want.

<comment><USER4> > Most people who consume your API will have no idea what HATEOAS is and they > sure as hell won't care. What will they care about? The URL to hit to get > the data they want.

A few years ago, before Rails made "REST" popular, this exact same statement was made. "Nobody is going to want to learn about PUT and DELETE. They just want to do everything over POST."

> They will hardcode that URL in their app, too,

This is an education problem. We're still in the early days of this stuff.

> One for when the URL works like it should, and another for when it changes > and they have to use hypermedia to find it again.

This is a tooling problem.

> You're seriously telling me that you want to force users of your API to make > N * depth of resource requests, instead of just going straight there, when > each request takes 100 ms or more?

One thing that's nice about hypermedia is that you can lead clients to wherever you want them to go. You're free to change things at any time. Have a mobile app? Make less features, and make the request paths shorter. Have a desktop app? Make them longer, add more stuff. It's up to you. Nobody says that you have to complete N steps to do anything unless your server does.

> Users (and search engines) will save your URLs, so if you break them, you > will lose traffic.

Right. So what do we do on the web? We 301 redirect until we don't have enough traffic to care anymore. Same thing with your API.

> The future is almost never $elegantly_designed_complex_system. It's almost > always a pile of garbage, with a few humans sitting around sorting through > the garbage to find the thing they want.

Luckily, Hypermedia is the _opposite_ of Big Design Up Front. It lets you evolve over time. Partial application still yields benefits.

Please see my comment over here: <http://news.ycombinator.com/item?id=4949311> GitHub, BalancedPayments(YC W11), and others are finding real benefits today.

<comment><USER11> You might be right, as you've clearly thought about this more than I have. I have a couple of thoughts though.

> A few years ago, before Rails made "REST" popular, this exact same statement > was made. "Nobody is going to want to learn about PUT and DELETE. They just > want to do everything over POST."

> This is an education problem. We're still in the early days of this stuff.

The developers I was working with also had a lot of trouble with REST. I had to explain repeatedly what PUT, DELETE, GET and POST were (they weren't really familiar with HTTP at all), and that they weren't doing RPC calls. On the one hand, _I_ feel REST is a simple and elegant concept and its not much to ask someone to understand, but on the other hand, when I build an API, I want it to be easy to use and accessible to developers of wide skillsets and levels of experience. To me, sacrificing some of my desire for elegance is worth it if it reduces the level of "education" required.

> One thing that's nice about hypermedia is that you can lead clients to > wherever you want them to go. You're free to change things at any time. Have > a mobile app? Make less features, and make the request paths shorter. Have a > desktop app? Make them longer, add more stuff. It's up to you. Nobody says > that you have to complete N steps to do anything unless your server does.

Either I'm not understanding you, or this seems totally backwards to the way APIs should work. You build an API so that users can do things with their data/your service that you haven't thought about, or cover edge cases that aren't interesting to you. I feel like my API should enable people to build the things they want, not limit and encumber them so that I can have my pretty hypermedia. In other words, I'm not interested in leading clients places. I'm interested in where they lead me.

<comment><USER12> "The developers I was working with also had a lot of trouble with REST. I had to explain repeatedly what PUT, DELETE, GET and POST were (they weren't really familiar with HTTP at all), and that they weren't doing RPC calls."

You should try using the SQL analogy instead of RPC, as this is much closer to what a "traditional" developer might expect. With REST, one is not doing a _procedure_ call, but one is querying to create/retrieve/update/delete data. Of course, there might be some procedures behind it, but to a client it's just a data query (just like SQL has stored procedures of its own).

<comment><USER4> HTTP verbs do not map exactly to CRUD.

<comment><USER12> Of course they don't; but that's good enough for giving an intro to a "traditional" developer.

</comment></comment></comment></comment></comment></comment><comment><USER13> I dunno. It seems like the future is _arguing_ about Hypermedia. If the idea doesn't take hold ... is that _our_ fault as developers? It reminds me of the OpenID hype - engineers solving a problem that doesn't exist or, worse yet, solving a future problem that has yet to materialize.

If the answer to this is "educate people" then that's the wrong answer. Or maybe it's the right answer if you're an IT type that loathes their users. I know I might sound crusty about all of this but how many years are we going to chase an idea that only a few people seem to understand?

Can we just admit it, right here, right now: _all of this is academic_. We should be focused on extracting patterns and practices that provide value and _cycling on that_.

The rest is noise.

</comment><comment><USER14> I think these ideas would get a lot more traction if they were presented in terms of something like "Semantic REST" (S-REST?).

Basically, I'm thinking that the ultimate benefit to having somewhat standardized interactions with our web APIs (with more included metadata) is that they can be mined, discovered, and manipulated intelligently by algorithms.

It helps people, sure, but just like using contact microformats helps give the user a phone number they can click to call, it's equally if not more important that it gives a crawler an easy way to say "hey look, a phone number!"

This is the main criticism I have with DHHs comments on hypermedia: they seem to focus too much on the human consumer (and often the programmer) and not enough on the algorithmic potential.

<comment><USER4> "Hypermedia" was chosen at a term because it gets right to the core of what "REST" (made popular by Rails and other frameworks) misses from REST: the hypermedia constraint.</comment></comment>


Library of Babel - <AUTHOR> http://libraryofbabel.info/ <comment><USER2> For those who haven't read the Borges short story, this might seem a bit confusing.

For those who have, another book to read is _The Unimaginable Mathematics of Borges ' Library of Babel_.

It definitely confirms my suspicion that sometimes Borges wrote horror stories for anyone who touches combinations.

<comment><AUTHOR> Yes, quite right, I should have included some context. Here we go:

[http://en.wikipedia.org/wiki/The_Library_of_Babel](http://en.wikipedia.org/wiki/The_Library_of_Babel)

Original text, translated into English: [http://hyperdiscordia.crywalt.com/library_of_babel.html](http://hyperdiscordia.crywalt.com/library_of_babel.html)

</comment></comment><comment><USER3> Wow, I just found the refutation for P=NP in your library: it's at ... NOOOO, I accidentally closed the other tab damn should have written it down

<comment><USER4> Reopen the tab from your browser history :p

<comment><USER5> Cursed are those that search in private browsing mode.

</comment></comment></comment><comment><USER6> It should be possible to link to individual pages. ( So that one can show where the opening line of _Neuromancer_ or the first paragraph of _The Library of Babel_ is located.) But otherwise really cool.

<comment><USER7> On average, the link would be at least as long as the book.

<comment><USER8> Not if you let the first linked page be site.com/1, the next one site.com/2 and so on, I think.

<comment><USER7> Yes, even then.

Sure "1" and "2" are pretty short. And even "57834573495879436129386943" is pretty short.

But the average link would have so many digits, I couldn't post it in this comment.

<comment><USER9> The bookmarks would only need to map to the book locations - you would only need sufficient values for the number of bookmarked pages, not the entire range of possibilities.

</comment></comment></comment></comment></comment><comment><USER10> Very cool. First thing I did was to search for "it was the best of times, it was the blurst of times".

However, I notice that the site routinely locks up my Chrome browser. What client-side processing is it doing that causes everything to freeze?

<comment><USER9> Hey DanAnderson,

I'm the programmer of libraryofbabel.info - thanks for letting me know about this. Which pages is it happening on?

<comment><USER10> When I'm at the search results at [http://libraryofbabel.info/search.cgi](http://libraryofbabel.info/search.cgi) and I try to click the "Location" link for a result, everything freezes for about 15-30 seconds.

</comment></comment></comment><comment><USER11> First of all is there a source code you can share? Just in case something was to happen to the Library, we wouldn want to lose all the knowledge like we did with Library of Alexandria (Although I'm sure we can find all the missing manuscripts in this library :) )

Second, maybe it would be worth adding ability to flag pages/books but also include a flag that signifies that there is nothing really worth reading there. :)This way you could have people flag content that does not make sense in any known human language.

<comment><USER9> Hey mariusz - interesting ideas. for the first, I was new to coding when i started this project, so i dont really know how people go about sharing their code. I imagine I could put it on github or something like that - but I'm a little apprehensive that someone might be able to find a way to do something malicious to the site if they were to know its inner workings. So I'm not sure - I too think about the posterity of the library, though.

As for the second idea, the forum exists for librarians to share any sorts of discoveries they make in or thoughts they have about the library. But I would never say there could be a page with nothing interesting on it! After staring at these pages for a possibly unhealthy length of time, I can tell you that there's something interesting to be found in all of them.

and keep in mind what Borges said: "In truth, the Library includes all verbal structures, all variations permitted by the twenty-five orthographical symbols, but not a single example of absolute nonsense. It is useless to observe that the best volume of the many hexagons under my administration is entitled The Combed Thunderclap and another The Plaster Cramp and another Axaxaxas mlö. These phrases, at first glance incoherent, can no doubt be justified in a cryptographical or allegorical manner; such a justification is verbal and, ex hypothesi, already figures in the Library. I cannot combine some characters - dhcmrlchtdj - which the divine Library has not foreseen and which in one of its secret tongues do not contain a terrible meaning. No one can articulate a syllable which is not filled with tenderness and fear, which is not, in one of these languages, the powerful name of a god."

<comment><USER5> Putting the code up on, say, GitHub would also help with stopping people from doing malicious things to the site, because people aren't always mean and villainous, they can be nice as well and help out. One of the benefits is people fixing your code for you!

If you're worried that somebody will do something malicious... what is there that one could maliciously do? As long as you don't have, for example, credit cards on there, not much to steal then. Perhaps somebody is malicious enough to decide to take down your website for their perverse pleasure, in that case anybody can have their own, local copy of the library in case the internet- facing one goes down.

EDIT: What I meant and managed to completely fail to convey well in the first paragraph is that by obscuring the code, the vulnerabilities that you're afraid of people finding don't go away. And people can find them nonetheless. By opening the code, other people can fix vulnerabilities, etc. But keeping this paragraph in mind, to relate it to the others, seeing as how your library still exists, nobody seems to have bothered to try to destroy the library in the first place using their own means, so what difference will putting the source up make? :)

tl;dr: Security by obscurity? For shame. Put the source on GitHub! There's nothing to lose, and everything to gain. :)

</comment></comment></comment><comment><USER12> How long will it take for the takedown for copyright violation notices to start coming in?

</comment><comment><USER13> Neat. How on earth does search work? (Particularly the "with random English words" part)

<comment><USER13> Oh. Nevermind. Duh.

<comment><USER13> (Since the address is the content, just generate a random block of text that has whatever properties you want, and then "re-encode" it as an address. "Search" complete!)

</comment></comment></comment><comment><USER14> Is this actually valuable in some fashion to humanity? (not hating, serious question). It seems more efficient to store every book than to predict every book ever by holding GB's of nonsensical sequences. Or is this just a for fun thing?

<comment><USER15> "Others, inversely, believed that it was fundamental to eliminate useless works. They invaded the hexagons, showed credentials which were not always false, leafed through a volume with displeasure and condemned whole shelves: their hygienic, ascetic furor caused the senseless perdition of millions of books."

</comment></comment><comment><USER16> the best part is that you can search for a particular string.

</comment><comment><USER17> I notice that all books seem to be exactly 410 pages long.

<comment><USER9> The parameters of the books are mostly set by the story - you can take a look at the link klunger posted, or read a pdf here: [http://libraryofbabel.info/Borges/libraryofbabel.pdf](http://libraryofbabel.info/Borges/libraryofbabel.pdf)

</comment></comment><comment><USER18> How long did it take to generate all of the books?

<comment><USER9> I didn't pre-generate the books - that would have taken longer than the lifespan of the earth (that didn't stop me from trying, though!). The books are generated by a pseudo-random algorithm which uses the "location" as the seed. So you get random seeming text, but the same page is in the same place every time. I described some of the coding process here: [http://libraryofbabel.info/theory4.html](http://libraryofbabel.info/theory4.html)

</comment></comment><comment><USER19> Abulafia?

</comment><comment><USER20> I found an interesting tidbit:

        ...
        axropabbwz  xplxvzny,putmgqmcgbyftxqzdp
        uwdlwdnzdmxeynijv.oazyxminlztkcqmwer.m fi
        mkvchlofjdlmvriu lnqcghyzqaboxlicq taggnj
        ynqemzyope,.mvfahenmxfgtjxldonzygqpjbn.
        hc sfcadlbkn,ln,
        lcnjgtsufin the beginning god created the
        heaven and the earthhu

It tells me that this is at:

          0y8b3mgqtuhukt3gbh1p9fh97ua5hsgr92x3h3frswm7vswg2jauwq33c7kgeo019j16yyln5w1vohcniu0zk3rdwcqhzqrf21mime48njfsd4j3lojjpo2gxjbl8i39x3i84u6jfce0fyvimtlspgql8alqnyc4nu4ofsyezlkhb4m9ibuhdj1l5p6wyllp5hcw549fz32znr3dxg7zdo5qe05maufsl1iiurv758cavczrhk7o9zyql1pddw3xmu00dc87ia5vwd4knjclkhzajni38z4cldyhcw4d77yg3mm1ls0m16rummla284hg1uanpjsvxk93qkqjs5o0bl8n6q1j2u0g1jakvkz4aiu021bm6859akw2ewaknqyloy8f4ilgnhu2h7k5aph0yevs4r0pjbz2w9u2wzcr4l265xr7ollq796vvgrl8hqdqjrk618raj5y5w55r9z0k79t2reqohqym9vsjva71jaw23yjn2lnbkhnvw901hohcef8tdhdijxquts91wbdhsypr09r6lfjirgpckhgmxpsiffelivnrcfsuvft5rl9jg5tn9ecnmom2b05aloe9sofdkfpbjdmopd8md2rv1wihkepgt2pl7g3pexi5ppr4rukzjdoy6cno2xiu2nv61r8ho42p4nxkdgr5twvocmr2jk64isp4n1fl3e9h93869buan6ql2aouip6r8xmlh9lmu7xsfsy9m2n15jytglizhvt0h8vkui24kd7lxagpe2q9wrvmdd3e06lvbv6fw4kiyyhuykv9br3384sw25zs8ycd3uso3wf89vrao28ty7e38z19ox1hqvdjobsf3mjcsbeol8ggshhieg8zc2iuc1v0eau5g5vyj57eb968snxqy4ttlvtb43a67anwhhvexwvr0nirxsmdovfjnfikr2v5q87awohe171mbyth6qtmur5eua0q1khf1h0yznzztgxtrfp5y05sd7hsm86io79lwjvi9qj89n1v7y0v9vga800rq7kanb9jpoewq6f7q6vz5s1kigy4jwconz9ffc9y87v6eao3tug3qt9groij2i1glxszzr6dfzip3rvsx86bpfnlps4aeuw27tgjpcaqta3mrtdh97uh6y15foa357gytl6a826h7hxtss5jjavhq3phd4d0dby37j3y6w2ei5k9b45k4nvg3l34x3kku1m2o1fydfi8lqkqxpur5dtp59vrbzl1bpr5oxe28xy506scgk3nsbae2zl52kzvoja27eqn29z4d3lvitjtefeuieqqdy3qzun30cesbj88bp62kc1h26zb47j06k58m9r4sce9dk2sthhh9cnqt8zpkf3a6j1e55aoncslus4t7djh506a4j3h9cd8fr2w7e63pn4xtaqnm58tmx04gq6s668jw21k3i4ze40ivv1v5nvc78p9sbmrrb2myj6ftga5trstbabvdrmebs063guwnxonnvkoxmue5afxxstqtn6bjerg4h6387x6jeksdajgo55en5fvg70oeykfg6hy8zfffo204vyzncph5xxehstsv66k9fd3ywznohiwpt181dx3ua4p550fw8nzm8gfi3164uzxn629u8wbvccoi4vq0mjmy1niw8xtwetvlkjs5w7kwl38vn1zasjonsrb2asxq8jqta53xo7ek355vzyg61i9zr3ugxapxsq60e1zl2vljflazhw0q0f5t7ch8srh5k7gf86voqq3nrj92ovvh6k85kdexc8tlv7rl7djc407me6hj9reozhx3wntcugwr54t545i1apxgqbuug11vp9fu1tt3zwp5lvvkmznt66dh28s1x9upzmloqe6b7w28o8geb1uw78d3u1btza78zicleh70h757mnvw0rps0c3lt9mlswbgzwb18i6fa7t0x9d0gth3gpglaxbhgp6nvpdyrqhcq4659l13ybn2para0ott5entop8bspu8ajzlys07mohuvd0gyjzbqzaxh3q8u281xk44rnmqmgabd3sltvg1dsrzbsemcd9hjdipjyi8mdh9mf7uguw0xjyeps1xfiswdoxtbma5hkdxhj2ft2qd15zqi7bb9gccdsn1erqzrc9bxos9ups6oj2mnfwq6bmbx2gj9jg6jb1rozlypxdszbkcwkycwjpntkhw6x3p9optfgaqim3959qhiw2l1mh6yjiv4i1vn0lx6r4ru04w46edldcj60awb9ihceddwpvmlrvj232krvypgui16uktfcrukx7toixlmdzvl6gmgrw6q3mtqxto3dezljsuw88ri1muwgsnbnokhev444ihjr4zgzi1p2ghop2n3wvsb8i2v6wfwnsy86rk8urbkar43swhmcfghexieagnoe41seo8osfl4zy7i6eczsvq0d7jx2fe0ublz018xk8pftb79ccr4s5jr4itx6bubehm16e5fhxpgnuzbitdxa7a92pr6m6q4f6i3iu4c2dpdkxouxqhurfg80eez9nnj1p5tezt9zfqcf5nx8f4cxzrl7hqyv10fk0l00v7c62lct8c7mgckcr5hju8szmoqdk4zpu3qiof7854gtiglhdl2vs3146znsp1868ofhs3o5bb39m37oqo4foia0qym9nnlysqwl3yklo3yae9eit8ocgqnhpq3r118ajahacjcrcwbu5cufmmyq6awb47fcgurlfn7x09oyrtgpal2qaeizfz8b7xxd339yjeduxwgne1xckllqrrfh6axqcu0i2wxhh6u0gdx8lluo9filbhkr3626x3fh5bdv6fugdejgi9a1vxek9n27iyi1ttfbq09ugfwcgrxgtqxqct4edhcuv92jilzi2orgg1m0ffg8r3nlbai6gi3hw0ob2ch45xgkpt0cgv3mpzr4b52ywjw07scwrj3d8b3gtegxq2ujoq1hwk7fv3aqhiep1rl042srut4p0kntgup9180haudlb4uf209r0io90sqpgpyw18g4dzcadqi1x4svfznd112wb6poqxw07nm8nsj7kl5qveq3glf75wjj8fj5fptg0h0vz8q32acdgpp68ejd4nxwcbep7pq4y03tnk08liq0o4uwc3dmeisakcrwao1z1bxuuvn74stkhhib5nn1id008u7k8z39b7b302o392gcl424glzg6034p5i8e3fk964xebkf7g5l8

Volume 28 on Shelf 4 of Wall 4

But when I go directly there using browse, it doesn't seem to be the same text.

<comment><USER9> Hey hliyan! I'm glad you're deeply exploring the library - did you make note of the page number?

<comment><USER20> Sorry, I just saw this. Sadly, I've forgotten the page number. But I'm sure it's recreatable.

Thanks for this. The Library of Babel was one of those ideas that first got me thinking deeply about the nature of knowledge as a kid.</comment></comment></comment>


Running a 20 People Business from the Beach - <AUTHOR> http://blog.mobilejazz.com/running-a-20-people-business-from-the-beach/ <comment><USER2> Is it just me, or does keeping up with the 30 or so web tools they've outlined sound exhausting?

Being able to work remotely sounds great. But I can't help but wonder if the time saved not commuting all gets eaten up by project management overhead and just logging in to all the services every morning.

<comment><USER3> Exactly my thought! Many of these tools seem overlapping, and to keep these overlapping areas in sync doesn't sound like fun. Also, as you mentioned, logging into multiple places all the time is sub-optimal.

<comment><AUTHOR> Hi there, I work at Mobile Jazz and it actually takes me less than 1 minute to log in to every service (that's when I have to log in, cause most of the times I'm already logged in). In terms of Syncing, Asana does a nice job integrating several services via some extensions (like Harvest for example to start your timer directly from a task), also, most of the time we have a central document per project where we keep all the different links and references that are relevant to it. I work half of the time at the office, some days from my apartment in Barcelona and at least once a year I travel to Mexico and keep working from there while I spend some time with my family. Over the time I have get used to the remote working mindset and I must say that I find it very efficient.

</comment></comment></comment><comment><USER4> Business on a beach? I want to relax on a beach, not squint at a screen and work :)

<comment><USER5> These kind of posts sound like "sat on a deckchair on a beach with laptop in one head and a rum coconut in the other"

But I imagine they are more like "sat in a room in a house near the beach"

Same as a normal office, just a better view.

Well that is how it is for me living by the beach in the tropics. I can see the surfing point, I can see the parachutes pulled by speedboats, just a distraction and increases the desire to be "there" and not staring at a screen trying to fix bugs. Sometimes, I just move the table to face the wall.

And don't get me started about trying to work in a bar / cafe / restaurant near the beach. Mosquitos, noise, disturbance, tuts from staff that you are sitting there too long not buying enough.

Each to their own though. I suppose I am jealous of the posts that _imply_ you can work 4 hours on tropical island, earn a western wage and live like a King. Which is far from the case in my experience. Even if I know these posts are trying to sell something based on this dream, or convince themselves they are doing the right thing (despite what they may hear from friends and family)

I dear say some contractor with fantastic contacts and get a great paying contract with a couple of phone calls. Most people don't have this.

<comment><USER6> I tried working on a beach for a few weeks in Thailand once. With the mosquitoes, bright glare on the screen, sunburn, and the lack of anywhere to get a decent cup of coffee, I found it to be not nearly as pleasant as I'd imagined it to be.

It sure is a nice dream to have during rainy winters when you're stuck in a cold office though.

</comment></comment></comment><comment><USER7> I can't believe they entrust all their family jewels in Google services. My experience is that even when we used the paid Google services their customer support is at best atrocious ... try not having access to your company's most important collaborative documents for a few days ... never again for us.

<comment><USER8> What alternative would you recommend / do you use now?

<comment><USER9> I also use some google products but if you're worried about the _big prying G eyes_ and looking for a self-hosted replacement I guess sovereign is a good (and easy to deploy) starting point ....

[https://github.com/al3x/sovereign](https://github.com/al3x/sovereign)

</comment></comment></comment><comment><USER10> I wonder if this is being upvoted for the idea of running a company from the beach, or for the particular technology suggestions (everything Google + Hipchat, Asana, Trello, and Harvest).

I've been remote for a long time, and have had better results with a different suite of products.

<comment><USER11> What tools do you use?

I'm really interested in remote working and how to improve the day to day to work and communication.

<comment><USER12> Skype has bad quality from e.g. Thailand. Google Hangouts none of my clients (including myself) understand or can use. Still looking for a reliable and proper internet audio/video solution. I wonder if there's better solutions.

<comment><USER13> One of my customers (maybe two of them) use WebEx. They're paying for it but WebEx has a free plan to make a call with up to 3 people. The web client requires Java but a native client was preinstalled into the Samsung tablet I bought 6 months ago. Quality is good. Screen sharing seems to be better than with Skype and Hangout.

By the way, I had too many problems with Hangout so I can't recommend it, even if today I had a four people conversation (audio only) and it was perfectly fine.

</comment></comment></comment></comment><comment><USER14> I'd be interested to know how you go about getting new business, if you wouldn't mind sharing?

<comment><USER15> Hey, I'm one of the co-founders of Mobile Jazz.

Initially most of the work came through our personal networks. Myself I grew up in Germany and lived in San Francisco for some time. My co-founder had a strong network in Barcelona through his previous job at Deloitte. Today we still get requests through our personal network, however, most of our new incoming work is simply referral based. We don't even have a single sales person. So we try our best possible to do quality work, but also are actively involved with our clients and partners and help them improve their own businesses. This is not just an extra effort on our side, but also makes the whole "work" thing way more exciting and interesting for ourselves, rather than just being a sweatshop that churns out code.

We mostly work with startups, some of them as clients and some as partners. But we also have been recommended and introduced to some big corporations in various industries. And while we don't like the bureaucratic hassle that comes with it, it does give us quite some stability. For example, a big medical company just switched to use as their main supplier for web development, which immediately gave us 30 new long-term projects.

Overall we always try to give a lot, be involved and pro-active and help where we can. In the end you always get something in return. This has been my personal philosophy since ever and always and we've also established this as one of the cornerstones in our company culture.

<comment><USER9> Hi Stefan... your company culture sounds amazing. I see you're currently hiring (via API... lol). I'd like to know what can a candidate expect regarding salary. Do you have a reference/range?

<comment><USER15> It's difficult to give a general answer to this question, for two reasons:

1) Everyone at MJ can choose how much they work. The the salary is depended on how much you actually want to work. And that can change monthly and in extreme cases even weekly.

2) We share the company profits with all employees. We basically pay out quarterly bonuses, which are depending on the time worked, but multiplied with a "subjective" of how much you've contributed to the overall success and growth of Mobile Jazz.

Even though I can not give you a specific number, I can say that we pay very high for Spanish standards and slightly above average for Germany. But still below what a top engineering in SF would earn. But we might get there at some point. That said, it all depends on where you live.

</comment></comment></comment></comment><comment><USER16> I run a ~10 person completely remote SaaS company and that matches our tools exactly, even down to the way we set up our HipChat rooms* .

For MobileJazz or others that operate completely remotely, how often do you meet face-to-face? We assemble the team twice a year. It's definitely possible to run a company completely remotely, but I've still seen no remote substitute for the interpersonal bonding that happens in meatspace.

* We don't invoice, so no Harvest.

<comment><USER17> I work for a company that is completely remote, and we have an all-hands meeting 2-4 times a year. As an employee I find I need these meetings just to put faces to names, at the very least.

It also means you get a broader view of what everyone else is doing. It's easy to have a very narrow view of just the project you're working on, so you lose some of the cross-pollination of ideas between projects.

<comment><USER18> At Mobile Jazz we have a weekly meeting with everyone. We share what we have been working on and especially the learning experiences of that week, regardless if they are related to work or not. This helped us know each other a bit better and also speak about things unrelated to work.

Also we pair people from different projects to work together for a couple of hours per week, two different people every week. This helped a lot in learning from other team members but also in general helping each other more often, because we now know better who to ask when a certain question appears.

</comment></comment></comment><comment><USER19> I got a toast notification saying that one of their developers is in the top 3% in the world (according to stackoverflow).

Interesting.

<comment><USER20> Top 3% in stackoverflow doesn't mean much. SO says I am top 2% and doesn't make me feel special. I just answered a couple of questions through the years.

<comment><USER9> Agree. I wonder if it's top 3% _overall_ , or top 3% on a given period... During the end of 2014 I was in the top 0,06% and don't even consider myself a good dev. In fact I'm better at design...

<comment><USER15> Top 3% overall. And I agree. It doesn't mean much. Just that I've answered a lot of questions that people liked and upvoted. Nevertheless, it works for marketing.

</comment></comment></comment></comment><comment><USER21> I presume that you have to work with crappy internet connections in exotic places, how do you deal with it? It's interesting because, at Redokun, we switched away from Hipchat and migrated to Slack only because it worked much better on our slow/crappy connection.

<comment><USER15> Actually in many places in the world (e.g. Middle East, Asia and Africa) you've crappy landline Internet connection, but if you use your phone you get 4G/LTE with high-speed and relatively low latency.

</comment></comment><comment><USER22> The biggest challenge we found in working with a globally dispersed team, is the time difference - between say, India/Thailand and the US. When it is working hours in one zone, it is bedtime in the other. How do you deal with that?

<comment><USER18> Hey pitchups. You're right, this is a complex thing.

We try to have a minimum amount of overlap where we all work together every day. If you realize our offices are only in one half of the globe approximately, so this is usually possible. The trip to Thailand was a special case, for example, where we had to shift a bit the regular working hours in order to make it work. But the important part is that we could make it work :)

Also we try to have a lot of communication asynchronously via Asana tasks or Google groups for things that don't need urgent attention.

</comment></comment><comment><USER23> Mobile Jazz? Is that like radio on the internet?

</comment><comment><USER24> Just wait until they try Slack..

<comment><USER18> Ha! We tried it! But so far we're happy with HipChat.

One thing that makes Slack better for a company like us is the possibility to be in several teams at the same time. This is something HipChat should definitely look into!</comment></comment>


Show HN: AWS S3 outage test from across the world - <AUTHOR> https://pulse.turbobytes.com/results/55c8751aecbe400bf80005f2/

<comment><USER2> Is there any really way to design your application to handle S3 failures like this? S3's SLA has 99.99% availability, but is there a way to handle the 1% so your application is not affected? Options I can think of:

      1. Using a CDN to serve files can help in some cases
      2. On-prem systems may be able to use gateway-cached volumes and use the local disk cache vs S3

Other ideas?

<comment><USER3> Slightly OT, but there's an interesting phenomenon at work now that so much of the internet depends on Amazon's infrastructure. When it goes down you might not even need to worry about it that much, as so many sites/apps will be broken that most users will just assume that the internet is broken.

<comment><USER4> It happened a few times before. And no Internet was not broken and not many sites are using AWS as you would assume. A lot of services still run on their datacenters or on-premise servers, or maintain a hot backup that they can switch to immediately.

>> When it goes down you might not even need to worry about it that much

I'm afraid this is pretty much getting the entire cloud thing wrong.

<comment><USER3> _lot of services still run on their datacenters or on-premise servers, or maintain a hot backup that they can switch to immediately._

I'm not an idiot, I'm well aware of that. My point is that when a large number of consumer-facing sites go down, users (who aren't aware of Amazon cloud servers) simply assume something is wrong with the internet.

Obviously if you have a mission critical service this isn't acceptable. But for a lot of average sites/apps it might not be worth the investment in time/effort to cover relatively small outages such as these.

 _I 'm afraid this is pretty much getting the entire cloud thing wrong._

Not really. It's the utility of the cloud - if there's an outage there are already a lot of people working to fix it. If you're self-hosted, that's on you.

<comment><USER5> The first thing I do when Netflix goes down is complain on Facebook. Facebook up, Netflix down? It's not the Internet that's broken.

</comment></comment></comment></comment></comment><comment><USER6> "Oh no, our server made a boo boo. Please try again."

<comment><AUTHOR> Pls try again. The server had crashed due to "too many open files". I'm leaking file descriptors somewhere.

<comment><USER7> You should fix that error message too, it's pretty awful.

</comment></comment></comment><comment><USER8> I just re-ran the test and got an error rate of 1.23% (vs. 41.98%): [https://pulse.turbobytes.com/results/55c88a0fecbe400bf800073...](https://pulse.turbobytes.com/results/55c88a0fecbe400bf800073e/)

edit: S3 seems to be back up and running according to the AWS status page.

</comment><comment><USER9> Yeah, I got 2.53% error rate, but that's nothing to worry about - using 79 servers, that's exactly 2 errors, both of them in China, which kinda makes it feel less Amazon's fault, than the Great Firewall's.... Maybe there should be some more meaningful error measure than the raw failure percentage.

<comment><USER10> Vancouver, Tokyo, Cebu, Singapore, Bangkok, and Kharkiv all show EOF.

Paris, Roubaix, Manchester, Portsmouth, Budapest, Tokyo, Bilthoven, Vleuten, Utrecht, Manila, Sovetskaya Gavan, Tyuven, Singapore, Bangkok, Taipei City, Kiev, Kherson, Ashburn, Mountain View, Rowley, amd Newark all show 503s.

It's more than just the Great Firewall. :-)

<comment><USER9> It depends, you have to re-run the test...

</comment></comment></comment><comment><USER11> Got some alerts from a couple services that rely on S3 this morning. Perhaps this is related, but everything is back up for now.</comment>


Linus Torvalds: "So please don't stop." (2004) - <AUTHOR> https://lkml.org/lkml/2004/12/20/255 <comment><USER2> I feel the same was as Linus, which is why I started the Trivial Patch Monkey (and hence my mention in this post). Note that it's still going, as mentioned in MAINTAINERS:

      TRIVIAL PATCHES
      M:	Jiri Kosina <firstname.lastname@example.com>
      T:	git git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial.git
      S:	Maintained
      K:	^Subject:.*(?i)trivial

</comment><comment><USER3> This is a mailing list post about the values (and costs) of submitting "trivial" patches to the Linux kernel, particularly from new contributors (in case this headline wasn't sufficient to give you any idea _whatsoever_ what the post is about).</comment>