Getting hyper about hypermedia APIs - <AUTHOR> http://37signals.com/svn/posts/3373-getting-hyper-about-hypermedia-apis <comment><USER2> I wrote the spec for application/hal+json that got compared to WS-star , here's where I'm coming from:

JSON doesn't have links. Establishing some basic conventions for that makes complete sense. Defining those conventions is called a spec. Giving a payload that follows conventions a name also makes sense. Establishing that is called registering a media type identifier.

It makes no sense to keep reinventing the linking wheel in every API . Pretending like a very minimal media type like hal+json is akin to WS-* is incredibly disingenuous and/or stupid.

Establishing a standard media type like hal+json with a bunch of conventions allows us to build generic tooling that can help with both serving and consuming payloads that contain links.

Being pragmatic is great, but misrepresenting a genuine effort to improve the status quo and improve the API ecosystem in a reasonable, non-complicated fashion as 'hand-waving' or 'unnecessary' is not very constructive.

<comment><USER3> Are there any examples of what, for example, the Flickr or Twitter API would look like under hal+json?

I'm struggling to see how application/hal+json would help me write a client to upload a photo. I imagine that with or without hal I'm ultimately going to POST the data to some endpoint. The question is, how do I figure out what endpoint to use?

Without hal+json I need to read the API docs to discover that "/photos/" is the correct endpoint to post to. But with hal, it seems I still need to read the docs to discover that within the hypermedia file, the key "photos" holds the value for the URI template. So either way, I need to read the docs, and either way, my client breaks if the meaning of the string "photos" changes.

<comment><USER2> Your applicaiton's link relations should be URLs. This means that you can expose each bit of documentation for each rel at their URL. So every time you see a link in a hal+json document, you can follow the _rel's_ URL, and fetch the documentation for it.. that turns out to make the API very easily discoverable, and gives you a way to model/manage your API's documentation in a consistent way.

You can actually see what that looks like in practice here (click one of the book icons on one of the links on the left):

<http://haltalk.herokuapp.com>

<comment><USER3> Do you mean like <http://haltalk.herokuapp.com/rels/signup>? But every site will have different requirements or mandatory fields on signup, so there's still no "discoverable" way to sign up to Amazon and eBay and Google without a human looking at <http://amazon.com/rels/signup> and <http://ebay.com/rels/signup>, etc., in which case you're back to still needing to read the documentation.

I guess it's a slight improvement to know exactly where the documentation for a particular link relation will live instead of having to search for it, but it doesn't seem to solve an especially difficult problem.

<comment><USER2> it is an example of a discoverable API with discoverable documentation though, which is what you were asking for I think.. maybe not?

<comment><USER3> Ah, maybe I misunderstand what you're trying to do. I thought Hal was trying to make it possible to write clients that can buy from Amazon or eBay without knowing anything about the specifics of either. But actually what it's attempting is a sort of machine-assisted documentation system? (Since you still need a human to read the docs to find out how to signup, place orders, and so on.) I can see how that might work, but it doesn't seem like an important problem to solve.

<comment><USER2> That is one of the benefits that relates to documentation, which is the part of your comment I was responding too.

The larger goal of hal+json is to establish some conventions for linking that allow the development of generic tools for doing hypermedia. Not to create magical machine clients that can interact with any random API you point them at. Nobody made that argument so I don't know why DHH addressed it in his post. I'm guessing he ran out of things to be an angry-pragmatist about.

<comment><USER3> Well ... the spec does say "HAL is a bit like HTML for machines, in that it is generic and designed to drive many different types of application." To me (and perhaps DHH), that suggests that it's designed to be used to create clever clients. Perhaps you can clarify this point in a future edition of the draft?

<http://stateless.co/hal_specification.html>

<comment><USER2> I have no idea why you would conclude that from that sentence. HTML allows browsers, the same way that HAL allows generic libraries. I'm struggling to understand why this is controversial, there are already 16+ examples of generic HAL libraries across a bunch of different languages.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> DHH's API philosophy (send simple JSON serializations over the wire, mostly from the server to the browser) got him through the 2000's OK, but the longer he argues against hypermedia, the less relevant Rails becomes for API design.

These are not particularly good arguments he puts forth here. I believe they are in response to Mike Kelly (designer of the HAL+JSON hypermedia format) and his recent post: [http://blog.stateless.co/post/38378679843/hypermedia-apis- on...](http://blog.stateless.co/post/38378679843/hypermedia-apis-on-rails-why- dhh-should-give-a-fk)

DHH's comment that URLs instead of IDs are a good idea is true (but he even gets that part wrong in how he implements it, leaving a '.json' extension on the URL). And the rest of the article is trying to hand-wave away the value of hypermedia (links -- all hypermedia means is links, at its core).

And comparing HAL+JSON, a blessedly lightweight standard, to WS-* is just dirty.

This post didn't convince me of much other than that DHH may run out of steam on this issue before too long.

<comment><USER5> _These are not particularly good arguments he puts forth here_

Would you care to explain why you think they're not particularly good? You're blaming DHH of being "hand wavy" but I don't see any more depth from your comment.

<comment><USER4> I spell out some concrete advantages to hypermedia APIs elsewhere in these comments. <http://news.ycombinator.com/item?id=4948652>

But to specifically address DHH's arguments from the article:

* Enabling Discovery is a strawman argument; no one expects an API to use / as its only documentation (at least no one expects it of a good API).

* Standardizing API Clients is also a strawman; no one expects to have one generic client magically make sense of any API.

* Comparing HAL to WS-* in order to paint it as committee-driven standards bloat is not fair to HAL, which is an admirably tight and cogent specification.

</comment></comment></comment><comment><USER6> Bias alert: I'm the newest Rails committer and one of the bigger proponents of Hypermedia APIs in the Ruby world.

Anyway, many, many other companies _do_ find hypermedia principles to be useful. See Balanced Payments, for example:

> Fun fact: our internal statistics show that client libraries that construct > the uri receive roughly 2 orders of magnitude more 404 status codes from > Balanced than clients which use the uri directly. > > > [http://www.theatlantic.com/magazine/archive/1999/03/the- > mark...](http://www.theatlantic.com/magazine/archive/1999/03/the-market-as- > god/306397/)

GitHub is starting to add hypermedia stuff to their responses:

<http://developer.github.com/>

Here's their main API guy talking about the advantages even this partial implementation has achieved: <https://twitter.com/pengwynn/status/281849041707474944> <https://twitter.com/pengwynn/status/281849329243787265>

Twilio has always had elements of hypermedia in their API, and are considering moving further in that direction in the future. They had me speak at their conference for the last two years in a row about the topic specifically.

That said, if not doing hypermedia doesn't hurt you, don't change what you're doing! If you're interested in evolving your API over time while supporting old clients and behaviors in a simpler way, then consider checking it out. REST/hypermedia APIs are focused on long-term stability, evolvability, and massive scalability. If you don't need those things, you don't need hypermedia.

That said, I'd be happy to answer any questions on the topic, though I'm really busy today, so it might take a while to get back to you.

<comment><USER7> What are the best options to implement that for an API backend with Rails <-> a javascript library/framework for the interface?

<comment><USER6> The current Rails Party Line is "Jbuilder + you don't need integration" <https://twitter.com/dhh/status/281802247480958976> <https://twitter.com/dhh/status/281802391316201472>

Some members of the core team and I have started a "Rails API" project specifically to explore these possibilities, and to extract common patterns from the apps we build: <http://github.com/rails-api>

This will encompass hypermedia and non-hypermedia APIs, as well as SPAs with a Rails backend. Most of us are focusing on Rails 4 at the moment, and will start working hard on this post-Rails 4 release, but about 23,000 people have installed the main gem, and I know several running it in production already.

I personally feel the best option right now is "Rails API + ActiveModel::Serializers + Ember.js". But we want to encourage a multiplicity of options. Not all apps are identical.

<comment><USER8> We also have developed custom Python tools and frameworks that help us with our API at <https://balancedpayments.com/>

If there's enough interest, I'd love to share some of our ideas with the Rails community and discuss some potential downfalls and successes we've had and why we needed to build these tools internally.

I'm sure Rails would benefit a lot from these.

Love love love the hypermedia work -- keep it up!

<comment><USER6> That would be fantastic. If you find the time, we have a discussion list over at [https://groups.google.com/forum/?fromgroups#!forum/rails- api...](https://groups.google.com/forum/?fromgroups#!forum/rails-api-core) , please feel free to share thoughts there!

</comment></comment></comment></comment></comment><comment><USER9> His questions about discoverability seem to assume that people are suggesting not writing any documentation at all.

"The idea that you can write one client to access multiple different APIs" is a straw man.

The connection he's making between HAL and WS-* is ridiculous.

</comment><comment><USER10> hypermedia APIs mean that our API's basically become web pages and every client should act like a browser. That's great if you want... a browser app.

If you want to just expose data to let people build interesting things with it, hyperlinks in your api are a bit silly. They might be nice if you want someone to write a browser for your api, I don't see where else it would be awesome. Maybe if you want a web crawler to crawl your api.

Also, they add a layer of chattiness to your app. If you have well documented, unchanging url's people can write apps against those. If people have to go to a resource to find the url to another resource, you're going to have a lot of API requests just to look up URL's. People will realize that is a waste of bandwidth, and will hard code URL's anyway.

I rarely agree with DHH, but he's right.

<comment><USER11> So you could write an app against the API of some service, then a competitor of that service comes and implements the same API, and so you want to switch.

What should you do? Replace every URL? What if you want to support _both_ (say, Twitter and Identi.ca)? Should you implement a map from codename ⇒ url?

Now what if you want to support _every possible implementation_ , even if you - the developer - don't know about them? Why shouldn't the user be able to plug-in the entry URL and use your app?

People snickering about hypermedia APIs seem to me like people ridiculing the idea of having standard ports and protocols for devices, because they can't imagine a world where you don't have to install yet another crappy 200MB driver that is only available for Windows 95 to use a damned mouse.

<comment><USER10> I think that assumes some serious hand waving on the part of both your app and the api's. Assuming that just pointing at the root of another API is going to mean your app will magically behave properly using both API's is making a lot of assumptions either in your app about how the API's work. For example, say one API calls a delete call "delete", another calls it "remove", another calls it "obliterate", and at the same time one API takes an id on that method as input, another takes an email address, another takes a username. In that scenario your app might be written to remove something based off an id and when you switch API's the new one expects you to obliterate based on username. Hypermedia doesn't save you there.

I'm all for pluggable protocols and standards, but Hypermedia links in your API don't magically get you there. They might help, but no more than a standard spec with standard URL patterns would.

</comment></comment></comment><comment><USER11> _According to hypermedia lore, you will be able to willy nilly change your URLs without needing to update any clients. But that's based on the huge assumption that every API call is going to go through the front door every time and navigate to the page they need. That's just not how things work._

This is missing forest for the trees. The point is not that you, the API implementor, will be able to change URLs willy nilly. It's that I, the client, can support a _different_ API implementation by just changing the entry point URL, without changing the application.

So, if my application supports the API that Flickr implements, and tomorrow someone creates Blinkr, which implements the same API, the user could just copy-paste the entry point URL and use it, just like I use my RSS reader for all the blogs and news sites out there.

Of course, this depends on using standard document formats and a restricted, standard set of methods. Rings any bells?

 _Thinking that we can meaningfully derive all that by just telling people to GET / and then fumble around to discover all the options on their own just doesn't gel with me._

Well, am I glad that straw-man was burned to the ground!

<comment><USER12> > and tomorrow someone creates Blinkr, which implements the same API,

So the benefit of hypermedia apis is predicated on something happens that almost never ever happens?

<comment><USER11> On the contrary; REST, of which hypermedia is a constraint, is an architectural style derived from the observation of a very successful implementation of just that.

It's called HTTP + HTML, and there are millions of services implementing it.

</comment></comment></comment><comment><USER13> This post seems hyper-defensive and hyper-reactionary. URLs are _fundamental_ to HTTP web services - they're how the web works. Imagine if you visited a website in your browser and instead of link to the various pages, the page just printed IDs and you had to copy/paste them into the address bar after the domain name.

</comment><comment><USER14> Sounds like a redefinition of HATEOAS. I don't put URLs as my ID's, but every resource has an "href" value which is the URL which makes it somewhat discoverable by tools such as <https://github.com/jed/hyperspider>.

<comment><USER6> "Hypermedia APIs" are just "Real REST." Arguing over terms isn't productive, so many RESTafarians just stopped: <http://blog.steveklabnik.com/posts/2012-02-23-rest-is-over>

HATEOAS sounds big and scary. "include some links" is much more understandable.

</comment></comment><comment><USER15> I use <http://www.remobjects.com/ro/>, and have something similar, called RODL Files (<http://wiki.remobjects.com/wiki/RODL_Files>), this is how look like:[http://wiki.remobjects.com/wiki/RODL_Library_(Service_Builde...](http://wiki.remobjects.com/wiki/RODL_Library_\(Service_Builder\)). This are like SOAP, but better. With them, I can parse the RODL file, and build a python client automatically in seconds, with a python script that output a python client for the remobjects server, with all the class, method calls and that stuff. RemObjects also generate automatically the clients for .net, delphi, js, obj-c, php ([http://wiki.remobjects.com/wiki/Generating_Interface_Code_fr...](http://wiki.remobjects.com/wiki/Generating_Interface_Code_from_Your_RODL_\(Xcode\))). Is something I miss very much when try to do a REST server without RO.

And the documentation is embebed in the RODL file, so is possible to output the client with the docs inline, and get it to show in the IDE when a call is made to the python client...

So, I think a meta-data about the service is VERY usefull. But the hypermedia is a poor attempt at that (IMHO).

</comment><comment><USER16> I think these are good points for general public facing web APIs that are specifically designed for being mashed up or will be consumed by third-parties that will never talk directly to your company. I feel hypermedia lore has some really worthy ideas for APIs designed for business use where workflow and data integrity are under strict control but will change frequently and where several client apps will be made by the same company or companies working closely together.

</comment><comment><USER17> Wow. Very surprising (in a bad way).

Basic Web architecture: 1. expose resources, 2. resources have names (URL), 3. allow basic actions on these (GET, DELETE , PUT, POST ) as needed. 4. Include URLs (as links/forms) in representations.

OK, _now_ build your API . Please do NOT start w/the API and work back to basic web architecture. Servers should _always_ provide URLs (url templates are fine), NOT the client (by way of snowflakey construction algorithm).

</comment><comment><USER18> Hypermedia hype has always struck me as parallel to the "semantic web" nonsense from the late 90s.

"If we just use RDF triples to encode everything then machines can learn that apples are fruits and fruits are good for you, thus apples are good for you! Huzzah!"

Then, Microsoft invented SOAP.

"If we just have a WSDL that explains all of our API, then we can have automated methods to communicate between services! Programmers can just auto- generate code and life is great!"

Except it doesn't work that way. Most WSDL parsers auto-generate code that you then hand-edit and maintain over time. And, you, as the programmer have to _know_ what methods to call. SOAP is just excessive ceremony transmitted over XML - another excessively rigid structure.

So then we got JSON REST APIs. Simple text structures. Reasonable defaults. Basic vernacular. Easily understood.

What concerns me about Hypermedia APIs is that folks are using the same sorts of grandiose, architecture astronaut-y stuff that we got out of the last two failed revolutions.

Hypermedia API proponents say that REST APIs are "highly coupled" (to the data model and versioning) and don't expose workflows.

Heck, that's why REST is so pervasive - they're super easy to write and consume/interact with. Enterprisey folks are so focused on long-term extensibility and maintainability that they overlook the cognitive overhead and inability to work with it on a daily basis. And that you in fact, tend to move _slower_ in development because you can't comprehend or follow what is going on. And that all abstractions leak, leading to libraries and tools (SavonRB) that don't quite work if an API doesn't follow the spec exactly (and they never, EVER follow the spec 100%).

My attitude is the opposite: _Don't_ design your APIs as if you expect them to be the next 1,000 year reich. If your API stays small and nimble enough, your consumers will also be able to be flexible to accomodate it. Yes, if you have a User REST API and you decide that your app no longer has users, well, then you have to get rid of it. But adding another layer or two of hierarchy and ceremony on top via Hypermedia wouldn't solve that either! Fundamental universe changes ought to break shit!

From what I've seen (I have a subscription to designinghypermediaapis.com which is very well written) Hypermedia APIs are an over-complicated solution to a problem that has a reasonable solution. Yes, links are nice. Want to propose a "Standard" to handle links? Okay, although that's REST! If you have an Object and you want to DELETE it, I don't need to know the URL. I have a convention via REST that allows me to derive it from my data. If that's different, OK, use this thing. But Hypermedia APIs are a _lot_ more than just links (state machines, workflows, media types, etc.)

<comment><USER4> _What concerns me about Hypermedia APIs is that folks are using the same sorts of grandiose, architecture astronaut-y stuff that we got out of the last two failed revolutions._

You owe it to yourself to read the HAL+JSON specification: <http://stateless.co/hal_specification.html>

Anyone who considers a three-page spec like that astronautics probably has a low capacity for complex thought.

<comment><USER19> A hint: when trying to convince, it's not a great idea to passive-aggressively insult the other party. Since they're human too, they will probably just get pissed off and write you off as an asshole. Like what I'm doing right now.

<comment><USER4> Given the level of respect I have for the OP's exhaustively researched points, I should not have said a thing. Someone on the Internet is wrong. My mistake.

</comment></comment></comment></comment><comment><USER20> Aren't links also proper global IDs, allowing us to address objects cross- service ?

<comment><USER6> Yep. URIs are 'universal.'

<comment><USER20> I feel there is a lot of noise around IDs as URIs yet I've seen nobody (I may not have looked deep enought) talk about their main benefit as I perceive it, which is putting resources into the perspective of a global namespace. Thus allowing a service to reference resources provided by another service. _This_ to me, looks like a practical revolution. We may at last have our contacts served by one entity, our statuses by another, our shared files by still another, and yet all of them could be handled as one distributed API.

The code glueing the whole to produce, say, a distributed facebook-like app, would still have to know the specifics of each provider (before standards emerge), but it would effectively break the isolation and allow us to composite services much more easily.

Where am I missing the point ?

<comment><USER6> > their main benefit as I perceive it, which is putting resources into the > perspective of a global namespace.

Sometimes people do. I mentioned this problem in my talk at Øredev this year: <http://oredev.org/2012/sessions/designing-hypermedia-apis>

> Where am I missing the point ?

Well, you're on HN: everyone wants to control everything. Startups don't 'win' by playing nicely with each other, they drive their competitors out of business.

</comment></comment></comment></comment><comment><USER21> One neat trick that I think is worth adding to the discussion: URL templates.

Returning URL templates as part of your API response can give you the benefits of having a clean way to access sub-resources, without the headaches and bloat of having to enumerate every possible desired sub-URL.

For example, in DocumentCloud, a document's canonical representation has a unique URL for the content of every page as plain text, and as an image, in several different rendered sizes. Instead of doing something silly like this:

        resources: {
          text: [
            "http://www.documentcloud.org/documents/1/pages/page-1.txt",
            "http://www.documentcloud.org/documents/1/pages/page-2.txt",
            "http://www.documentcloud.org/documents/1/pages/page-3.txt",
            ...
          ],
          largeImages: [
            "http://www.documentcloud.org/documents/1/images/page-1-large.png",
            ...
          ],
          thumbnailImages: [
            "http://www.documentcloud.org/documents/1/images/page-1-thumb.jpg",
            ...
          ]
        },
        ...

... where you might have 5,000 pages in a document, you can imagine how unacceptably large that response might become. Instead, a single URL template can do the work. (<http://tools.ietf.org/html/rfc6570>) The spec has a whole bunch of goodies in it, but we just need the most basic interpolation feature for this case (real example, you may have to scroll sideways to see the complete URL):

        "pages": 5058,
        "resources": {
          "page": {
            "image": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}-{size}.gif",
            "text": "http://www.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}.txt"
          },
          "pdf": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.pdf",
          "published_url": "http://documents.nytimes.com/sotomayor-s-senate-questionnaire",
          "related_article": "http://www.nytimes.com/2009/06/05/us/politics/05court.html",
          "search": "http://www.documentcloud.org/documents/21939/search.json?q={query}",
          "text": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.txt",
          "thumbnail": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p1-thumbnail.gif"
        },

Basically, all of the simple URLs a Viewer might need to use in order to browse the document, search the text, and view related resources. In the past, when we've needed to change or expand the number of resources (adding HTTPs- only support, changing the URLs at which the page images are stored, or adding larger sizes of page images), it's been relatively easy to do, without breaking the viewers, or invaliding previously-valid JSON representations of the document. Here's the complete link to the above:

[http://www.documentcloud.org/documents/21939-sotomayor-s- sen...](http://www.documentcloud.org/documents/21939-sotomayor-s-senate- questionnaire.json)

<comment><USER22> > One neat trick that I think is worth adding to the discussion: URL > templates.

100% agree - URL templates enable succinct discoverability. Moving link generation to the client reduces computation time on the server which no longer has to generate inline links as well as the payload size of the response.

The RFC6570 spec is great, has numerous implementations[1] and a through test suite[2]. I am also surprised that this is not a standard part of every hypermedia implementation.

[1] <http://code.google.com/p/uri-templates/wiki/Implementations>

[2] <https://github.com/uri-templates/uritemplate-test></comment></comment>


Pandora misses revenue and profitability on 81.3m quarter - <AUTHOR> http://techcrunch.com/2012/03/06/pandora-misses-revenue-up-71-percent-to-81-3m-but-swings-to-a-loss/

<comment><USER2> I love Pandora, I use it a lot - but I don't pay for it. :(

I hope they do well - but this is a service I don't have a budget for, even though they are one of my favorite startups.

<comment><USER3> I pay for it (since they're right down the street, and I wanted to support them), but only ever use it while working on servers in a datacenter (with wifi for the phone) while wearing headphones -- for at-home listening, I prefer my own collection, and for driving or other mobile use, ATT coverage isn't good enough to keep it streaming.

Somehow I have a station per datacenter. A location-specific music service would be pretty awesome.

<comment><USER2> Thats a brilliant idea: GPS controlled playlists.

When I am [en route to work, and the time of day is X], play this.

When I am [at work] play these

etc...</comment></comment></comment>


NodeCloud: Node.js resources - <AUTHOR> http://www.nodecloud.org

<comment><USER2> I just picked up node recently and found this genuinely useful - for me at least, it doesn't matter if it's ranked in exactly accurate order or not.

In fact, I really wish I knew of more lists like this. Every time I start learning a new programming language, there's a period where I go shopping for useful sites and tutorials, usually resulting in a lot of wasted time with bad tutorials and unhelpful sites. If I knew of a one-stop ranking system like this, I would make use of it.

</comment><comment><USER3> I'm not big fan of the name (I would assume it's another node.js SaaS provider), but otherwise it looks like a great directory for node.js beginners.

<comment><USER4> Yeah its quite close to our first name: <http://www.nodejscloud.com> :)

</comment></comment><comment><USER5> I finally had a chance to play with node over the weekend. First impression is that the library ecosystem is way bigger than I'd have thought and documentation is generally very well done.

Node really seems to have captured the imagination of a lot of people. I'm looking forward to using more libraries and building out the toy app I started.

</comment><comment><USER6> Very newbie-oriented/general, but nice to have a place to point to to get people started.

</comment><comment><USER7> Could use some user interaction on-site for discussing node; this would be a good place to showcase node based chat / social apps by integrating them into nodecloud.org

<comment><USER8> yes, i was expecting a forum for node :D

</comment></comment><comment><USER4> Just an idea, maybe allow people to login with Twitter, and add comments, and simply click 'I Like', and that score effects the position as well.

</comment><comment><USER9> Please dont use Alexa for ranking sites. Their numbers are almost always wildly inaccurate. The downside is there aren't many better alternatives.

<comment><USER10> sounds like a place to start a business!</comment></comment>


digiKam 5.9.0 is released - <AUTHOR> https://www.digikam.org/news/2018-03-25-5.9.0_release_announcement/ <comment><AUTHOR> Even thought I prefer geeqie [0] I always look forward to the new digiKam release. Managing your photos in linux is tedious to be frank, I run my lightweight system and mostly manage it through shell scripts. If digiKam was more lightweight I would certainly use it again, but using this full blown kde app gave me more than one headache (not to say that digikam is not great if someone asks me what program he/she should use for managing photos I wouldn't think twice)

I tried every alternative I could find f-spot [1], shotwell [2], gthumb [3] and some more. Some alternatives without packages or updates in the last 5 years like mapivi and jbrout I didn't even try. Digikam was unstable at times and sometimes a bit on the slow side, but functions wise it went above and beyond anything the other programs had to offer (the only feature that was missing was sorting images by orientation and ratio). If at any point in time I'll be looking for an alternative to my current solution the first thing I will take a look at will be digiKam.

[0] [http://geeqie.org/](http://geeqie.org/)

[1] [http://f-spot.org/](http://f-spot.org/) (last update 2010)

[2] [https://wiki.gnome.org/Apps/Shotwell/](https://wiki.gnome.org/Apps/Shotwell/)

[3] [https://wiki.gnome.org/Apps/Gthumb](https://wiki.gnome.org/Apps/Gthumb)</comment>


The FBI's latest mission: Be cool enough to recruit hackers - <AUTHOR> https://www.washingtonpost.com/news/the-switch/wp/2016/09/02/the-director-of-the-fbi-explains-his-agencys-quest-to-be-cool-enough-to-recruit-hackers/ <comment><USER2> Arguing _pro_ -encryption is _so 2013_. What's cool in 2016 is arguing _against_ encryption - apparently:

[http://www.theregister.co.uk/2016/08/31/fbi_wants_adult_conv...](http://www.theregister.co.uk/2016/08/31/fbi_wants_adult_conversation_about_backdoors/)

</comment><comment><USER3> Coercion has always worked too. Bust a hacker, threaten a decade in prison, and BOOM! the Feds recruited a hacker.

</comment><comment><USER4> They have to compete with the CIA and NSA for hackers.</comment>


Rock Paper Scissors Playing Glove - <AUTHOR> http://grathio.com/2010/03/rock-paper-scissors-training-glove.html

<comment><AUTHOR> By the same guy that made the secret knock detector.

[http://grathio.com/2009/11/secret-knock-detecting-door- lock....](http://grathio.com/2009/11/secret-knock-detecting-door-lock.html)

</comment><comment><USER2> Awesome cool glove. I want one. Any kits available?

I'd bet this is pretty susceptible to data poisoning, though. Open your first 200 games with scissor and you can probably win the next 50 opening games using paper.

<comment><USER3> you are still loosing 200-50 :-).

I'll guess in the long run the only sound strategy is to be totally random and that will get you only 50% of wins.

<comment><USER4> no, in the long run you are playing against a known algorithm (assuming you've read the code), and so:

\- in the case that it is a deterministic algorithm you should be able to win every hand

\- in the case that the algorithm employs randomness then you should be able to win more than 50% of the time (unless the algorithm used is to employ completely random play in which case no matter what you play you will win exactly 50% of the time)

</comment></comment></comment><comment><USER5> Is there a lizard/Spock add-on yet?</comment>


What Really Goes On Inside An Apple Retail Store, And Why You Should Be Afraid - <AUTHOR> http://readwrite.com/2012/12/21/sex-drugs-theft-and-destruction-what-really-goes-on-inside-an-apple-retail-store-and-why-you-should-be-afraid

<comment><USER2> Yeah -- this is no different than geeksquad or any other computer repair place.

Going back in time I used to know people who worked in a photo lab that did the very same thing.

I wonder, what regulations or laws are in place at the state and federal levels to prevent this?</comment>


Ask HN: Do you think chatbots will be like websites? - <AUTHOR> Hi guys, I was wondering if chatbots are going to rise and gain revenues for companies just like websites and mobile apps did for them in the past.
Right now chatbots are efficient as websites were back in the days of Web 1.0. In most cases, they don't really help the user get things any better.
I'm talking about companies which chatbots is not their main business. Do you believe that once AI / ML will be more mature, companies will adopt chatbots as an additional communication (and revenues) channel?
Really interested to hear your take on that.
Tomer <comment><USER2> I think the niche for chatbots is in working with existing customers/users. Unlike a website, a chatbot is not (typically) going to attract new customers/users. Nor is a chatbot likely to provide entertainment or general information. More or less a chatbot is useful for getting 'work' done and not really for entertainment/leisure. Yeah sure there are entertaining chatbots on Slack, but the context for Slack is mostly work.

But I could be wrong.

</comment><comment><USER3> Not really. The one way I can see it working is when we get AGI. No one would want to interact with a chatbot unless it was something they feel like they could trust.

To trust a chat bot it would need to provide real, consistent value over a long time period.

Until they can provide a better experience than current channels I don't see how it could make any kind of impact.

<comment><AUTHOR> The experience is poor now because of the technology. AGI seems far fetched right now, so the revolution will happen only in the far future, as you say. Maybe there is no business need in these chatbots (that I described) in the first place?

<comment><USER3> Yeah I agree. Unless you can fundamentally change the value people get from this it's not worth it.

I've seen some medical based apps that have gotten funding but it's just a decision tree that tells you what you might have.

</comment></comment></comment><comment><USER4> Voice will change the chatbot game ...voice recognition is already good enough...the true bottleneck is lack of methods to create generative models..at the moment all cbots run on a retrieval model.. it's likely to change in next 5-6 years..

</comment><comment><USER5> No</comment>


Bankruptcies hang over China's electric car market - <AUTHOR> https://www.ft.com/content/1ed039c8-7e14-11e9-81d2-f785092ab560 <comment><AUTHOR> 'China's electric vehicle sales have grown tenfold since 2014 and last year it became the first country to see new energy vehicle sales surpass 1m, about three-quarters of which were pure EVs and the rest hybrids. But that growth has been dependent on subsidies averaging at Rmb70,000 ($10,100) per vehicle, allowing companies to lower prices. This month, subsidies will be cut to about Rmb25,000 for most vehicles. That will force companies to raise prices, wiping away one of the key reasons for their sales at a stroke, or accept lower margins on businesses that are already lossmaking.'

</comment><comment><USER2> Paywalled.</comment>


Uber loses court battle against Australia Tax Office - <AUTHOR> http://www.smh.com.au/business/uber-just-lost-a-crucial-court-ruling--which-might-push-up-the-price-of-a-fare-20170217-guf8ql.html <comment><USER2> What would make sense would be to tax the 20% cut that Uber (the company that makes >$75,000/year in revenue) is taking from the rides (see any discussion of what exactly is GMV).

Then, if an individual driver does make more than the threshold, he'd also lose his GST exemption. This would be a fair and correct application of the existing legislation that takes into account that drivers are self-employed and that Uber is a large company.

I like that Australia is attempting to maintain the rule of law, but Uber has a point: if they are to be treated like taxis they ought to have all the privileges of taxis, including street pickups.

This particular solution sounds more like the government wants a certain outcome and is bending the rules to achieve it, which is in the long term damaging to the rule of law.

If the issue is one of implementation (such as the government being technically incapable or legally barred from doing a simple SQL query to determine revenue per driver in a financial year) then they should outright say so.

</comment><comment><USER3> This makes good sense to me, Uber drivers contribute to degredation of expensive infrastructure (roads) and will also contribute to pollution (noise, air, run off into waterways), affecting the the host country in which Uber is conducting business. Therefore, it makes sense that anyone making a profit from such services should be giving back to the hosting community, just like anyone else running a business would have too.</comment>


Ask HN: What are some good startups that a CS grad student should intern at? - <AUTHOR>

My thinking on what could make a particular startup a good place to intern would be - 1. It has a strong engineering team, so the internship provides good learning and an opportunity to build good network. 2. The product is exciting and requires strong CS skills (as opposed to one that requires stronger marketing) 3. The startup is doing good, so there's a possibility of future full-time job. <comment><USER2> What have you got experience in beforehand? What are your interests? What technologies are you skilled in & which you want to be skilled in? Are you looking to found your own company, or get a job at one?

The answers should guide the company you choose, not emailing the latest hot company on HN.</comment>


The worst case scenario happening when starting a company - <AUTHOR> http://www.maximise.dk/blog/2009/06/worst-case-scenario.html

<comment><USER2> This happened to me around 1½ years ago, and I just thought I'd post a follow- up to a very negative story here.

After this whole thing fell apart I sat down and thought about what I could have done differently, and how it could have made our chances of success better. A lot of the things that happened (break-in, heart attack, etc.) were black swans that probably can't be avoided in the future. I did learn some important lessons though.

\- Make sure your co-founders are as engaged as you are. I would even go as far as to say that if you can't find someone that really believes in the idea and is willing to put some serious work behind it it's better to go it alone.

\- Make sure expectations are squared off, and you don't oversell the promise of riches. Most startups fail, and your co-founders need to know this. Otherwise you'll end up with broken friendships, which is never a good thing.

\- Learn to code. When I looked back I saw that if I had been able to code I would have been a much better manager and founder, and would probably have been able to do a lot of the stuff that gave us so much trouble myself. I've since become pretty proficient at PHP, MySql and CSS. And I'm learning python and javascript.

The stuff I'm working on now is coded by myself, and I'm doing it alone. Not because I don't want a co-founder, far from it, but I don't want to have a co- founder that drags the whole venture down.

<comment><USER3> > A lot of the things that happened (break-in, heart attack, etc.) were black > swans that probably can't be avoided in the future.

Neither a break-in or a heart attack is a black swan—people take out insurance against both all the time. If you're working with an organization, rather than an individual contractor, they most certainly have measures in place to ensure you never feel the brunt of such an event.

<comment><USER2> When your co-founders have heart attacks and break-ins I would consider it abnormal. Sure you can insure your way out of it, but it's a constant tradeoff. A normal insurance (which we had) didn't help much since it only covered new equipment after the breakin, and didn't cover anything in the heart-attack case since we weren't working for a client with a contract. As a startup you have limited time and money and you've got to decide where to spend it. Large corporations have the luxury of being able to have all- covering insurance policies,redundancy, policies for replacing employees, well-established back-up plans, etc. Most startups don't.

The very things that make a startup cheap are also what make it vulnerable.

</comment></comment></comment><comment><USER4> If you have an idea but not the skills to execute it, you will most likely need a company (with contract and all) to deliver the product. Hiring friends as designer and as programmer to work remote and for shares of the company has a high chance of disaster, especially if you are not a developer yourself.

One thing that might help could be setting up a schedule where you work together at a co-working place or your local hacker club, etc.

<comment><USER2> Right on. That's one of the things I learned from this.

</comment></comment><comment><USER5> Thanks for the story, I read it with great interest because I can really relate (like many around these parts I'm sure).

I'm glad to see that you found the energy to see your project through until the end, even if it didn't work out eventually. It probably cost you quite a bit, but that's the name of the game. You can't know until you try (well that's what I keep telling myself these days).

You're right that the combination of friends and work especially is a delicate one, something that I did not foresee either (working on my first big project).

Also I agree with your advice, about needing co-founders that are as engaged as you are. However this is a very hard thing to achieve. Not everyone is as crazy as we are. And communicating a vision is a very hard thing to do as well.

Although it might not seem so for someone with not much prior experience with coding, the simplest thing to do is still to have a go at it alone, at least in a first stage. It's probably easier and less risky than managing a team to execute your crazy idea for you.

Anyway, don't leave us hanging! What is the url of your website?

<comment><USER2> The url is <http://localhost/> :-)

No seriously, since there was no income model, and I was generally quite fed up and didn't see a future of the project I didn't see any value in leaving it up.

</comment></comment><comment><USER6> "First, the sites I was using to draw in users blocked my profiles ... worse they had put mechanisms in place that prohibited me from making new profiles."

I want to know what this guy was doing to get banned like this; such treatment is usually reserved only for the most serious of spammers.

<comment><USER2> Basically the plan was to use some of the existing social media sites (One in particular that was large in our target market) and locate the opinion leaders on that site by analysing how many friends they had, etc. We then personally contacted around 100 people and tried to talk them into helping us getting off the ground.

When they found out that a competitor was using their network to garner interest for a competing site they used every means at their disposal to shut us out. Understandable, I would probably have done the same thing.

</comment></comment><comment><USER7>
         Second the income model we were using was made illegal
         because of a new law that was hurried through parlaiment
         due to issues unrelated to us. This was the definitive
         blow. With no near-term prospect of turnover in a
         country where it isn't easy to go out and get external
         investors aboard there's nothing more you can do.

What kind of business model was it? Can't really think of anything "illegal" you could even do with a social network website

<comment><USER2> The businessmodel was based on overtaxed SMS services. You'd be able to sign up for updates about parties, where your friends were, etc. and each SMS would cost around 20 cents. What made this work well was the fact that the overtaxed SMS is initiated by the server meaning that users that are signed up can receive a SMS and the 20 cents are immediately drawn from their mobile account. We had a a system set up so that people had to not only opt-in, but also accept a SMS when they signed up for this so that they wouldn't feel cheated. We had other things such as a monthly maximum, etc. set up to make sure everyone was OK with it.

As you'll quickly realise the fact that overtaxed SMS services are initiated from the server can be misused - get a bunch of random phone numbers and send them all a SMS that costs $10. Someone did this, and the law was quickly changed so that a load of back-and-forth SMS'es were required to make the payment go through. Since each SMS costs money to send it ended up actually costing money to have this service running.

</comment></comment><comment><USER8> Damn Max!

But hey, if you're going to fail, might as well do it spectacularly. I can't imagine a more horrendous way to fail than giving all your users a surprise charge of $200. I guess you could break into their house and kick them in the groin, but I'm not sure that would be worse. :)</comment>


Why millennial women don't want to call themselves feminists - <AUTHOR> http://www.pbs.org/newshour/making-sense/column-why-millennial-women-dont-want-to-call-themselves-feminists/ <comment><USER2> The entire appeal to women for Clinton by some of her folks is kinda tone- deaf, in my humble observation. It's basically saying "Hey, this person is a woman, you're a woman, therefore you should vote for her." This is the exact same sort of oversimplification pandering and objectification that a lot of women have fought very hard to separate themselves from.

Additionally, the increasingly hardline message of "Fuck men because men" from certain radical feminists, both internet nutjobs and old-guard aging intellectuals, is not something that appeals to Millenials.

Sanders doesn't rely on messages of hate, and doesn't rely on playing one population off against another. There's only one person who can do that effectively in the current race, and it's Trump.

EDIT:

(updated a little wording above on the first paragaph)

The article, in my opinion rightly, points out a good point: there really isn't the same sort of systematic bias in the workplace and economy and culturally that there used to be (in the United States). Objectives achieved, one would expect the second-wave feminists to settle into the long haul to mop-up the last holdouts and to make sure that the next generation was vigilant to prevent backslides and to appreciate what they'd gotten.

Instead, it seems that third-wave and later feminism failed to consolidate those gains and has instead begun descending into self-parody and besmirching the good name of feminism. This timing happened to coincide with the rise of social media, and as we all know, that makes the problem of isolating a movement's more radical elements even harder and serves to discourage moderate participation.

<comment><USER3> The problem with Mrs Clinton is that she is an anti-feminist. She went viciously against the women who have been speaking against her husband, she supported legislation that disproportionately affected women when in Senate, and she never gave two hoots about women's issues abroad when she was the secretary of state.

Being a woman is her only redeeming quality, and she's not doing a very good job of that either.

<comment><USER4> "The problem with Mrs Clinton is that she is an anti-feminist. She went viciously against the women who have been speaking against her husband, she supported legislation that disproportionately affected women when in Senate, and she never gave two hoots about women's issues abroad when she was the secretary of state."

BOOM! She's doing her job as a politician by trying to get votes from women with fake empathy. All of them will do that. So, for me, I'm pretty much organizing them in order of tyranny from most to least then voting for the least. Clinton and Trump are high up on the tyranny list. Sanders by default maybe.

Far as feminism, all I see online in forums I'm on is the anti-male, SJW style. I'd love to see a moderate one that fights _real_ abuse while allowing men to be men w/out tons of self-censorship. Know any articles by feminists pushing that?

</comment></comment></comment><comment><USER5> > The term "feminism" has been hijacked by a minority of vocal extremists who > have redefined it as "gender feminism," claiming that gender is a > patriarchal social construct created in order to oppress women.

Well, radfems have to justify their existence, after all.

<comment><USER6> I assume that you are aware that "radfem" is a distinct concept from "feminists with whom I disagree."

<comment><USER3> "Disagreement" assumes a rational discourse. People who are not willing to subject themselves even to the most basic rules of logic and civility can only be opposed, and should be ridiculed.

<comment><USER6> I was attempting to be generous.

</comment></comment></comment></comment><comment><USER4> I like to think they barely give a crap about ideology. Especially the vocal minority of SJW's that seems to aggressively dominate discussion on "feminism" while not representing any consensus by majority of either feminists or women. They just want to have a chance of a good life with same opportunities as anyone else while minimizing any significant, rather than made-up, abuses. So, they focus on issues most likely to harm or help them instead of gender politics as usual.

Thing that jumped out at me was that the writer implied women were stupid for thinking voting for Hillary was voting for "the Establishment." Hillary allegedly started in Watergate, tried to strip Constitutional protections in various ways, supported the torture program, supports dragnet spying, and recently (with Trump) mocked the concept of free speech. Most of her actions, like other tyrants, remove power from the voter and consolidate it in the federal government. So, yes, she's very Establishment and in pretty bad ways.

Good to see a lot of women are noticing that. Maybe they'll vote better women into office or end up being one themselves.

</comment><comment><USER7> That's because a lot of women actually like men, The majority do. And the majority of men and women in western countries respect each other. And they don't see the premise invented by modern feminists. A premise that sadly PBS is also trying to push as well on their various cool hip properties. The conspiracy of the patriarchy just doesn't hold up.

And people are discovering thankfully so that the most toxic people are modern feminists. Look at various movements that have been hijacked by feminism. Occupy wallstreet, the skeptic movement, video gaming and feminism itself have been ruined by feminism.

If you want a better layout of the problem of feminism, see the recent videos by Thunderf00t.

The sad thing is how modern feminists claim that in western cultures men hate women, but these same feminists completely ignore and therefore give a pass to muslim countries, where they actually should be campaigning for rights.

</comment><comment><USER8> The whole article seems to take the approach that these young women are wrong, and don't know what they're doing.

Seems kind of an arrogant viewpoint? :(

<comment><USER3> Stop mansplaining!

<comment><USER8> That's sarcastic yeah? ;)

<comment><USER3> How could it possibly be a serious comment, what's wrong with you, people?

<comment><USER8> Well, if I'm not sure I ask. And I wasn't sure. :p</comment></comment></comment></comment></comment>


The last days of big law - <AUTHOR> http://www.newrepublic.com/article/113941/big-law-firms-trouble-when-money-dries

<comment><USER2> This is a bit like writing about the programming business as if were all Google and Facebook. I knew a lot of people who started law school, most but not all of whom saw it through. Of them two or three joined fair-sized firms, one a small one, and the rest did all sorts of stuff--corporations (aerospace, insurance, ??), the military, the government.

"No relationship in the legal profession is more fraught than the one between partners and their money."

Another fine reason to avoid situations where you might have to hire a lawyer, no?</comment>


N2O: Erlang Web Framework on WebSockets - <AUTHOR> https://github.com/5HT/n2o

<comment><USER2> This makes Websockets so easy. Thanks for this, it's great.

</comment><comment><USER3> I like it, great work Maxim!

Some slides here (from Synrc Institute from Kiev):

<http://synrc.com/framework/web/>

</comment><comment><USER4> Why N2O? Is this erLanG on Laughing Gas?

<comment><USER5> It's based off the Nitrogen Erlang framework, but it's not clear to me why _that_ framework was named Nitrogen, either.

<comment><USER6> It sounded cool. :) Also, elements.

Source: I created Nitrogen.

Excited to see these guys pushing boundaries; that's exactly what inspired Nitrogen in the first place.

My hat is off to the synrc.com team for their work on N20, as well as Jesse Gumm (<https://github.com/choptastic>) for his hard work maintaining and improving Nitrogen as project lead for the past two years.

</comment></comment></comment><comment><USER7> I don't see the database support, what do they propose from that point of view?

<comment><USER8> You can rather easily use DETS (Disk Erlang Term Storage) for simple applications and Mnesia for larger ones. This is a simple approach that works quite well IMHO.

This looks really great, Elixir plus this would be quite a nice web stack.</comment></comment>


Faster python for Intel CPUs - <AUTHOR> https://software.seek.intel.com/python-distribution <comment><USER2> But Why? Why can't they upstream their tunings to general Python?</comment>


Raspberry Pi Project Ideas: How I Built Call Mom Button - <AUTHOR> http://plivo.com/blog/raspberry-pi-project-ideas/

<comment><USER2> I'm Andy, the author of the post and creator of the project. I'll watch for questions here or on the post.

It could setup a conference call, but that's a trickier API so I didn't use it.

\- Andy Fundinger

</comment><comment> jack-r-abbit I guess I was a little confused about what this actually does. This does not do any calling, right? It just issues a command to Plivo to call Mom's number and then, when she answers, call a second number (assuming this to be the house where this device is located). At that point the RasPi is totally out of picture as the conversation happens phone-to-phone (possibly getting Plivo out of the picture too?). So you've created an external one-touch speed dial. Is that accurate?

Side note: Is there something that prevents me from building this and putting in two numbers I am not associated with? Push button. Two random strangers get calls from each other. Hilarity ensues. That type of thing is kind of scary.

<comment><USER3> I haven't used Plivo, but IIRC, with Twilio, you had to verify you owned the number you used in the Caller ID field, either by buying a DID from them or by entering a confirmation code sent to that number via a call and/or SMS.

<comment><AUTHOR> Hi chhl. With Plivo, you also need to get legitimate numbers to send SMS messages. (numbers purchased on Plivo or numbers ported to Plivo)

As you know, this is a preventive measure against spoofing; however there are some edge cases where allowing custom caller IDs might make sense. (e.g. when you want to have people you called get back to a certain number—custom phone line numbers at a large organization.)

<comment> jack-r-abbit That all makes sense. I assumed there would be some safe guards for that very reason.

<comment><AUTHOR> True, I think that's fair concern.

</comment></comment></comment></comment></comment><comment><USER4> You can use something like easy cron to schedule a call. Scheduling a call with a single http request example at

[http://blog.kookoo.in/2013/09/scheduling-audio-conference- wi...](http://blog.kookoo.in/2013/09/scheduling-audio-conference-with- single.html?m=1)

<comment><USER2> Yeah, that would work with a ping to Plivo and a dummy callback response too. Same trick as I used here.

</comment></comment><comment><USER5> the next step is probably adding an automated call mom button where it automatically connects to individuals. I think this would be awesome for conference calls etc...

Is this possible with Plivo/SMS' apis?

<comment><AUTHOR> Hi Kumarski, as Andy (author of the post) mentioned earlier, you can connect two (or normally a lot more people) using our conference API and XML and you could use something like cron to schedule the conference call. (if you'd use Python?)

</comment></comment><comment><AUTHOR> Hi, it's DH from Plivo. Happy to answer any questions regarding Plivo. :D

<comment><USER5> Is the thing I'm asking for possible. Will I ever be able to connect 2 individuals by creating an automated call with Plivo.

Problem: I hate having to call in some number to do a conference call with a pin code etc....

Is there any way I could get something to automatically connect two people at the same time using Plivo?

<comment><USER2> This is exactly what the Call Mom button does. Call one person and tell Plivo when the call connects to respond by making another call. The limit for that is two people however, more requires their conference call API.

<comment><USER5> I must have misread it. thanks for documenting everything.</comment></comment></comment></comment>


How Tinder Got Their First Users - <AUTHOR> http://benchhacks.com/growthstudies/tinder-growth-hacks.htm <comment><USER2> In my experience their first users were probably spammers/scammers/bots trying to find people to rip off. I can open Tinder here in Indy and I can hit a dozen or more profiles in a row that are blatantly fake, highly pixelated model photos or camgirl/porn stills with phone numbers pasted across them MS Paint style.</comment>


San Francisco's minimum wage will rise again to $10.55 - <AUTHOR> http://www.sfexaminer.com/local/2012/12/san-francisco-s-minimum-wage-will-rise-again-1055

<comment><USER2> "voters approved a local ordinance tying the minimum wage to the regional rate of inflation" Isn't inflation naturally affected by wages (among other things)? So this creates a feedback loop driving inflation upward.

This whole situation reminds me of the "B Ark" from Hitchhiker's Guide to the Galaxy. A whole planet schemed to get rid of people they felt were in useless jobs (e.g. telephone cleaners, and hair dressers). If the minimum wage in SF is now $10.55, who will sweep the floors?

In the story, the rest of the planet was soon wiped out by a telephone-born virus or something :)

<comment><AUTHOR> Robots, doh!

<comment><USER3> No, the same people that do it today. Only difference is they'll get paid 3% more per hour.

<comment><USER2> FTA: "The study by economist Aaron Yelowitz concludes that earlier studies failed to recognize groups who are losing out on work opportunities because of the higher labor cost — specifically teenagers."

So, it won't be the same people. There will be fewer teenagers in the workforce.

<comment><USER3> That would be what the article points out as "funded by a restaurant and beverage industry lobbyist". That industry has an interest in keeping the minimum wage lower.</comment></comment></comment></comment></comment>


A Few Thoughts on Ray Ozzie's "Clear" Proposal - <AUTHOR> https://blog.cryptographyengineering.com/2018/04/26/a-few-thoughts-on-ray-ozzies-clear-proposal/ <comment><USER2> The work I did on mobile encryption was framed thusly:

\- Deriving a key for all devices from a single key creates a single, catastrophic failure mode for the solution where all devices become vulnerable together. As soon as customers figure this out, nobody serious will adopt it because they can't afford to accept that known risk exposure.

\- We're assuming that the HSM we're using doesn't have a bias in its key generation RNG to limit the real key space, because if I were an intel agency, that's probably the first lever I would pull.

\- The entropy of the additional derivation components we can source from the individual device to locally diversify keys is really limited, and some really smart people are going to be reversing our code. Apple (and unrelated, in my own work, I never worked for anyone affiliated with them) relied on limiting number of attempts in hardware (effectively) to mitigate this risk.

Personally, I think the Ozzie proposal is a red herring to give the feds rhetorical leverage by providing their side with something few people understand, but can get behind politically because it's sufficiently complex as to be "our" magic vs. "their" magic. This is to drown out technical objections and make the problem a political one where they can use their leverage.

As The author (Green) notes, we can design some pretty crazy things, and if the feds came out and said, "build us a ubiquitous surveillance apparatus, or at least give us complete sovereign and executive control of all electronic information." that is technically solvable problem, but in the US, legally intractable. So instead, they want those effective powers without the overt mandate.

</comment><comment><USER3> > It literally refers to a giant, ultra-secure vault that will have to be > maintained individually by different phone manufacturers

We can't even trust manufacturers to provide updates in most cases. Placing that much trust in them is nothing short of lunacy.

</comment><comment><USER4> I don't see anything new in the alleged proposal, this is the same old crypto war. This is "just" key escrow.

One might as well propose to have the manufacturers build in the governments public key (and autobrick phone usage) such that the phone can detect if it is really the government reading the phone.

Another note:

"Ozzie's proposal relies fundamentally on the ability of manufacturers to secure massive amounts of extremely valuable key material against the strongest and most resourceful attackers on the planet. "

This is not true: the phone encrypts the users passcode against the manufacturers public key. If the government tries to read the phone, it will get the encrypted passcode (useless) and send it to the manufacturer who decrypts the passcode. A single private key is not massive amounts of information. Not that it changes anything about protection needs: wheither its a piece of paper containing the say 4096 bits (512 bytes), or in Matthew Greens misinterpretation billions of 512 bytes (half a terrabyte) on a single HDD, they both have the same value. The whole code base needs similar protection anyway: their bootloaders already are signed by the manufacturer.

All this centralization is bad, leave the crypto genie out of the bottle please...

<comment><USER5> I do prefer the idea of storing it on paper... at least it's a little easier to lock up. Even a big camera will only take a few thousand pictures before it fills up, and physical access is a lot easier to enforce.

If we make 2 billion phones a year (Apple itself is just over 200M) and you have a line printer running full blast (66 lines = 1page per sec) you could do Apple with one printer... and the world in 10. It would be a lot of boxes of paper though... about a box an hour.

edit: to be clear I was assuming that almost every dot in the matrix was a valid bit and there were 66 keys per page... 80 or even 132 columns at 7x5 wouldn't be enough for 4096 bits otherwise.

<comment><USER4> impressive calculation for the per phone case!

but as I wrote, its not necessary in Ozzie's scheme: Apple only needs to store the single private key. All the phones contain the same public key corresponding to it. All phones encrypt the user passcode to the same public key. When a user tries to unlock his own phone with his correct passcode, the phone encrypts his passcode and arrives at thee same encrypted key, unlocking the phone. When the government seizes the phone, with a special device they have the phone show the encrypted pass code, dump the encrypted GB's of encrypted phone contents, and burn an irreversible efuse in the processor disabling it. They send the encrypted passcode to Apple, who verifies its the government indeed. Apple uses its single private key to decrypt the user passcode. Apple sends this pass code to the government. The government can decrypt the image.

In the proposal there is no need for a massive database of key material. It's nonsense.

(in practice Apple would use treshold crypography, so that at least k out of n private keys each belonging to specially trained and screened employees are necessary to decrypt)

(in practice each phone has a hardcoded random nonce in efuses and instead of encrypting the user passcode, it encrypts [passcode+nonce], otherwise the government could just bruteforce 10^4 encryptions to the public key)

I am only saying that this can be done efficiently, not saying that I agree with the desirability of key escrow. This idea of key escrow is as old as cryptography.

<comment><USER5> Totally agree that they only _need_ a single secure key and a BUNCH of insecure nonces. However, if I was forced to keep a key in escrow and wanted it to be secure I'd put a uniquely generated (lots of lava lamps?) key for every one on paper and force anyone who wanted to look them up to do it physically, in person, with paper. Out go the digital public keys, in stay the paper private keys in a well observed building full of a zillion boxes of paper. Most insecure part is still the key generator.

If the feds want to audit, they can... but everyone will see it on video, what boxes they opened, and what pages they (could have) looked at.

I'd hire some magicians for pen testing too.

edit: pi*10^7 sec in a year is a useful approximation

<comment><USER4> 1) regarding BUNCH: the nonces are random and hence there will be as many as there are phones drawn from suitably large n-bit space, but Apple does not need a local copy of the nonces, if the government requests a decryption and Apple agrees, it will decrypt and find the user pass code and the irrelevant nonce.

2) "and force anyone who wanted to look them up to do it physically, in person, with paper" I dont understand your proposal? If the government wants to decrypt a phone, they should come to Apple in person with _what_ paper? How do you insure that everyone knows when a phone is audited? (Ozzie's proposal or what we read of it in the article does not adress insuring the populace finds out whenever a phone is decrypted). In your scenario the well observed building is operated by Apple or by the populace?

<comment><USER5> The idea is that each of the secret keys (not nonces) would be kept on paper in a well observed location, with only the public keys leaving. The building would be operated by whoever is responsible for generating the keys and showing that their keys aren't (hopefully yet) compromised. They could allow the public to observe and perhaps the boxes could be marked with the range of IMEIs/keys contained within. If the cops want to go in and get a key out of a box, they can get a warrant to do it but everyone can know which few hundred thousand phones have been compromised.

It doesn't completely prevent malfeasance... it just makes it a PITA.

<comment><USER4> suppose Apple owns the building:

* there is no advantage in having cops come over: either a secret is revealed or it is not. Any information to convince Apple concerning a specific case or phone could just as well be sent over the internet. Allowing them to enter looks like a serious threat vector to me, they could plant things, smuggle things out...

* Either Apple is faithfully reporting each count of the cops unlocking a phone, or it isnt. In the case of requesting over the internet the cops can't bring in devices to look through closed boxes or whatever.

* Is your fear rooted in a perceived sense of insecurity because of the small passcode (4 decimal digits) and the effect that would have on the security of the encrypted(passcode+nonce)? because that is exactly why the random nonce is there, in theory the user could select his own nonce and have it burned in efuse memory, but he would only be able to change the nonce a limited number of times. Then the user can roll as many dice as he wants and xor bits to smithereens ;)

but it all stays crap key escrow, its just a big "Eureka!"-show trial balloon to gauge public acceptance, no?

<comment><USER5> The real risk is that whatever the key-holder thinks is air-gapped storage isn't and the whole lot is secretly lost to crackers, state sponsored or not... that's a lot harder to do with 1000 tons of paper.

The point is that even a dedicated party trying to keep the keys safe probably can't do it (for any length of time) on digital media.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> The main difference between this proposal and the previous ones is the bricking step, which is supposed to make it transparent when the key has been revealed. But once the key has been revealed, what prevents an attacker from replacing the main board of the phone (keeping the phone's exterior and its SIM card), and copying all the data to the new board? A non-technical user (and even most technical users) wouldn't know the difference.

</comment><comment> irq-1 Bricking the phone works _against_ law enforcement by only allowing raw access to the data. Even if Clear worked correctly, law enforcement couldn't open apps and see the data in the correct context. They'd have raw data files full of indexes, hashes, and cached data. Worse, apps would start to encrypt data on the client specifically to avoid Clear.

The only significant change between plain key escrow and Clear (bricking the phone) would defeat the usefulness of Clear.

<comment><USER7> Apps could potentially work in read-only mode. Plus it's pretty easy to design a tool to pretty print iMessages given raw data, and that alone would be very useful for law enforcement.

</comment></comment><comment><USER8> > [..] and keep the secret key in a "vault" where hopefully it will never be > needed.

That's only in bullet point one and where it already falls apart.

</comment><comment><USER9> I cant think of anything worse than my plastic metal and glass friend being forced to snitch against me. Its like my best friend betrayed me. Beyond creepy, key escrow proposals are the very definition of totalitarianism.

</comment><comment><USER10> Can anyone explain why the government wouldn't just mandate that they be given all the keys from the start? Why would they put up with Apple as a middleman who could potentially refuse their requests?

Also, this key escrow scheme is near impossible to scale to more than one government. Now we need a way to authenticate government agents, good luck with that.

<comment><USER7> The government would be a single point of failure so it's cheaper and more secure to privatize. Also, private control of keys acts as a check upon government abuse.

</comment></comment><comment><USER11> But why? Why give the government such a ripe target for abuse? Why tilt the balance of power even further in its favor?

<comment><USER12> Also a good question, but this post focused only on why it's a stupid idea technically on purpose.

<comment><USER11> Point taken. He briefly talks about "policy" in the beginning. Policy is the instrument through which the ruling class gradually strips away individual freedoms. We need to prominently feature freedom in these conversations, because ultimately that's what at stake here.

</comment></comment></comment><comment><USER13> If you'd like to look at the proposal yourself rather than to interpret it through others:

[https://github.com/rayozzie/clear/blob/master/clear- rozzie.p...](https://github.com/rayozzie/clear/blob/master/clear-rozzie.pdf)

</comment><comment><USER14> I'm not sure the benefit to Apple or other phone manufacturers. This looks like a substantial cost with zero benefit to those others than law enforcement. And substantial new risk for misuse or abuse.

What's Ozzie's true motivation? Is he looking to start a company running Clear and raking in patent revenue? I get why the governments want this, but not why a citizen would propose this.

If it weren't Ray Ozzie, I would think this was just part of some propaganda push.

<comment><USER15> Money. His true motivation is money. Secondary to that is prestige; he "solved" this problem.

<comment><USER16> You might be right, but you're guessing. I'm pretty pro-crypto. I'm an anarchist. I am generally oriented towards non-governmental solutions to everything, including violence.

But even me, even with that bias, I still worry quite often about what evil can lurk behind cryptographic structures, and what effect wide availability of strong crypto will have on that.

I don't know that it will be positive or negative... my gut says positive, but I worry. And so it's not crazy to me to think Ozzie might be legitimately worried about people's safety.

</comment></comment></comment><comment> valiant-comma Just a nitpick. Matthew Green uses the analogy of signing keys being leaked often as evidence that Ozzi's proposed system would be similarly not secure. This is a weak analogy: signing private keys are often leaked because their use case requires them to be "online" in some fashion (code must be signed with the private key so it can be verified with the public key). Similarly, CAs must use private keys operationally (to sign customer CSRs), increasing the risk of key compromise.

In Ozzi's proposal, the private key never actually has to exist outside the environment it was created in, only the public key does. As pointed out in other comments, LE would not need access to the private key, either, they could simply submit the encrypted passcode to the manufacturer, who would then decrypt it on their behalf using the private key.

<comment><USER7> Code signing and decryption both require access to the private key, possibly through a hardware security module. I don't see why decryption has less exposure.

</comment></comment><comment><USER17> Extremely exceptional access only, in cases where thousands of people's lives could be at stake or millions. Since we can't create a fully unbreakable software/hardware security systems anyway, if ever, companies can use technology + psychology. Unintentionally create an extremely difficult to find bug that requires extremely talented engineers and large hardware resources to break, then unintentionally share it with at most discreet way probably just verbally to very few trusted 3rd parties. And it is not officially approved by the top management or even knows about it. We don't live in a perfect world and we don't have a perfect solution. JUST COMMENTS, NOT A SUGGESTION!

<comment><USER18> > Extremely exceptional access only, in cases where thousands of people's > lives could be at stake or millions.

And how do we determine when that's actually the case and when it's overhyped or flawed intelligence?

> We don't live in a perfect world and we don't have a perfect solution.

Exactly, so focusing on phone encryption is probably a waste of time.

</comment></comment><comment><USER19> I don't want this scheme. I don't want key escrow. But, a critique in the document is a 'if lost, lost forever' moment. If the escrow DB is compromised, the article says all phone are now pwned. For that point in time, true.

But phones are online devices. why does the escrow key have to be a constant, which if the central store is compromised means all phones prior to that date are compromised forever?

eg, re=spin the per-phone keygen on some cycle, and you define a window of risk, but it passes. re-spin clearly has to pass through some protocol, but we've been doing ephemeral re-key forever with websites.

</comment><comment><USER20> He talks about "massive amounts of extremely valuable key material" needed to be stored for billions of devices.

It's not like this would be Fort Knox. All that data could be stored on a couple of USB sticks which, really, makes it even scarier. Someone could hold the entire contents in the palm of their hand walk away with everything.

<comment><USER21> The article makes exactly that point:

> If ever a single attacker gains access to that vault and is able to extract, > at most, a few gigabytes of data (around the size of an iTunes movie), then > the attackers will gain unencrypted access to every device in the world. > Even better: if the attackers can do this surreptitiously, you'll never know > they did it.

</comment></comment><comment><USER22> What if someday we get political leadership so awful that, hypothetically, a former CIA chief feels compelled to warn that is is fundamentally dangerous to the nation?

One answer might be that we deserve such an outcome, and there is no reason to insulate encryption from the negative consequences. But is that a good answer?

</comment><comment><USER23> > Also, when a phone is unlocked with Clear, a special chip inside the phone > blows itself up.

no thanks

<comment><USER24> Assuming "blows itself up" means it's bricked rather than "does a Samsung", I'm ok with that. As the article explains, it's the only form of intrusion detection in the whole thing

</comment></comment><comment><USER4> Personally I believe real world actions should be the focus of surveillance. The empires are simply trying to cheap out by focusing on surveillance of computer activity.

This is the most profound part of Matthew Green's piece in my opinion:

"While this mainly concludes my notes about on Ozzie's proposal, I want to conclude this post with a side note, a response to something I routinely hear from folks in the law enforcement community. This is the criticism that cryptographers are a bunch of naysayers who aren't trying to solve "one of the most fundamental problems of our time", and are instead just rejecting the problem with lazy claims that it "can't work". "

I believe the most fundamental problem is how can we decentralize real world security? I am FOR mass surveillance but AGAINST _centralized_ mass surveillance.

Assume every crook and cranny of the world was covered by _community_ cameras, and the cameras encrypted the streams with treshold cryptography, such that the populace has different parts of the secret, then one needs "enough" citizens agreeing to reveal the contents seen by a specific camera at a specific time. This way its public for all or public for none. Every accident, every murder, ...

Suppose a body is found, then the group decides to reveal the imagery: oh yes, in this case the person was murdered! look the perpetrator is walking out of view to the next camera, then the next,... we can trace him to where he is now. Properly trained citizens (in a now authorized police role) go and arrest the guy. He is now in prison waiting for his trial (also with _community cameras_ , so no broomsticks in prisoner ani). At trial time, if the person denies, or claims to be a different person from the arrested one, we can trace through all the imagery from his commiting a crime to his sitting in court right there and then.

So yes, there is a real conflict between cryptographers and centralized law enforcement. We dont need no spooks!

And the spooks can not decode the camera imagery: a large enough number of citizens (chosen at random by cryptographic sortition) running instance of _good citizen client_ software need to release their part of the shared secret.

EDIT:

So there is broadly speaking 2 kinds of crimes:

* meatspace crimes (murder, negligence, rape, making childporn (automatically rape), ...)

* cyber crimes (copyright, child porn, ...)

I argue that not implementing such a _community camera_ system is a form of negligence in itself.

It does not adress things like copyright infringement, but ... thats not exactly the most popularly supported concept.

Then there is the problem of child porn: fake and real.

I argue that with deepfake any faked child porn will eventually become indiscernible from real child porn.

Which leaves the problem of official child porn recorded by the _community cameras_ used to apprehend perpetrators (since these also sign the imagery to testify authenticity!).

Due too taboo many victims of child abuse didn't realize, or only had doubts that they were suffering abuse, enabling the abuse to continue. Without concrete visual examples for them to explore, to asses if they are or are not suffering child abuse, how can they alert others of their situation? We send these children extremely mixed messages: absolutely tell us if you are being abused, but absolutely never falsely report a person. Merely asking someone else for advice is automatically interpreted as a child reporting child abuse. How can a child asses his or her situation? With abstract questions using words and connotations it does not know?

I believe the number of reported child abuses would go up if we used these _community cameras_ for decentralized mass surveillance.

Also for crime in general (theft, murder, ...), the knowledge that you will with extremely high probability be caught, will decrease a lot of crime. I would not be surprised if the crime rate of "impulsive" crimes (where the criminal was supposedly not able to control his urges) would drop substantially, revealing that in the current system they often get off the hook.

There will still be rude people, getting fines for squeezing women in the ass while drunk. But for any actual crime in general, both victim and perpetrator would know that the victim can simply report this to the group, and that the perpetrator can not escape by lack of evidence. The current lack of evidence constantly discourages people from reporting crimes (as there is risk involved: financial: lawyers, emotional: potential incredulity at police station, ...).

One might think that this will cause criminals to escalate to murder: "if you rob a victim, you should kill her, or else she will report you" but hiding a body will be very hard, and if a person goes missing the friends and relatives will report this, and instead of following the criminal we can follow the missing person from the time and place she was last reported seen!

As long as cryptographers only draw the _privacy_ card, the law enforcement community has a point. As long as the law enforcement community only draws the _centralized_ power card, the cryptographers have a point.

Only when we have decentralized mass surveillance can we have _both_ privacy (as long as you don't commit crimes or go missing) and real law enforcement.

Common FAQ:

What if say a stalker repeatedly reports his ex as "missing"? Cry wolf to many times, or be blocked to report a person missing, and the _good citizen client_ software that the citizens individually run, will refuse to comply.

What if a stalker or group of them repeatedly reports a "murderer" in a celebrities bedroom? we can send a local but randomly selected properly trained (group of) citizen (in police role) to go check the room, if the supposed dead body is not there, no reason to unlock the imagery.

(I will add more as people ask)

<comment><USER7> Regarding your distinction between real and cyber crimes, digital evidence can certainly be relevant in a murder case, e.g. iMessages, location history, search history. Also, the read-only bricking chip tries to allow search but exclude ongoing surveillance, though I don't think it's technically feasible.

<comment><USER4> "Regarding your distinction between real and cyber crimes, digital evidence can certainly be relevant in a murder case, e.g. iMessages, location history, search history."

But the cameras are supposed to completely cover society, so we don't _need_ the cyber info. Indeed, perhaps the perpetrator has a secret paper diary, written in code, where he writes down his exploits. Who cares? We have signed imagery, of him commiting the crime. Any extra information is useful in the statistical sense (to understand what drives a person to do this or that, or to better prepare citizens on how to prevent falling victim to such and such crime), but should be unnecessary to convict a person. The most relevant are the actions themselves I think.

About location history: the camera system is more reliable than the cell phones since a cell phone may be given to a friend willing to provide an alibi, alternatively GPS spoofing etc.

The major reason these cell phone messages, search history etc are highly relevant is simply because we lack the _community camera_ system.

Another problem is phone evidence is highly irregular: some people are more aware of mass surveillance then others (which is also highly correlated to status in society!) when communicating, some people refuse to have a cell phone on them, ...

When they lack enough evidence, the prosecution is forced to grab at straws (irrespective of guilt or innocence of the defendant), and then the value of computer/phone activity seems very high, especially if boots on the ground or scientifiic investigation of crime scenes is so much more expensive. Then it is easy to view this digital data as highly relevant and reliable.

</comment></comment></comment><comment><USER25> We need a new way of thinking about caches of secrets. It comes from this unpleasant truth: all secrets eventually leak. The evidence of the past few years teaches us that even state actors with unlimited resources cannot prevent their secrets from leaking.

A "leak" here happens when a trusted entity loses control of the secret to one or more untrusted and malicious entities. That's just a definition, not a claim that any particular government, company, or person is a trusted entity.

To counter this, we need multiple layers of defense.

One is the business of bricking the phones when the leaked secrets are exploited. That makes it plain that the secret has leaked. It's a valuable layer of defense.

Another is to make the secrets have limited useful lifetimes. Expiration and revocation for TLS certificates is a way to do that. Credit/debit card numbers can be deactivated and replaced rapidly. That's another way to limit the lifetime of a secret. Ozzie's proposal does not include a way to limit secrets' lifetimes. (Social Security numbers are problematic secrets: they too have unlimited lifetimes.)

A third layer is making the secrets have limited utility. If debit cards had daily spending limits, their secret numbers would be less useful than they are today, for example. Day-one exploits are secrets with vast utility, for another example. Ozzie proposes a secret to unlock an entire phone. How about limiting that to, say, the phone's call log or SMS log?

A fourth layer is to keep the caches of secrets as small as possible, so a breach affects as few people as possible. Ozzie proposes the opposite of this.

A fifth layer: holders of caches of secrets must know they are strictly liable for breaches proportional to the damage they do. It must not matter whether the breach was due to negligence, carelessness, espionage, or salt water rusting out the safe after a storm. Large scale key escrow cache systems will never be able to meet this standard: nation states won't honor that liability, nor will they pay private companies enough to cover the insurance for it.

(Strict liability is not unprecedented: workers' compensation and the vaccine injury victims' compensation fund are two reasonably successful examples.)

People, companies, and governments holding secrets necessarily must consider what happens when (not if) they leak, and provide at least some defenses in depth like these.

Ozzie's proposal has weak and incomplete in-depth defenses. That's why it's dangerous.

<comment><USER26> What if the proposal is changed so that the keys do not need to be kept forever? What if Apple is allowed (or even required) to publicize the private key(s) after, say, 10 years?

</comment></comment><comment><USER27> I agree about the security of a centralized vault being a key weakness, but the article omits a few key aspects of Ozzie's proposal:[0]

* A court order is required. It's not up to the tech vendor.

* Physical control of the device is required. No remote exploits.

* Access is enabled only to one device at a time. No mass hacking.

The point of security is to increase the cost to the 'attacker' (here we'll use that word even for legitimate government purposes); there's no perfect security; law enforcement can access data on iPhones already. Also, attackers focus on the weakest (i.e., least expensive) link and there's limited value in increasing the cost beyond the 2nd weakest link.[1] Except for the centralization of key storage and two other issues (see below), Ozzie's proposal might increase the cost to the level of law enforcement's alternative, acquiring a hacking tool. In fact, I've been thinking of something similar (court order, physical access required, notification to user) and might even have posted it to HN at some point.

Using hacking tools is much worse than Ozzie's process: There's no court (or at least it's not as enforceable, because there's no tech company checking for a warrant), no tech company, the user doesn't necessarily know their data has been accessed, remote exploits are possible, and so is mass hacking.

Also remember that private citizens can still encrypt their data at the file level using other tools, though of course most will not.

Here are weaknesses I see:

A) The use of other means of accessing devices would have to be outlawed, or law enforcement will continue to use hacking tools and citizens gain nothing.

B) Solve the centralization problem. Probably, the keys shouldn't be in the hands of the tech giants and should be distributed widely. EDIT: Perhaps require two unrelated parties for access?

C) If these new access tools are built into mobile devices, what happens in countries where people's rights have been taken away? The courts are often ineffective. I suppose the fact that the phones get bricked at least informs the user, and the authorities can use hacking tools anyway, so perhaps nothing is lost.

____________

[0] [https://www.wired.com/story/crypto-war-clear- encryption/](https://www.wired.com/story/crypto-war-clear-encryption/)

[1] If I increase the cost of exploit A to $100,000 and exploit B costs $50,000, attackers will use B. If I increase the cost of A even further, to $200,000, it won't provide much more security - the attackers still will use B.</comment>