Memex: Browser Extension to full-text search your browsing history and bookmarks - <AUTHOR> https://github.com/WorldBrain/Memex <comment><USER2> This looks cool. It reminds me that someone did a lightning talk at RailsConf 2018 about a Memex app they built and it was pretty mindblowing to me at the time:

[https://youtu.be/Ld414EypQzg?t=2709](https://youtu.be/Ld414EypQzg?t=2709)

</comment><comment><USER3> Tried using this for months; it never worked well. Settings changed back to "intrusive & useless" every update (seemed like) and the few times i needed it -- it hadn't been indexing anything for the past week, sorry.

Hope it's working better now

<comment><USER4> Oliver here, from the Memex team.

Yeah we know it has not been really well functioning in the past few months. Had a lot of bugs.

We finally got a bigger round of funding and are now able to fix those. Until February our main focus is improving release stability through fully integration testing the UI and backend, fixing all major bugs, improvements to UI/UX and the release of the mobile Apps+Sync.

Too bad this post has been upvoted so much now, a few weeks later things would have already been much much better :)

Regarding the bugs you mention I would like to know more if you could help me out here: 1\. Which browser and system are you working on? 2\. When you say the settings changed back to "intrusive & useless", what do you mean by that? 3\. When you say "it never worked well" what were you expecting to be working but it didnt?

Thanks and sorry for the troubles Memex causes.

<comment><USER3> Sorry to give you a bad review, and i look forward to seeing the version where everything works and it just magically does what you want.

My browsers would be firefox on windows. To expand on "intrusive and useless": I found myself turning off the "respond when i type" setting several times. the few times i did search for something, it hadn't been "seen" because the indexing was off again, or terms just were not found.

<comment><USER4> > Sorry to give you a bad review

Please don't feel sorry. :) If something does not work then we deserve those things to be said, also publicly. It's what after all helps us to improve and keeps us accountable.

> "respond when i type" setting several times.

I am confused about that setting. Not aware we have that anywhere? Can you explain a bit more what you mean by that?

> the indexing was off again

That can be, we had some issues with FF permissions for things to be copied to clipboard. We had to manually add that permission to the install process (only for FF). And I missed doing that a few times. (sorry about that). We are about to automate that process in the next releases so that does not happen anymore. I assume what happened is that the extension was turned off because on update it required some of the permissions to be granted again.

<comment><USER3> it has been a month or more since i turned it off; so i can't be more specific; but it was popping the sidebar up in circumstances where i didnt want it, and i recall chasing down a setting that stopped it.

Your explanation of permissions sounds like a very likely cause for most of the things that annoyed me :) Text search is hard, I know that from mere experiments and demos; adding the "be a browser plugin" to that must make life fun indeed.

<comment><USER4> > but it was popping the sidebar up in circumstances where i didnt want it

Ok so the core issue was that the sidebar was popping up in places you didn't want to? Or was it also that the keyboard shortcuts were firing the sidebar to appear in places you didn't want?

> Text search is hard, I know that from mere experiments and demos; adding the > "be a browser plugin" to that must make life fun indeed.

Yes, it was/is quite painful, especially in the browser. With its changing APIs, and websites constantly changing and using different technologies, makes browsers a very hostile environment to work with. We can't even do test releases with the chrome/firefox store. So we have to roll out all updates to all users at once. Crazy.

Before we focused on building something to show how our vision could work at the expense of stability. It was in a lot of ways not a good move because we probably could have been financially independent with a little less features that were more stable and better worked out. However our approach also allowed us to show the vision and get bigger amounts of funding. Now is the time to make the current feature set nice to use and people feel like it is worth financially supporting.

</comment></comment></comment></comment></comment></comment><comment><USER5> I have been looking for a cross-browser solution that will allow me to bookmark, tag, highlight and group URLs, I have even been contemplating on writing my own solution.

I will give this a spin, on paper it looks like what I need.

<comment><USER6> I fondly remember del.icio.us which was an amazingly simple tag-based bookmarking tool.

I believe it shut down shortly after being acquired by Yahoo and appears inop today. There may be a decent alternative out there but my system for bookmarking things in that way died with their service, sadly.

<comment><USER4> Oliver here from the Memex team.

glad you bring that up :) After we are finished with the current work on making Memex more stable and user friendly, we are working on shareable collections. With that you can share lists of websites, papers, notes and annotations with your peers, or co-curate those collections together.

<comment><USER7> Any connection to darpa memex?

<comment><USER4> No that is an entirely different beast :)

Both projects took the inspiration for the name from the same thing though: [https://en.wikipedia.org/wiki/Memex](https://en.wikipedia.org/wiki/Memex)

</comment></comment></comment></comment></comment><comment><USER8> I've been wanting this for Firefox since Opera exploded. Nice!

It also amazes me this kind of "store everything [locally] so life is easier" mindset isn't more common in today's world of extensions. It's one of the few things that can make the computer work _for you_ instead of you working in a way _the computer_ understands.

<comment><USER9> opera exploded?

<comment><USER8> By "exploded" I meant when Opera ditched Presto in favor of Blink, becoming basically a Chromium shell.

</comment></comment></comment><comment><USER10> Analytics should be opt-in, not opt-out.

<comment><USER11> Opt-in analytics generally don't work.

<comment><USER12> Asking people to do things without a threat or weapon to coerce them with often doesn't work either.

Just because something doesn't work well when done right, doesn't mean it should be done wrong. There are ways to gather data necessary to inform product decisions without surveillance, or turning your users into non- consenting test subjects.

(EDIT: Nevertheless, I'm very happy about the values they describe and that they chose to store the data locally. It makes me trust the authors that much more, and conversely, if it wasn't local, I wouldn't even consider using it.)

</comment></comment></comment><comment><USER13> This is definitely a step in the right direction! Ideally, I'd like to have something that I can set up on my phone and desktop(s), so I can have an aggregate search over all the things I visited, and on whatever device. Is there a way (architecturally) to make that happen, without having to rely on (say) using one browser on every device?

<comment><AUTHOR> Memex is a WebExtension, so it works on all browsers that support that API, which is most of them.

But there are many ways you can do that architecturally.

Here's one off the top of my head that might work:

1\. Set up a DNS server on a home server.

2\. Run a cron job to automatically wget every URL checked by the DNS server.

3\. Set all of your devices to your DNS server.

4\. Set up a local installation of one of the thousands of open source search engines and set it to go through all of your pages once an hour.

5\. Enjoy.

<comment><USER14> Doesn't DNS server only see the hostname instead of the URL? The approach would allow to only scrap home pages then I guess. Also, what about pages behind login?

<comment><AUTHOR> Correct for the first two, I hadn't thought about that. Could be done by uploading history once every hour but that's annoying. Pages behind a login aren't ideal for this under any approach because they tend to be dynamic.

</comment></comment></comment></comment><comment><USER15> I haven't looked over the whole site, but I find it curious that this wasn't mentioned here (or possibly there):

[https://en.wikipedia.org/wiki/As_We_May_Think](https://en.wikipedia.org/wiki/As_We_May_Think)

Because it seems this is yet (another) iteration or attempt on the device Bush described?

Really, hypertext, HTML, the web, browsers, etc - are all a part of such a system, but what is needed to really complete the loop - so to speak - is for everything to be P2P - so anyone can easily create and share information (hypertext "documents" \- whatever a "document" may be) and share contextual annotations (highlights and notes) on those pages, with maybe some manner to incorporate them into the main "text" as needed when they become too cumbersome outside the main text, along with easy P2P indexing and searching of the collective corpus of "works".

But this is completely at odds with how the current "web ecosystem" works, and certainly doesn't appear to be how this particular incarnation works (this seems to be a siloed system, if anything). Today's system takes control away from the users for the most part, unless they pay extra (ie - to create and maintain a server or whatnot for their own personal content - or not pay, but pay by giving away other information that the company giving them "free hosting" or whatnot can use). A complete P2P system would upend that model, assuming it could be made to work well (the asymmetry of the broadband infrastructure for consumers doesn't help, either).

I know there are more than a few P2P competitors out there that do much of what I am speaking of, but I don't think any of them do the complete "Bush Memex" system. Then again, maybe that's a good thing - otherwise we might end up with an unholy combo of wikipedia coupled with 4/8chan with a dab of reddit or something like that...?

<comment><USER16> Memex was Bush's term. The project's name is an obvious nod, directly referenced in the README.

</comment></comment><comment><USER17> Random factoid of the day: A fictional memex machine is featured prominently in cstross laundry files.

</comment><comment><USER18> I was always looking forward to something like that, since at least 15 years. Today's minds are shaped by what we read online and this is our source of knowledge on many topics (apart from books and other media), so why not organizing it? But then I noticed that this would add another source of data to be managed, evolved, etc, and today I'm more thinking that it's more about being organized as an individual rather than simply relying on just another tool for that.

Anyway I will give it a spin! Looks like a good piece of work.

<comment><USER19> I'd say the biggest problem with something like this is that it's a silo. You're suddenly 100% reliant on them providing the right tools and functions to access your data in the way you want/need. And if your needs/preferences change, then you're entirely reliant on whether the silo has foreseen the new use case.

I already have an existing knowledge base (that not only consists of webpages, but also org files, videos, pdf's, etc.) that's accessible and synchronized across multiple devices - and while having a complete searchable history of all my browsing would be fantastic, there's no way to integrate it into my system (or any other system) with my own tools.

<comment><USER4> Oliver here from the Memex team.

Yes! That is indeed a major problem with all knowledge management tools. They tend to not be interoperable enough so you can easily integrate them into your existing workflows. Also they are not built to be adaptive to the individual workflows of people, so you have to wait for the dev's priorities to be high enough to implement your features. You can't do it yourself.

We are also not there yet when it comes to the level of interoperability or flexibility needed. However we started from a fundamentally different angle by changing our economic model and not taking venture capital money: [https://community.worldbrain.io/t/why-worldbrain-io-does- not...](https://community.worldbrain.io/t/why-worldbrain-io-does-not-take- venture-capital/75)

In essence what we want to achieve is that you can copy/fork Memex, adapt it to your needs and still use your old data and social connections. Once that transition is complete you'll be able to even use 2 different Memex tools at the same time, both maybe serving different use cases for you.

May I ask what tools you use and how Memex in your ideal world would integrate them? What is the workflow you'd like to implement?

<comment><USER19> After experimenting with everything under the sun from Evernote to OneNote to TiddlyWiki and everything inbetween, I've settled on plain and simple files in a deep folder structure. The whole folder (now 80GB in size) is permanently kept in sync with SyncThing across my Android, laptop and desktop.

Using normal files allows me to store anything I need, whether it's webpages as html files saved with SingleFile (FF extension), videos downloaded from YouTube, notes made with emacs orgmode, podcast MP3's, eBook PDF's, etc.

Folders are deeply nested according to field/topic, and I have a git repo that ignores all non-org and non-html files. This lets me use ripgrep or emacs Helm to immediately search text for whatever I'm looking for. z allows me to traverse the tree without double-clicking through a deep tree of directories or cd'ing and typing crazy amounts.

So tools can be anything - Firefox with extensions, ripgrep, emacs, vim, git, z, or even whatever Python script I write to fill a unique use case that I discover for something that feels tedious. Normal files mean that if I find a cool program that does something useful with files, I can easily integrate that. I'm also working on ways to give me easier/quicker access to the metadata like the most recent files of a subtopic, or even add my own metadata like ratings and tags.

Ideally, something like Memex would provide some sort of api from which I could automatically query for all the browsing/history and text data, so I could potentially add it to my knowledge base in some way. Or maybe if Memex synced automatically to a DB file or some other file(s) that are well- documented that I could easily access, parse and sync.

<comment><USER20> reading your comment, I feel a sense of dejavu because its almost the same as the response I would have written. I've gone through a similar path of trying every knowledge base under the sun and settling on a text based, git managed custom knowledge base with plain text files (currently managing +10k notes this way).

currently building a service that can index and query across text based knowledge bases. you can find demo here: [http://demo.alphacortex.io](http://demo.alphacortex.io)

would love to hear your thoughts and talk further about organizing knowledge :)

</comment></comment></comment></comment></comment><comment><USER21> Looks like there's huge potential value with a tool like this.

Does it have a way to handle page content that's less relevant? For instance plenty of genuinely useful articles have footers full of clickbait ads & comments - whilst trying to avoid such content, sometimes the article you need is only available with that dross tacked on the end, and yet ideally users wouldn't want it polluting search results (unless they really were looking for "that amazing trick that only grandmothers know"!)

<comment><USER4> > Does it have a way to handle page content that's less relevant?

It already collects some basic interaction data like visit frequency, stay time and scroll %. This data could be used to clean the db a bit.

Other than that there is definitely room to improve to clean out the terms that are captured but not really add value. Its a difficult task though because every page is so differently structured.

What ideas do you have to reduce the number of unhelpful terms? A spontaneous one is to detect the footers of this OutBrain et. all crap and remove them from the HTML before filtering out the words to index.

Right now we are focussing on developing a stable service with better UX and that does the current feature set really well. I'll take up your suggestion so we can think about how we can use them in the upcoming overhaul of the search.

Thanks for your input!

</comment></comment><comment><USER22> I have been using it for a few months, maybe a year. Maybe I'm doing it wrong, but I can't seem to get any benefit out of it.

Texts highlights are a pain to use (so I don't, even though I love highlighting text while reading), the search doesn't find anything despite the keyword being in the title of the page and all the context menu and side bar are doing are unexpectedly hijacking my clicks.

I really hoped for more and still believe that it has the potential to be useful. I hope one day it will be.

<comment><USER4> Oliver here from the Memex team.

A bummer, sorry you experience troubles.

> Texts highlights are a pain to use (so I don't, even though I love > highlighting text while reading)

Why are they a pain to use for you? What would be 1-2 things we could improve that would make it a lot better?

> the search doesn't find anything despite the keyword being in the title of > the page

It might be that you have some indexing settings changed. Usually the page is only indexed after visiting it for at least 5 seconds, but you can change that setting

> all the context menu and side bar are doing are unexpectedly hijacking my > clicks.

On which pages does this happen to you?

Good news: We finally got a bigger round of funding and are now able to fix those. Until February our main focus is improving release stability through fully integration testing the UI and backend, fixing all major bugs, improvements to UI/UX and the release of the mobile Apps+Sync.

<comment><USER22> Highlighting:

I think it used to be worse, with lots of glitches, but I'm glad those now seem to be gone. I'm still annoyed by the multi-step process to highlight something, however. Select something -> Click "annotate", sidebar opens -> Find the annotation you just created -> Save it without typing in any text (remember to save it, because there's no auto-saving for some reason!). It should have been done with "Click 'highlight'". Since clicking the highlight still expands it in the side bar no funcionality would be lost by modifying the annotation feature to directly just save a highlight (If you still don't know want I would like to see, have a look at Medium's implementation).

Search:

Thanks. After testing it out again half-heartedly, it seems to find things. However it's just frustrating when you want to find back to an obscure site you have been on weeks ago, Firefox history is comically useless and you think "Oh, nice thing I have Memex installed" and then it _doesn 't_ find it, even though I remember that being the promise.

Click jacking:

When I want to open a context menu on some text using my finger I have to long press on it, but the degraded accuracy can lead to the click landing on the Memex menu (admittedly this seems hard to fix). Trying to reach far out page elements can lead to the side bar obtruding it. Both of these cases are very annoying.

I don't generally oppose the popup: It would pretty convenient for quick highlighting if highlighting was actually quick. But then again highlighting is _not_ very quick and the "share highlight" feature, (that is very cool) is not only of much rarer use, but also extremely confusingly implemented. Why do you need to be able to share a highlight before it has been created? Why is it not an option for an actually created highlight? Why does it not show me the link, but try to copy it to my clipboard, which doesn't even work on my machine (Manjaro/KDE/X11/Firefox)?

I like that there is a close-button on the popup, but I do find it way too obtrusive. It's not a feature a normal user (or at least I) will want often. That it's shown at the same priority of the other items confuses me hard, same with the options in the side bar. I think I will disable it, since the icon does all the things I need.

Why are there collections AND tags? Don't they do the exact same thing?

Why is there no option for a denser page listing so I can see more than 6 pages, (which would be good while trying to find something, and why else would one open it)? Why can I only have it sorted chronologically, not by number of visits? Can I even see that anywhere? How about length of visit (do you record such things?)? Minor UI addition (hey. seems like the place for that): The opening of the pop-up looks pretty jarring. A more fluid animation would be nice :-)

These are mostly complaints, but I'm glad you're working on this project.

I'm also really happy that it seems to be continually improving. I wish you lots of success going forward!

<comment><USER4> > These are mostly complaints, but I'm glad you're working on this project.

Wow. Thanks so so so much for taking the time to write this all out. This has been super useful.

I added all your feedbacks to our UX/UI/bug prioritisation board. Some of the things we had already on the radar (like improving the way annotating works) and will be improved very soon.

</comment></comment></comment></comment><comment><USER23> Great idea, implementation is a bit intrusive. Not a fan of the by default sidebar ribbon or the highlight popup menu. Luckily this can be disabled.

Importing history and indexing that is cool.

<comment><USER4> Oliver here, from the Memex team.

Thanks for the feedback. Except it being able to be turned off, how could we make the implementation less intrusive for you?

</comment></comment><comment><USER24> Installed the extension and added some pages, but it's clearly not indexing content of the pages because there are no hits for words that appear on those pages.

<comment><USER4> Oliver here, from the Memex team

By default that setting is disabled, but you can change to full indexing in the onboarding or in the settings.

<comment><USER24> Thanks, I found it.

The label for the setting is unclear.

> [ ] Visited for at least n seconds

Given that the setting above it says "Make title and URL always searchable (recommended)", I think the setting for this option also should be something like:

> [ ] Make pages visited for at least n seconds full-text searchable

It's difficult to connect the grey text that says "Which websites do you want to make full-text searchable?" to the label of the option that controls full- text searching.

context: [https://i.imgur.com/2p7OyHJ.png](https://i.imgur.com/2p7OyHJ.png)

<comment><USER4> Great feedback. Will incorporate it in the upcoming UX/UI updates. Thanks for the effort to make us aware.

</comment></comment></comment></comment><comment><USER25> This looks interesting. I wrote something quite similar back around 2000 as a proxy; today, of course, you'd have to implement it as they have, as a browser extension, because of https. It was a fun little project, and for the feature set I included (full text search of your history), it was quite easy to implement. At the time, I even did it with no dependencies that weren't in the Python standard library.

</comment><comment><USER26> I was trying to build something similar based on Firefox sync, but it took quite a while to get a client working, and I never got back to it. (I found some bugs in their documentation along the way and helped them fix that, so that's good?)

Maybe I should pick it up again. My intent was that it'd run on a server and be your own personal search engine covering activity on all devices.

<comment><USER25> I would be interested in helping you on this, if it's in a language I'm familiar with. Having it integrate with Firefox sync solves the problem of not being on any one device enough to make it your full history.

<comment><USER26> It's in python3 [https://github.com/jimktrains/ffsyncsearch](https://github.com/jimktrains/ffsyncsearch) I just pushed some of my last changes and it'll grab and decrypt all of the collections stored. Like I said, I hadn't got it to a point where I saved and indexed everything, but my intent was to start with postgresql's full text search and go from there. I also wouldn't mind some help cleaning up the code some as it is more proof-of-concept right now.

<comment><USER25> Thanks. I don't have a ton of time to work on it, but I've cloned it and will look at it.

</comment></comment></comment></comment><comment><USER27> I've wanted this ever since Firefox 2 or so.

</comment><comment><USER28> Years ago I was using UltraRecall for this purpose. It has FF plugin that allows for quickly bookmarking a page and then within UltraRecall it would build FT index. It was also possible to do the same from any other browser as the browser support is to just push the link to the standalone application, but I never felt need to invest time.

</comment><comment><USER11> The idea is promising. I have some questions though.

1\. How do you handle content updates/corrections? Do you update the index on subsequent visits?

2\. How do you handle fully dynamic pages like Facebook feed, Reddit, HN etc.?

3\. What is performance and storage overhead of indexing "every word of all websites & PDFs you visited"?

4\. What about i18n, have some plans?

<comment><USER4> Oliver here from the Memex team.

1\. Yes the index is updated every time you visit. It appends the new terms, and keeps all old ones

2\. That is a bit more difficult and not as reliable unfortunately. If there is a lazy load on the page it often fails to capture the content, because it starts indexing the page after the initial page load is finished/successful. These are improvements we want to work on a bit later.

3\. For about a year's worth of history(~20k pages), without also capturing the screenshots, it needs about 400mb of storage. Indexing performance is still good with 20-25k pages but querying gets slower. So you won't feel the performance on your system with a reasonably fast computer (recommended 8gb of ram, and a dual core with at least 2GHz) We are about to work on performance improvements to make it fast and scalable much beyond that amount and with less resources.

4\. Unfortunately we didn't get around to optimise the indexing for CKJ characters, but all latin characters should work fine.

<comment><USER11> Thank you.

On 2, I mean these (feeds, topic lists etc.) are probably not worth indexing at all, esp. since you keep all old content in the index.

Is your code all home-built, or you're using some FT engine compiled to Wasm?

<comment><USER4> > On 2, I mean these (feeds, topic lists etc.) are probably not worth indexing > at all, esp. since you keep all old content in the index.

Yeah indeed. A better implementation for now would be to let people save single posts, or sync with likes, shares, tweets and retweets, so people can search with those facettes.

> Is your code all home-built, or you're using some FT engine compiled to > Wasm?

For the search we are using dexie.js, the rest is home-built. Our storage engine is: [https://github.com/worldbrain/storex](https://github.com/worldbrain/storex)

</comment></comment></comment></comment><comment><USER29> I feel like this idea pops up every two years or so; I was definitely running something like it in the early 2000s. How come none of the previous efforts succeeded?

Firefox's built in history is so, so close to being useful. My impression is it's hampered mostly by limited UI.

</comment><comment><USER30> Been using this for a week or so and am liking it so far. I've used full text search extensions before but this feels less opaque and more usable. Looking forward to continued development!

</comment><comment><USER31> Sorry to bother you, in Indonesia, MEMEX means female genital.

<comment><USER32> [https://en.m.wikipedia.org/wiki/Memex](https://en.m.wikipedia.org/wiki/Memex)

Theres no bother, since its a different culture and words are bound to mean something else.

</comment></comment><comment><USER33> The intro panel on the homepage (worldbrain.io) is dark grey text on a dark grey background. Is that intentional or a bug?

</comment><comment><USER34> This is really nice, now I can get rid of HistorySearch which does the same thing but stores everything on the cloud.

</comment><comment><USER35> Doesn't the search bar already have the option to search your history and bookmarks?

<comment><USER34> This one seems to index also the content of pages.

<comment><USER35> Ah. May make sense for some people. For indexing, I personally would prefer to use a bookmark service (self-hosted) that handles this.

<comment><USER34> Oh, there's a self-hosted one? Any link?

<comment><USER34> Nice, too bad the organization part is card-based, would have loved a drag/drop interface like with Bookmarkninja [1]

[https://www.bookmarkninja.com/images/dashboard3-650.png](https://www.bookmarkninja.com/images/dashboard3-650.png)

<comment><USER4> Oliver here, from the Memex team.

are you referring to Memex being card based? In which part of Memex would you like to have that drag&drop ability to organise stuff?

<comment><USER34> For organizing collections, that would be awesome!

<comment><USER4> How would you like to organise it? Like in your screenshot? Is it the ordering inside a collection that is important to you? Would it solve it if you were able to order items inside a collection?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER36> I have wanted this for years! Thank you!

</comment><comment><USER37> Safari has this built in, I'm surprised chrome and Firefox don't

<comment><USER38> What do you mean by "built in"?

</comment></comment><comment><USER39> How does this implement full-text search?</comment>


Talks I have given - <AUTHOR> http://dtrace.org/blogs/bmc/2018/02/03/talks/ <comment><USER2> I dunno how I missed this until now, but the lightning talk on tail -f is great

[https://www.youtube.com/watch?v=vm1GJMp0QN4#t=41m18s](https://www.youtube.com/watch?v=vm1GJMp0QN4#t=41m18s)

When you realize halfway through that the implementations of tail -f you've written are probably broken too :(

<comment><USER3> Well, my `tail' (GNU coreutils) watches the file for being truncated. So does a log file follower that I wrote. Is it really that difficult to imagine the log can shrink, especially if one spent plenty of time in logrotate's configuration? There are many daemons that can't reopen their log file.

<comment><USER4> _can 't_? good grief.

Can you trick them into logging into a FIFO, at least? Then put something sensible on the other end draining into regualr files that typical followers/processors can deal with at their pleasure?

<comment><USER5> Depends on the daemon. The most common version of this would be a daemon that starts as root, opens its log file, and then drops privs. In that simple case, yeah going to a named pipe would work fine. But if you stop reading from the pipe, it will presumably block when writing.

</comment></comment></comment></comment><comment><USER6> _Before we get to the list: if you only watch one talk of mine, please watch Principles of Technology Leadership (slides) presented at Monktoberfest 2017. This is the only talk that I have asked family and friends to watch, as it represents my truest self — or what I aspire that self to be, anyway._

Let me just say, this is an extremely good talk[1]. Very uplifting and depressing at different times. It goes through serious turns, and very funny periods. Ultimately though, the message is very important and worth hearing. I highly recommend it.

1: [https://www.youtube.com/watch?v=9QMGAtxUlAc](https://www.youtube.com/watch?v=9QMGAtxUlAc)

</comment><comment><USER7> I also found the somewhat more casual BSDnow podcasts that featured Mr. Cantrill quite insightful[1,2,3]. He shows up around 15 or 30 minutes in.

[1] [https://www.youtube.com/watch?v=l6XQUciI- Sc](https://www.youtube.com/watch?v=l6XQUciI-Sc)

[2] [https://www.youtube.com/watch?v=Ya6h2zKlpaQ](https://www.youtube.com/watch?v=Ya6h2zKlpaQ)

[3] [https://www.youtube.com/watch?v=dR3ntE9TXOE](https://www.youtube.com/watch?v=dR3ntE9TXOE)

</comment><comment><USER8> A few months ago I started binge watching every talk by Bryan I could find. I haven't found one that wasn't informative, entertaining, and just all around interesting.

I think there is something about the value system he puts forth that I appreciate. It brings some sensible balance to the industry for me.

Thanks Bryan. I hope you keep giving more talks.

</comment><comment><USER6> I love how reading his opening paragraph gives a good feel for his presentation style. I'm not sure if it comes across as well before you've seen one of his talks, but for me it evoked strong memories of humorous moments watching him.

</comment><comment><USER9> These are some awesome talks I've never seen, so thanks for sharing! I can start binging now.

Has he ever given a talk or written a blog post that gave pointers/tips on giving awesome talks? A lot of it is practice, I'm sure, but just curious. Did a quick search and couldn't find much, although I did find some good suggestions for running a successful conference.

</comment><comment><USER10> Thanks for the list! I've listened to several of your talks, but didn't know about some of these.

Nit: "unision" should be "unison" (in the description of the oral history talk). This misspelling lept out at me because of the way my screen reader mispronounced it.

<comment><AUTHOR> Fixed -- sorry about that!

</comment></comment><comment><USER11> I think this guy would do good in politics. Bryan for president !? About "making the world a better place" you would have a far better chance in politics then in software.

</comment><comment><USER12> I watched your Principles of Technology Leadership last night. Thank you for putting this up on HN, I may never have seen it otherwise.

The link to the slides for the above talk is broken: [https://www.slideshare.net/bcantrill/principles-of- technolog...](https://www.slideshare.net/bcantrill/principles-of-technology- leadership)</comment>


Librem One faces privacy and security criticisms at launch - <AUTHOR> https://twitter.com/YalePrivacyLab/status/1139039274987327489 <comment><USER2> This is terribly dissapointing.

I've been keeping my eye on Purism for awhile now, eagerly awaiting a more polished product lineup before purchasing. It wasn't even really a question of "if" I was purchasing, just a matter of "when".

This has stopped me in my tracks. I hope these issues are addressed by Purism, but I'm not going to keep my hopes up.

A security issue, although not ideal, is something that can be worked through and learned from. However, trying to underplay the severity of a security issue is a red flag for potential systemic cultural issues of the company which are much more difficult to fix than a vulnerability.

The issue of suggesting dangerous defaults for PGP/GPG is also worrying. Considering the target market demographic and the Purism mission statement, one would expect a large amount of research time into the defaults of any security or privacy related portion of the phone.

I hope to see more information on all of this - and I hope to be proven wrong. I'd love to see Purism succeed. We need more players in the security/privacy first market.

</comment><comment><USER3> my brother has been waiting for his purism phone for a long time now...i am eager to hear his review, but i'm sure news like this one help it much</comment>


Webcam face tracking in 25 lines of JavaScript. Very robust to poor lighting - <AUTHOR> https://jsfiddle.net/jeeliz/2p34hbeh/ <comment><USER2> Less than 25 lines isn't so great when you have to load in a separate library or two.

<comment><USER3> Yeah, you could do this in '1 line of code' if you just made another wrapper.

<comment><AUTHOR> Yes, but the library is still quite generalistic (it would not make sense for example to include the drawing of the yellow rectangle in the lib or to not have access to the raw detection parameters).

</comment></comment></comment><comment><USER4> Error while parsing the 'allow' attribute: 'midi;', 'geolocation;', 'microphone;', 'camera;', 'encrypted-media;' are invalid feature names.

<comment><AUTHOR> I think this error comes from JSFiddle : its executes the code in an iframe which has allow attribute. You can also test almost the same code here : [https://jeeliz.com/demos/faceFilter/demos/canvas2D/faceTrack...](https://jeeliz.com/demos/faceFilter/demos/canvas2D/faceTrack/)</comment></comment>


Goa: Design-First API Generation - <AUTHOR> http://goa.design/ <comment><USER2> This was/(is?) one of the goals of Swagger[0]. Lots of promises, but have yet to see it work in the wild. I want to use whatever Stripe uses for their API design.

[0] [http://swagger.io/](http://swagger.io/)

<comment><USER3> Yes, Swagger is intended to be a service description in this space. But goa seems to allow you to do your design in their DSL and then generate a Swagger spec from it [1].

[1] [http://goa.design/design/swagger/](http://goa.design/design/swagger/)

<comment><USER2> We must go deeper.

<comment><USER4> Todo: write a thing that makes Goa from a RailsAPI server. The goal is to have it be able to generate the same exact RailsAPI code. The usefulness is that a different team not as comfortable with Rails could use the generated Goa code to generate the same RailsAPI server in the language they're comfortable working with, make their contributions, and have _that_ generate new Goa, which the first team would use to generate the updated RailsAPI code.

Kind of a Rosetta Stone for APIs, but only useful if it works both ways, picks up _every small change_ in exacting detail, and works server ->spec ->same server. Not sure if that's even possible, but it'd be cool.

<comment><USER2> Have you seen this? [https://github.com/mame/quine- relay](https://github.com/mame/quine-relay)

</comment></comment></comment></comment></comment><comment><USER5> The creator of goa is presenting on code generation and APIs at Gophercon this year. I'm also presenting on goa at abstractions.io in August. I'm quite biased as a contributor, but I think goa is a great solution for APIs because it encourages you to think through the design completely and generates the code from that design. Generated code is idiomatic and clean. You can see me present at the Tampa Go Meetup here: [https://www.youtube.com/watch?v=tCFrgWikEX8](https://www.youtube.com/watch?v=tCFrgWikEX8)

<comment><USER6> How does this compares to gRPC which has many languages supported already?

<comment><USER5> gRPC is a great framework for generating RPC servers and clients, it's a different space. goa is built for creating an entire REST API.

</comment></comment></comment><comment><USER3> I'm really happy to see this because I hear people talking about using Go to implement microservices, but fewer mentions of what tools they're using to implement them.

This looks like a good 'toolkit' to ease the development of such a service. There are also some code examples, e.g. to implement authorization with JWT or OAuth2: [https://github.com/goadesign/examples/tree/master/security](https://github.com/goadesign/examples/tree/master/security)

</comment><comment><USER7> Both Swagger and Goa can generate API client code from their respective spec languages. Goa additionally has the ability to generate a Swagger spec from a Goa spec.

This makes me wonder why Goa just doesn't rely on the Swagger code generators? Are they not very good?

The Goa page talks about generating Swagger so you can use the Swagger doc generator. Maybe that's the only part of Swagger that the Goa developers think is useful?

Also, Swagger has more traction right now. Maybe the ability to generate a Swagger spec lets people not feel so locked in to Goa, the newcomer?

<comment><USER3> They approach the same problem from opposite directions. Goa's DSL will let you code a server, and then export a Swagger spec that clients can consume (and codegen their end from).

Whereas Swagger's code generators start with the Swagger spec already written. You can use them to generate a server, which will create the scaffolding but leave you with the logic to implement.

<comment><USER8> Swagger can also generate its spec from an already written server, and that's how I'm using it primarily.

<comment><USER3> Would you mind linking to some docs on how to do that?

I only found this quote, which hints that it's a manual process (if your framework doesn't produce the spec):

"Either you create the definition manually (using the same Swagger Editor mentioned above), or if you are using one of the supported frameworks (JAX-RS, node.js, etc) [1], you can get the Swagger definition generated automatically for you." [2]

[1] [http://swagger.io/open-source-integrations/](http://swagger.io/open- source-integrations/) [2] [http://swagger.io/getting- started/](http://swagger.io/getting-started/)

<comment><USER9> There are many plug-ins to to generate OpenAPI/Swagger spec from annotations in the source code.

For C#, please try [https://github.com/domaindrivendev/Swashbuckle](https://github.com/domaindrivendev/Swashbuckle)

For PHP, please try [https://github.com/zircote/swagger- php](https://github.com/zircote/swagger-php)

For other langauges, just do a google search on "{lang} swagger" and you will probably find something useful.

</comment></comment></comment></comment></comment><comment><USER10> I've been looking for the Google open source beta version of this. It's been killing me that I can't remember what it's called... I believe it's done in either C++ or Golang, green/white home page

<comment><USER11> GRPC? Not the same, but allows you to specify the API in a static file and generate native language clients among many other things.

<comment><AUTHOR> [http://www.grpc.io/](http://www.grpc.io/)

[https://github.com/grpc/grpc](https://github.com/grpc/grpc)

[https://developers.googleblog.com/2015/02/introducing- grpc-n...](https://developers.googleblog.com/2015/02/introducing-grpc-new- open-source-http2.html)

</comment></comment></comment><comment><USER12> Which font is the one used in the first few editor screenshots on the Goa site?

<comment><USER13> Pretty sure it's PragmataPro. Dotted zero's, very thin 'm', looptail 'g'.</comment></comment>


Doug Hofstadter: A Person Paper on Purity in Language (1985) - <AUTHOR> https://www.cs.virginia.edu/~evans/cs655/readings/purity.html <comment><AUTHOR> "Perhaps this piece shocks you. It is meant to. The entire point of it is to use something that we find shocking as leverage to illustrate the fact that something that we usually close our eyes to is also very shocking. The most effective way I know to do so is to develop an extended analogy with something known as shocking and reprehensible. Racism is that thing, in this case. I am happy with this piece, despite-but also because of-its shock value. I think it makes its point better than any factual article could. As a friend of mine said, "It makes you so uncomfortable that you can't ignore it." I admit that rereading it makes even me, the author, uncomfortable!"</comment>


PHP 5 and 7.0 end of life - <AUTHOR> https://hackertarget.com/php-end-of-life <comment><USER2> Yeah, yeah. Go tell that to my 10x php self-taught guy who's still running php 5.4 on a windows server 2003 in a wamp installation. He upgraded the bundled mysql with mariadb last year by dropping the executable in the folder (edit: just checked out, and the drop-in has been reverted back to mysql 5.0 because the upgrade path is impossible because apache 2.2).

And of course this in-house guru for anything that happens on a screen patiently overwrites hacked 1.6 joomla installations by copying over an old copy and doesn't bother updating the passwords.

I am in the process of dockerizing almost everything in sight. There's pain.

<comment><USER3> Besides the "doesn't bother updating the passwords" point, this sort of "it's old so let's change it" attitude is unfortunately so pervasive in the software industry and I find it massively irritating because it's the same reason why a lot of software has gotten increasingly unstable over time. Instead of making only fixes and bringing a platform towards maturity they do the exact opposite, adding plenty of new bugs and breaking changes.

Keep in mind that the majority of people who need PHP (and technology in general...) have far more important things to do with their time than incessant trendchasing.

The saying about "known unknowns" and "unknown unknowns" is worth remembering too. The fact that a system has stayed working for over a decade should be enough to tell you that its creators did something right.

(I am in the process of restoring an 80-year-old refrigerator. ;-)

<comment><USER4> _" it's old so let's change it"_

That's very rarely the thought process in the real world when you're dealing with a production system though. Most developers who have the seniority and experience to make the call to upgrade something _aren 't_ just upgrading for the fun of it. It's more like...

"it's old _and hundreds of security patches and performance improvements have been released in versions that have come along since it was current, and the core developers aren 't supporting this version any more so we're in for a world of hurt if anything goes wrong, and we have the bandwidth to fix this problem right now where we might not in the future_ so let's change it"

<comment><USER5> But that sounds exactly what GP is doing, "I am in the process of dockerizing almost everything in sight."

Why do you need Docker to run a WAMP server?

<comment><USER6> You don't _need_ it, but it certainly helps you know what your environment is so you can upgrade/patch one piece at a time without affecting everything. Need a new libc for your database? Feel free to upgrade it without worrying if your webserver or plugins will break. Also, you can more exactly replicate your production environment on your local development machine or replace production if the machine dies.

I love docker for anything where I'm not going to be running whatever is in the repository of whatever Linux distribution I'm running.

</comment></comment></comment></comment></comment><comment><USER7> For a second, after reading the title, I thought PHP as a whole was end of life. That'd be fun.

Which also made me think - does anybody know an example of a widely used programming language that voluntarily discontinued?

<comment><USER8> ColdFusion and ActionScript

<comment><USER9> ColdFusion's still around. [https://www.adobe.com/products/coldfusion- family.html](https://www.adobe.com/products/coldfusion-family.html)

<comment><USER10> I miss ColdFusion. I really liked the idea of a tag based server side language. It will never have the power or expressiveness of other languages but for raw productivity of producing very simple dynamic apps with an incredibly low bar to entry for new/non-programmers, nothing else was even close.

<comment><USER8> > _nothing else was even close_

Well, I'd argue that PHP and Python were roughly equally easy, although they didn't "blend" into the HTML as well (which is good or bad, depending on your viewpoint). ASP was supposed to be just as easy, but I remember some of those code bases being pretty woolly.

Nowadays, I think we actually have it easier. I haven't used many of the other front-end frameworks, but you can design/generate a back-end in a few minutes using Open API Spec, add a persistence layer in a few more minutes, and wire it up to a Vue front-end very quickly. And the kicker is that it's modular and maintainable, which a lot of CF/PHP/ASP projects certainly were not.

<comment><USER11> > "Nowadays, I think we actually have it easier...."

no, it's not even close. you'd have to learn a whole javscript framework, read the open api spec, grok backend vs frontend, and really understand the modern web before you could even get started, not to mention the fomo induced by the hundreds of other options you'd have to discard in the process. for the average human, that's a non-starter.

with tag-based coldfusion, it felt like you were learning an extension to html, rather than an entirely different (programming) language. while php/asp were somewhat easy to learn, coldfusion was extra-super-duper-easy (i taught classes back in the day). people with no prior experience could create a dynamic web page in an hour or two and feel that sense of accomplishment that drove them deeper into it.

</comment></comment></comment></comment></comment></comment><comment><USER12> Note that multiple Linux distributions will still be supporting it, in some cases for a very long time after.

<comment><USER13> Exactly. These EOL dates aren't really helpful in a world where most people just use a handful of distros. WordPress keeps sending the same unhelpful message and I hate it.

Just the other day, a somewhat technical client of mine called me to ask if his PHP 7.0 installation has gaping security holes. I said no, you're using Ubuntu 16.04 LTS, you'll be fine until April 2021. If you force an upgrade now, you might run out of support even earlier.

My go-to version for production right now is PHP 7.2. I'm going to ignore all warnings about its EOL until 2028, when support for Ubuntu 18.04 runs out. Wait a sec, add another year for RHEL/CentOS 8 which is also going to support PHP 7.2 for the next 10 years. PHP 7.2 is going to outlive a whole bunch of future versions.

<comment><USER14> > My go-to version for production right now is PHP 7.2. I'm going to ignore > all warnings about its EOL until 2028, when support for Ubuntu 18.04 runs > out.

I'm not sure you can just hide behind the LTS assurance of the OS, they can't guarantee every single package in their repos will remain safe. Plenty of packages in Ubuntu LTS releases reach EoL far far before OS EoL.

<comment><USER13> True, but PHP tends to be looked after fairly well given its importance to the web ecosystem.

Ubuntu for example has a pretty good track record of backporting PHP security fixes. PHP 7.0 in Ubuntu 16.04 has been getting updates every few weeks despite the EOL last winter, and I remember observing the same with PHP 5.5 in 14.04 until the OS itself reached EOL earlier this year.

The fact that both Red Hat and Canonical are committed to supporting PHP 7.2 for the next decade probably means that there will be more eyes on that particular version, and more hands to patch it, for the foreseeable future. It's a nice coincidence for people who want a bit of stability now that PHP has begun to change rather quickly.

<comment><USER14> Interesting, that's quite a lot of effort to keep the old versions safe.

I wonder if any of that work get up-streamed into the old versions for the benefit of other distros?

<comment><USER13> Ubuntu and CentOS are maintaining versions that upstream has already EOL'd, so there's no upstream to push patches to. Most other distros (hello, Fedora) have much shorter support cycles so they don't really need those patches, either. There might be some sharing between Ubuntu and Debian LTS.

The majority of patches still come from upstream. If a new vulnerability is found in PHP 7.1 or 7.2, chances are the same bug exists in 7.0. So the maintainers check if their versions are also vulnerable, and if so, backport the fixes. As time goes on and differences accumulate, patches from upstream will become less relevant. It looks like Red Hat largely gave up on maintaining PHP 5.3 in RHEL/CentOS 6 after about 7 years. We'll have to see how long they actually stand behind PHP in CentOS 7 and 8.

</comment></comment></comment></comment></comment></comment><comment><USER15> Ha. I know that shop that played themselves into a corner many years ago by "customizing" their Doctrine installation, thereby cutting themselves off of _any_ future upstream updates and upgrades. If they were to get up-to-date now, they would have to rewrite their entire stack, including their magic Doctrine queries. They did hire one(!) Java developer to rebuild most of it in Java, but as usual in such cases, nobody has the time to explain to him how the old tangled mess actually works.

</comment><comment><USER16> Somebody should fork PHP 5.6. There is a lot of Software still depending on it and it is unrealistic to expect companies to migrate millions of lines of third party code to PHP 7.

<comment><USER17> I really hope nobody does this. OS vendors backport security vulnerabilities, and that should be it.

Like the other commenter said, PHO 5.6 to 7 upgrade isn't that hard. If it is, there's a very good chance they are doing something devastatingly wrong (like using mysql_ extension).

<comment><USER18> There are companies who do this for Ruby on Rails. If someone can't upgrade, for whatever reason, they can pay a third party for security updates. It works pretty well.

</comment></comment></comment><comment><USER19> isn't that ... too soon? the main reason i use PHP is because there is no time to rewrite stuff.

<comment><USER20> PHP 5 has been deprecated for several years now. PHP 7.0 was released in 2015, almost five years ago. The current version is PHP 7.3.

While upgrading from PHP 5.4 or lower to 7 can be... interesting, it's not all that difficult or time-consuming. PHP 5.5 or higher to 7 is much easier.

<comment><USER21> We personally struggle to just keep Ubuntu updated with LTS releases, let alone managing all the updates to ensure software works on the new packages installed on whatever LTS version.

The issue is that when you have smaller companies with small web development departments you end up with a lot of differents sites/tools made over time. So now when it comes upgrade time you have a large pile of stuff that needs upgrading. One developer making new websites over a few years time isn't that crazy, so unfortunately you end up with a build-up of projects to update.

Yet while having time pressures to update, you also have pressures to continue to work on new things or fixing problems users have reported.

It usually comes down to time/money. Unfortunately dedicating time to updating things is never as high of a priority as making new things, adding new features, or fixing bugs. Because it follows the traditional mindset of it's working, why update it.

<comment><USER20> I get it. I'm a consultant, so I'm familiar with code bases that are a little...well-worn.

If the business decides the risk is acceptable, _then that 's fine_. As an engineer, you're supposed to communicate and document the risks associated with technical decisions. After that, if your employer decides Technology X.Y is good enough, then accept it and move on. You made your case, and the business considered it and made a choice.

<comment><USER21> The risks are usually known, the problem is more that it's a slow process to get everyone to update their stuff and have it ready for new versions of whatever languages and tools they are using.

</comment></comment></comment></comment></comment><comment><USER22> Fwiw, I was really impressed with my hosting provider (Siteground), a couple of months ago they warned about deprecation and sent a report saying they could autoupdate PHP but a plugin on my Wordpress wasn't ready. Top service IMO.

It's [modded] cPanel hosting, with extras, so perhaps that's a cPanel thing but as a very small scale setup it helped a lot to be forewarned of incompatible plugins.

</comment><comment><USER23> I have one site with phpbb 3.0 also stuck on php 5.6. It has a bunch of mods which phpbb stopped supporting and switched to extensions.

</comment><comment><USER24> Are there any issues running PHP 5.6-based code on a PHP 7.x VM?

<comment><USER25> Very few. It's worth running one of the code analysers over your codebase to check: [https://github.com/PHPCompatibility/PHPCompatibility](https://github.com/PHPCompatibility/PHPCompatibility)

I upgraded a large project last month, and this found 6 changes I had to make in 20k LOC.

</comment></comment><comment><USER26> 6 Sites running NT 4.0??????

</comment><comment><USER27> clickable title

</comment><comment><USER28> Wait what?

PHP 8 isn't even out yet. How can PHP 7 already be deprecated and not even supported?

<comment><USER29> PHP 7.0 is eol, not php 7.1, 7.2, and 7.3.</comment></comment>


Litecoin SegWit activation reached - <AUTHOR> http://litecoinblockhalf.com/segwit.php <comment><USER2> Litecoin doubled in the last 2 months. IMHO, it will happen again in the next few months. How do you think SegWit activation will affect litecoin?</comment>


Census: Segregation hits 100-year lows in most American metro areas - <AUTHOR> http://www.csmonitor.com/USA/Society/2010/1214/Census-Segregation-hits-100-year-lows-in-most-American-metro-areas

<comment><USER2> I found this graph a while ago. It illustrates the segregation in Chicago:

<http://www.radicalcartography.net/index.html?chicagodots>

I moved from Arizona to Chicago a few years ago. While Arizona was far from perfect, I still find the segregation here shocking. While standing on a platform waiting for an El train, it is possible to determine which people will get on which train line based on the color of their skin.

Edit:

Another anecdote, but interesting to me. My wife and mother teach at two different "urban" schools on the south side. There are no white students at either school. My mom's school is 99% black and my wife's school is 95% Hispanic. I know it's a common situation but it still just blows my mind.

</comment><comment><USER3> I wish news articles would stop conflating segregation with racial imbalance. The former is but one possible cause for the latter.

 _"Racial imbalance is not segregation. Although presently observed racial imbalance might result from de jure segregation, racial imbalance can also result from any number of innocent private decisions, including voluntary housing choices."_ \-- Clarence Thomas

<comment><USER4> The author simply doesn't know what segregation was at all. If all the black people lived in one neighborhood and all the white in another, and they never crossed, why would you segregate bathrooms? And restaurants? And water fountains? And schools? And busses?

Housing was already integrated and the services were segregated on skin color. That was one of the quantifiable objections to segregating school other than the obvious moral reasons - the students had to walk or bus further to school when there was a perfectly good building closer to their homes reserved for white people. That's segregation.

Real estate agents preventing black people from buying near whites was called Blockbusting, and was different.

<comment><USER3> _Real estate agents preventing black people from buying near whites was called Blockbusting, and was different._

You are thinking of redlining and steering. Blockbusting was a form of fraud/extortion preying on racist homeowners:

Real estate agent: "Hey homeowner, blacks are moving into the neighborhood."

Homeowner: "I hate blacks! I'll sell my house at below market rates to get away from them!"

<http://en.wikipedia.org/wiki/Blockbusting>

</comment></comment></comment><comment><USER4> Journalists misuse (and overuse) the word "exponential" more than any other. As in:

"Yet growing residential proximity between whites and blacks also contains the potential for _exponential_ progress in bridging America's long-standing racial divide, especially as families from different racial backgrounds share schools and grocery stores." (emphasis mine)

Maybe there is potential for an improvement in the pace of reducing that divide, but I highly doubt it can be described as exponential.

Though I guess that is a small nitpick. Probably a bigger one is that in their words it is a "complicated story with lots of nuances" that they boil down at the end to simply gerrymandering congressional districts and how it will change the balance of Democrats/Republicans. I'm sick of every news story coming down to politics, and by politics I mean electioneering.

</comment><comment><USER5> It's interesting to me that segregation of Asians isn't mentioned. By my own observation (which may obviously be flawed), the integration of Asians outside of urban Chinatown-like areas is pretty much complete.

In particular, Americans of Chinese ancestry (with which I'm more familiar, since my wife came from China) can be found anywhere in suburbia.

<comment><USER6> Asian immigrants and especially Asian-americans have higher IQs and incomes than American whites. They can afford to go anywhere they want, while generally poorer black or Hispanic people get the neighborhoods no one wants.

<comment><USER7> lol, as an Asian American myself, while thankful for your comment, I'm sure that's all sorts of wrong.

<comment><USER6> Hold your head high, my friend!

<http://en.wikipedia.org/wiki/Race_and_intelligence> \- "Intelligence quotient (IQ) tests performed in the US have consistently demonstrated a significant degree of variation between different racial groups, with the average score of the African American population being significantly lower—and that of the Asian American population being higher—than that of the White American population."

<http://en.wikipedia.org/wiki/Asian_American> \- "As of 2008, Asian American households had the highest median income at $65,637; however, 11.8 percent of Asians were in poverty in 2004, higher than the 8.6 percent rate for non- Hispanic whites.[8] Much of this poverty is concentrated in ethnic enclaves such as Chinatowns in the cities[38] Census figures also show that a white male with a college diploma earns in excess of $66,000 a year, far more than similarly educated Asian men who earned more than $52,000 a year.[39] Asians however are more likely to complete higher education particularly and are the highest group by percentage with graduate degrees"

</comment></comment></comment></comment><comment><USER8> On a related note:

[http://www.gq.com/news-politics/mens-lives/200810/devin- frie...](http://www.gq.com/news-politics/mens-lives/200810/devin-friedman- craiglist-oprah-black-white-friends-obama)

This article, while somewhat old, does a great job of discussing just how shallow that 100-year "achievement" really is. We may live near each other, but that doesn't mean we interact in non-superficial ways.

</comment><comment><USER9> FWIW - the feds restructured how ethnicity is defined for the 2010 census. Now if you have multiple ethnicities you are reported as such - before it was only what you considered your primary race. This will balloon diversity figures in formerly non-diverse areas. I expect to see many media-filler articles such as this once the full census report is published.

There's some additional depth to how the Hispanic grouping is classified. Essentially if you self report any Hispanic lineage you are considered a primarily Hispanic individual.

</comment><comment><USER10> While I can see why Gerrymandering is/was necessary to maintain the minority voice in regions, has it become less socially relevant to do so? Does anybody have an idea when Gerrymandering based on geographic location will "end" because people are too well integrate? Not like that may be any time soon, but it will probably be more the case than not sometime in the future. I guess we're always looking for reasons to redraw the lines in favor of our parties...and we'll use any reason to do so.

While I see it as a necessary evil in some cases, by and large, Gerrymandering drives me insane.

</comment><comment><USER11> We've finally re-achieved the pinnacles of 1910.

</comment><comment><USER12> Eric Fisher who also produced the tourists/locals photo maps has some pretty dramatic visualizations of racial distribution in various metro areas: [http://www.flickr.com/photos/walkingsf/4981444199/in/set-721...](http://www.flickr.com/photos/walkingsf/4981444199/in/set-72157624812674967/)</comment>


Unsmear: Convert to and from timescales with smeared leap seconds - <AUTHOR> https://github.com/google/unsmear <comment><USER2> (Dustin Hoffman's voice) I hate, I Hate, I HATE Smeared Time!

I must admit my own personal bias in this matter, because I am in the process of implementing a GNSS receiver that very much cares about both the speed and phase of time. It seems that the computing industry would rather define the second based the mean solar day of Earth's rotations, despite the rest of the world defining it to be the number of shakes of a Cesium atom partway down Earth's gravity well.

But instead of slaving their clocks to the international system that actually measures the mean solar day (UT1R), they decided to muck with both the speed and phase of time. For just one 24 hour period. Every once in a while.

If I was king for a moment, and could pronounce just one edict it would be this: To transmit UT1R's offset in the GPS C/NAV message, and demand that the NTP timebase slaves itself to that.

<comment><USER3> > despite the rest of the world defining it to be the number of shakes of a > Cesium atom partway down Earth's gravity well.

Hang on, the number of shakes should be the same regardless of where you are in a gravitational well.

<comment><USER2> The timebases which are transmitted by the GNSS systems are in turn synchronized to the TAI network, which are all located on Earth's surface. Time does appear to be passing noticeably slower on Earth's surface compared to precision clocks in orbit. Even on the surface if you measure very carefully, you'll find that the SI second varies with respect to TAI depending on your altitude above mean sea level, too.

Civil time is therefore already partly divorced from the SI second due to this effect. Why not divorce it the rest of the way?

<comment><USER3> I'm not disputing that clocks at altitude _appear_ to run faster relative to clocks at sea level - as you said _time_ is passing slower down here. But a second is always and everywhere the same 'number of shakes' locally.

</comment></comment></comment></comment><comment><USER4> > Although this is not an officially supported Google product, you can reach > us on the unsmear-discuss mailing list.

I'd say that this component has probably much better support than any of those "officially supported" Google products. Not wanting to blame Google for this though, they have billions of customers and providing good customer support for them would be insanely expensive.

</comment><comment><USER5> Why were leap seconds (basically utc vs tai) accepted so readily and widely? What does it matter if civil vs solar time would have slowly drifted 10 minutes over a millenia or something like that?

<comment><USER6> Almost every device would be inaccurate and impossible to synchronize with military standards, causing huge headaches in defense industry.

Why not instead use a correct second?

<comment><USER7> GPS time has no leap seconds though.

</comment></comment></comment><comment><USER8> Oh man, I thought the point of smearing is to avoid the need for such conversions.

<comment><USER9> Smearing is very convenient if you use it because time stays "normal". But if you need to precisely correlate with outside sources, what other option do you have?

<comment><USER8> So when _do_ you need subsecond-acccurate correlation of timestamps with outside sources?

<comment><USER2> My ground control systems are sitting in a hosted datacenter with an entirely- unlike-UTC timebase.

My satellite is synchronized to TAI via GPS time.

I coordinate with a third party who manages the ground-based radio to coordinate contacts with my satellite system.

If something goes wrong, it would be nice to compare the logs from the three systems after a contact to see which node thought what was going on.

The fact that one or two of those systems gets its time from a not-UTC source makes this problem harder than it needs to be.

</comment></comment></comment></comment><comment> 1-6 Entering "2016-12-31 23:59:60" in a POSIX converter will fail and XML will reject such entry as "invalid time".</comment>


Ubuntu 16.04.2 LTS released - <AUTHOR> https://lists.ubuntu.com/archives/ubuntu-release/2017-February/004036.html <comment><USER2> I don't understand why Ubuntu is receiving so much bad comments from Linux community. Canonical is doing great job of putting, fairly reliable system on massive number of devices, something that other distributions can just dream about. In my opinion, currently, Ubuntu is the best general purpose Linux distribution for new and semi advance users.

<comment><USER3> I agree with you. I've been using Linux since the day Linus announced it on minix-list, and for the last 12 years Ubuntu has been my daily-driver system .. and it is amazing. I'm a musician, so when people see that I'm using Ubuntu Studio for my DAW, I usually hear a few chuckles .. until I fire up Ardour, show them my plugin list, the standard suite of synths and effects I use, and so on .. that usually shuts them up at least. But what gets them _really_ curious is when I show them how easy it is to break out the source code for _any_ of these powerful tools, make modifications, re-install and maintain my system with very, very powerful techniques.

There simply isn't any operating system as conducive to creative tinkering and progressive enhancement of key software as Ubuntu Studio. Its way, way ahead of the pack in this regard, and I think anyone who scoffs at the idea needs to be taught the lesson that Ubuntu - and of course, Linux and the ecosystem it promotes - is really worth the effort to know, learn and understand.

<comment><USER4> Looks like you got a lot of feedback from your comment, but if you have the time I'd like to ask a question too. I'm still getting my feet wet with more the more advanced aspects of Linux, primarily Ubuntu on my Chromebook. I always hear about people tweaking programs, but it never dawns on me what kind of tweaks are possible or even useful. I'd be interested in a couple use-cases you've come across, like problems you wanted to solve with a program and a brief overview of how you altered the source to do so. Maybe it's embarrassing to admit, but I've never tweaked anything outside of VS Code or Atom.

<comment><USER5> First, it is definitely not embarrassing to admit, and the answer may be overwhelming, but you can customize nearly everything.

One of the simpler things is to customize the look and feel. E.g. you could try out different desktop experiences and WMs (window managers). Most of them are just an `apt-get` away and when you logout and login again you can choose and try out a different desktop.

</comment></comment></comment></comment><comment><USER6> Here is a summary of what is said in [https://wiki.ubuntu.com/Kernel/LTSEnablementStack](https://wiki.ubuntu.com/Kernel/LTSEnablementStack) regarding the 16.04.x versions

1\. If you are happy with how Ubuntu 16.04 works for you, you get to keep it and you receive support until 2021.

2\. With Ubuntu 16.04.2, you get the option to switch to a new path of updated Linux kernels. If you do so, your Linux kernel will get updated every six months, until 2021.

For the first update with Ubuntu 16.04.2, you can enable to get the 4.8 kernel that was used/tested in development version of Ubuntu 16.10.

In the subsequent update with Ubuntu 16.04.3 (around July 2017), you will be updated to that Linux kernel that was used/tested in Ubuntu 17.04 (to be released in April 2017). And so on.

The command to switch you to the new path of updated kernels (updated every six months), is

sudo apt-get install --install-recommends xserver-xorg-hwe-16.04

<comment><USER7> One key item to note is that switching to upgraded kernel path breaks live kernel patching at this time.

I was considering switching I saw this caveat...

"For clarity, the Canonical Livepatch Service is only available and supported against the generic and lowlatency GA kernel flavours for 64-bit Intel/AMD (aka, x86_64, amd64) builds of the Ubuntu 16.04 LTS (Xenial) release. HWE kernels are not supported at this time."

[https://wiki.ubuntu.com/Kernel/RollingLTSEnablementStack](https://wiki.ubuntu.com/Kernel/RollingLTSEnablementStack)

Also, it's not clear if there is a different kernel/command for upgraded kernels on a server.

EDIT: looks like it's going to be "linux-generic-hwe-16.04"

<comment><USER8> Do people see evidence of the live patching system doing something?

For me, canonical-livepatch status --verbose has never showed me any fixes, running linux-image-generic on 16.04.

<comment><USER7> Not really so far. I enabled it a couple months ago but have not seen any changes.

It's not clear to me if upgrading to HWE would correctly disable the livepatching.

</comment></comment></comment></comment><comment><USER9> I think the coolest thing introduced here is that the HWE kernel is going to become a standard feature of LTS releases going forwards.

<comment><USER10> Unfortunately, Alternate release images are not published after 16.04.1.

<comment><USER11> So are you saying they won't make any more alternate releases for future releases?

<comment><USER10> Yes. It was that way for the 14.04 LTS, too.

</comment></comment></comment></comment><comment><USER12> Interesting bit from the release notes:

Python 3: Python2 is not installed anymore by default on the server, cloud and the touch images, long live Python3! Python3 itself has been upgraded to the 3.5 series.

<comment><USER13> What that means is that all of the build scripts in Ubuntu 16.04 have been upgraded to Python 3 and building no longer has a dependency on Python 2. One way of looking at it is that Python 2 is not included with the current release of Ubuntu for the same reasons that MIT Scheme and PHP and Forth are not. The system does not require them.

<comment><USER12> Well, yes, but the interesting bit is that some notable organization just finished their move from 2 to 3.

<comment><USER13> Some notable organizations are planning not to move and more or less forking: [https://opensource.googleblog.com/2017/01/grumpy-go- running-...](https://opensource.googleblog.com/2017/01/grumpy-go-running- python.html)

Python 2 will be with us for a long long time.

<comment><USER14> Any other links, bar this one (which isn't really much more than a proof of concept)?

</comment></comment></comment></comment></comment><comment><USER15> Newbie Ubuntu user here. Currently I have the 16.04LTS (I don't know if mine is 16.04.1 but I downloaded and installed this version of mine the day 16.04LTS officialy released last April 2016). Should I upgrade to 16.04.2? If so, how? I mean, do I have to download the 16.04.2 installer or is there a update command?

<comment><USER16> You can run the Sofware Updater any time you want, including setting it to run automatically.

[https://www.ubuntu.com/download/desktop/upgrade](https://www.ubuntu.com/download/desktop/upgrade)

<comment><USER15> I actually run that every morning. So good to know :)

</comment></comment></comment><comment><USER17> Anyone using ZFS with Ubuntu? Any problems? Anyone tried btfs?

<comment><USER18> I ran with BTRFS for a while. It was pretty nice overall, but you have to be aware of which features are production-ready and which ones aren't. And the wiki isn't up to date either. I had to ask questions in the IRC channel because I didn't feel like wading through mailing list archives which is apparently the only place up-to-date info gets written down.

Edit: This is new. [https://btrfs.wiki.kernel.org/index.php/Status](https://btrfs.wiki.kernel.org/index.php/Status) I was hoping that it would grow into a more stable, user-friendly project. But RAID1 was broken for months, it would let you create RAID5/6 volume even though the feature wasn't even _finished_ yet, and I personally ended up with a filesystem that will crash the kernel when I try to read certain files. I recovered most of the data using a virtual machine that I could reboot quickly. Maybe I'll look at it again in a couple years, see if someone is taking the project seriously.</comment></comment>


Our iPhones Are Depleting the Earth's Resources  - <AUTHOR> http://mashable.com/2012/09/28/sustainable-technology/

<comment><USER2> It seems tech writers will go to any lengths to get "iPhone" in a headline, even further if it will cast iPhone in a negative light. But this is truly absurd. As in so many of these sensationalist stories, the iPhone is by no means unique; all consumer electronic products use the same materials. In addition, "rare earth elements" are not rare. They are found in abundance all over the planet. The only issue is cost of extraction. If they get scarce, the price will go up and this will incentivize mining in places other than China.</comment>


A lesson in shortcuts (2012) - <AUTHOR> https://plus.google.com/101960720994009339267/posts/R58WgWwN9jp <comment> ainar-g This might be an out-of-nowhere question, but seeing how we're discussing Rob Pike and Plan 9 here, why does Plan 9's rc shell need the $path variable, when you could just bind all binary directories onto or behind /bin/?

</comment><comment><USER2> If you want to read the HN comments on this from 7 years ago:

[https://news.ycombinator.com/item?id=4331855](https://news.ycombinator.com/item?id=4331855)

<comment><USER3> That HN discussion is mentioned by the top commenter on G+, saying: "The discussions there were hardly interesting, however", which I thought was kinda funny.

</comment></comment><comment><USER4> From comments, not sure if this is a joke itself.

 _dd is horrible on purpose. It 's a joke about OS/360 JCL. But today it's an internationally standardized joke. I guess that says it all._</comment>


Broken Genius – the case of William Shockley (2013) - <AUTHOR> https://web.archive.org/web/20180905024656/https://rupensavoulian.com/2013/04/05/broken-genius-the-case-of-william-shockley/ <comment><USER2> Shockley was apparently awful to work for and reportedly tried to steal the credit for Bardeen and Brattain's transistor work. All in all he doesn't seem to have been a very nice person.

"He (Shockley) called both Bardeen and I in separately, shortly after the demonstration, and told us that sometimes the people who do the work don't get the credit for it."

"Shockley took the lion's share of the credit in public for the invention of transistor, which led to a deterioration of Bardeen's relationship with Shockley. Bell Labs management however consistently presented all three inventors as a team. Shockley eventually infuriated and alienated Bardeen and Brattain, and he essentially blocked the two from working on the junction transistor."

[https://alixus.wordpress.com/undeserved-nobel-prizes- transis...](https://alixus.wordpress.com/undeserved-nobel-prizes- transistor-2/)

Of course the FET transistor was actually invented by Lilienfeld twenty years earlier and was long since patented. Shockley and team were aware of it and referenced it in their own work. They were working around Lilienfeld's patent by doing things slightly differently with the BJT. So awarding Shockley's team the Nobel Prize for the invention of the transistor seems to overlook poor old Lilienfeld rather badly.

[https://en.wikipedia.org/wiki/Julius_Edgar_Lilienfeld](https://en.wikipedia.org/wiki/Julius_Edgar_Lilienfeld)

And in the end the transistors in your CPU are FETs derived from Lilienfeld's design, not the BJT of Shockley's team.

<comment><USER3> FETs weren't desirable at the time though - which is why they didn't attain widespread use. Its very difficult to get linear behavior from a FET and at that time the primary use of the devices was analogue in linear mode. The BJT was more than just a patent workaround.

These days we can make linear FETs although they are still rare. Most are intended to be used in saturation as part of digital circuitry.

<comment><USER4> Ironic that BJTs were developed to work around FETs' undesirable nonlinearities for analogue systems, while now FETs are used to work around BJTs' undesirable inefficiency in switchmode use.

Just goes to show that in engineering, "how good is X?" is a meaningless question without saying what X is used for.

</comment></comment></comment><comment><USER5> >Psychologists and biologists currently regard the controversy as outdated, and speak of the interaction between genes and environment.

As examined at length by Steven Pinker's Blank Slate, they regard the controversy as outdated because nature won. Definitely. We've found even enviroment itself is heavily impacted by genetics, as people tend to strongly self-select what kind of day-to-day conditions they live in.

[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4739500/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4739500/)

>Shockley, by advocating such genetic-determinist views on race and intelligence, seemed like an atavistic throwback, to a time in America's history when immigrants from Southern and Eastern Europe (particularly Jewish immigrants) were screened out because of their alleged intellectual inferiority to the superior Nordic races.

Considering the Manhattan project had so many Eastern European jews involved it was joking referred to as a "Hungarian high school science fair project" this seems unlikely. But it wouldn't be the first time a racist ostensibly interested only in 'the biology of intelligence' conspicuously ignored the Jewish community's intellectual accomplishments.

<comment><USER6> > As examined at length by Steven Pinker's Blank Slate, they regard the > controversy as outdated because nature won. Definitely.

How is this supported by your link, whose second bullet point is "No traits are 100% heritable"?

<comment><USER5> The state of the field previously was anyone could be anything in the right enviroment. For example the following quote from John B. Watson:

"Give me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I'll guarantee to take any one at random and train him to become any type of specialist I might select – doctor, lawyer, artist, merchant-chief and, yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors."

For many years this blank slate view predominated, on every topic of human nature. Leading to some tragic results occasionally, such as the case of David Reimer in which attempts to socialize him as a girl after a botched circumcision against his will likely contributed to his suicide.

Eventually the evidence became overwhelming that this view was faulty. But more than that, the wishy washy "everything is nature and nurture all mixing together" fallback position was although not explicitly wrong profoundly disingenuous. Who you are is almost entirely contained within your genes, with a minority contribution from enviroment and parents and schooling. Nature won the debate, as profoundly as it was ever possible to win it, and it annoys me seeing equivocation in the discussion of the matter.

<comment><USER7> > Who you are is almost entirely contained within your genes, with a minority > contribution from enviroment and parents and schooling. Nature won the > debate

It would be more precise to say that maximum potential is largely genetic. Watson was wrong in that sense, but right in the opposite sense: an extremely bad environment - malnutrition, disease, pollution, very poor or no education, and so on - can definitely prevent someone from reaching their genetic potential. Unfortunately, an extremely good environment does not seem to reliably eliminate a genetic ceiling (for want of a better term) to a similar extent.

<comment><USER8> An extremely good environment takes away any sort of existential risk in your endeavour of life, therefore reducing one's ability to ever take any sorts of risks.

In simple words, being born in a lab environment makes one adapted to that lab environment.

</comment></comment></comment></comment></comment><comment><USER9> Off-topic, but for anyone who didn't know, Bardeen really was one of the great theoretical physicists of the 20th century: [https://en.wikipedia.org/wiki/John_Bardeen](https://en.wikipedia.org/wiki/John_Bardeen) . Which makes this all the more remarkable: [http://www.physics.umd.edu/courses/Phys798S/anlage/Phys798SA...](http://www.physics.umd.edu/courses/Phys798S/anlage/Phys798SAnlageSpring06/Josephson%20Physics%20Today%205153075.pdf) .

</comment><comment><AUTHOR> Saw the movie Blackkklansman yesterday and in the new member induction scene, he is mentioned as a Klansman and a friend of the grand wizard. I got curious and did a lil research and sure enough...

</comment><comment><USER10> The thing that bothers me about Shockley is how he was actually not that involved in developing the transistor. Bardeen and Brattain got it working without much input from Shockley. But once they got it working, Shockley swooped in and tried to take most of the credit.

That's not how to live life.

</comment><comment><USER11> A lot of this material is also covered in _The Idea Factory_ , Jon Gertner's great book about Bell Labs.

</comment><comment><USER12> For some reason, I'm getting a 404.

[https://web.archive.org/web/20180905024656/https://rupensavo...](https://web.archive.org/web/20180905024656/https://rupensavoulian.com/2013/04/05/broken- genius-the-case-of-william-shockley/)</comment>


Stop Thinking Organizational Structure Will Solve All Your Problems - <AUTHOR> http://ralfw.de/2016/03/its-the-goals-stupid/ <comment><USER2> Interesting...</comment>


Bliss (image) - <AUTHOR> https://en.wikipedia.org/wiki/Bliss_(image)

<comment><USER2> Is a link to the windows xp default desktop of a rolling hill & likely artist inspiration.</comment>


How Google sets goals: OKRs - <AUTHOR> http://www.gv.com/lib/how-google-sets-goals-objectives-and-key-results-okrs <comment> c-rack OKRs are great. If you want to use OKRs, signing up on [http://signup.okrs.io/](http://signup.okrs.io/) might be a good idea.

Disclaimer: I am working on okrs.io, currently.</comment>


Need HN: Any San Fran hackers need a roomy? - <AUTHOR>

Getting ready to move to San Francisco. Anyone interested finding a place and creating a "hacker house"? <comment><USER2> I'm not a hacker (I do know some code though), but I just started doing biz dev for a YC startup and will be looking to move to SF within the next couple months or so. Find my email in my profile and shoot me a message!</comment>


Fintech Firms Get Chance to Apply for Banking License - <AUTHOR> https://www.wsj.com/articles/fintech-firms-get-chance-to-apply-for-banking-license-1489599873?mod=e2tw <comment><USER2> Does this mean that fintech firms get taxpayer bailouts?

</comment><comment><USER3> So they might actually have a level playing field now?

<comment><USER4> Not exactly. The proposed charters are limited compared to traditional bank charters, but it's a start.

<comment><USER5> So what will the Fintech firms gain from having a Banking license under this charter?

<comment><USER6> New fintechs who want to move money have to obtain a Money Service Business license in every state where they operate. In almost all cases, that means they have to get 48ish MSB licenses (a state or two don't require them). Each one can be both complicated and expensive to obtain and maintain.

This new license would let those fintechs instead obtain a single license (and set of rules, and costs, etc) for nationwide use.

</comment></comment></comment></comment><comment><USER7> This is still early in the process. It will take some time before we know what the final regulation is. The comment period is between now and January of next year.

</comment><comment><USER8> Interesting, now they get to constantly get scrutinized by the OCC, which, by the way, is no fun. Those audits can end up being brutal.

</comment><comment><USER9> Interesting opportunity for a bunch of them.

</comment><comment><USER10> Have a non-paywall source or copypasta?

<comment><USER11> [http://www.facebook.com/l.php?u=https://www.wsj.com/articles...](http://www.facebook.com/l.php?u=https://www.wsj.com/articles/fintech- firms-get-chance-to-apply-for-banking-license-1489599873?mod=e2tw)</comment></comment>


Uncaptcha: Defeating Google's audio reCaptcha with 85% accuracy - <AUTHOR> https://github.com/ecthros/uncaptcha <comment><USER2> > From there, each number audio bit is uploaded to 6 different free, online > audio transcription services (IBM, Google Cloud, Google Speech Recognition, > Sphinx, Wit-AI, Bing Speech Recognition)

Gotta love that they're using Google's cloud resources to defeat Google's reCaptcha

<comment><USER3> Didn't people use google reverse image search to defeat the image-based recaptcha a while back?

</comment></comment><comment><USER4> There are some confounding factors (from my reading) - a.) the more likely Google thinks you are a human, the easier the tests/less tests you have to pass - are they repeating the test on the same computer all the time or are they mixing it up some how b.) the README states Google uses some of your interaction speed times/history with internet/Google to determine level of difficulty of captcha - if it's the same computer - and they repeat the tests on the same computer, are they just getting easier tests the better it gets? (My experience with visual captchas is I would be happy to get 85% success rate with those). They probably are not seeding the computer with 9 days of simulated activity to get the easiest captcha suite to make it seem human or are they? c.) since there could be multiple tests if Google thinks you are a computer, are they counting successful attempts in an ultimately unsuccessfuly run - is passing 5 then failing the 6th count as 5 success /1 failure or 1 failure?

</comment><comment><USER5> I'd love for this to be a browser plugin. I don't stay logged into google or let them set cookies, and some days I think that instead of doing machine learning in house, they just farm it out to people like me by holdings random sites hostage.

<comment><USER6> Actually you're helping to classify their datasets for ML purposes, so in a sense you're tilling their farm for them (while also helping ensure the site you're on has less spam than it would otherwise).

<comment><USER7> I usually make small mistakes identifying things, intentionally.

<comment><USER8> What's the point? If it's to "stick it to the man!", it won't have any effect because they are probably using multiple ratings from different users to classify things (in average people will be right).

<comment><USER9> And it will now be, on average, a hair less right, or less confident, or maybe just less confident in sildur's classifications.

Either way I think being a bit adversarial towards this kind of stuff is a healthy, albeit small, protest.

</comment></comment></comment></comment></comment><comment><USER10> Suppose that audio captchas could be solved with 100% accuracy. Then what? How do we make sure that visually impaired users can still access the internet / solve captchas, but robots cannot?

<comment><USER11> I suspect at some point in the near future it will generally be impossible to differentiate between humans and computers on the internet. At that point I wonder if we'll be required to use registered identifying information to use certain sites and apps.

<comment><USER12> This problem has been on my mind for some time now.

Requiring some kind of unique identifier seems like a difficult thing to sell to users, but otherwise we may be at the mercy of bots.

And it's not the scammers and normal spammers using bots that worries me. It's the potential to shepherd public opinion on social networks.

<comment><USER13> > It's the potential to shepherd public opinion on social networks.

And the unique identifiers need to be made public, and publicly verifiable - otherwise you've made it harder for, say, russian politibots - but easier for, say, facebook, who could post all the fake comments they want, and people will be more likely to trust that each commenter is unique and was vetted.

So basically, abolish private communication. I'm not mocking you, in case it sounds like that - I see the same dilemma, and don't know the answer. Personally, I prefer just making sure that the big communication sites are as vigilant as they can be and make it prohibitively annoying to engage in spam- like activities. It's not perfect, but it's the least awful choice (IMO).

<comment><USER12> > So basically, abolish private communication.

Not _private_ communication. You and I should be able to talk privately all we want. Just _anonymous_ communication.

And really our ability to communicate privately would be pointless if we weren't sure about one another's identity.

But you wouldn't event have to abolish anonymity online entirely... Just on major social influences.

The other option, imo, is that people become more skeptical and approach all content as though it could be politibots and/or advertisement. Then the notion of using the internet to "sample the crowd" is lost entirely because the crowd is tainted by bots and skepticism.

This seems difficult because people seem to have built in systems for relying on peer samples to establish norms.

</comment></comment></comment></comment></comment><comment><USER14> Also out this week in Science is George et al. "A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs": [https://www.vicarious.com/2017/10/26/common-sense-cortex- and...](https://www.vicarious.com/2017/10/26/common-sense-cortex-and-captcha/)

</comment><comment><USER15> can captcha go away now? its such a lazy garbage solution to the problem

<comment><USER16> What's better?

Hashcash is interesting, but it's probably not applicable to every scenario and it hasn't seen much adoption. There might even be a bit of a stigma now around proof-of-work algorithms because of cryptocurrency mining.

<comment><USER17> Hashcash. If you want to add money revenue (like reCaptcha does by making me work without payment) use something like Monero mining (Coinhive for example).

I'll take the PoW captcha any day.

</comment></comment></comment><comment><USER18> Nice work. It also publishes a file mfcc.py which uses Mel spectrogram to solve the audio offline. With enough data, a model based on MFCC should work much better than any cloud service (general speech recognizer).

<comment><USER18> Another interesting fact is that TensorFlow 1.4 supports native MFCC spectrogram tensors.

</comment></comment><comment><USER19> This is worthy of a bug bounty, and I feel it should be handled as such: reported privately to them first. The people trying to hack reCaptcha at scale are not good people.

The temporary fix Google might have to do in the mean time however is to stop all audio reCaptchas, blocking the people who vitally depend on it.

<comment><USER18> Blocking audio means blocking visually impaired people from accessing websites with reCAPTCHA.

Hacking reCAPTCHA is not only for bad people. There are several use cases where solving reCAPTCHA automatically is needed.

<comment><USER20> What are these legitimate use cases for automatic reCAPTCHA solving?

<comment><USER18> Automating searches on a government website that decided to use reCAPTCHA just because it wants to look modern. There are dozens of them in Brazil for example.</comment></comment></comment></comment>


End-to-end API lifecycle – Node.js style - <AUTHOR> https://gigaom.com/2014/12/16/strongloop-arc-aims-to-manage-the-whole-node-js-api-lifecycle/

<comment><USER2> The link is surprisingly light on details and doesn't even link to the actual release announcement: [http://strongloop.com/strongblog/arc-node-js-api- announcemen...](http://strongloop.com/strongblog/arc-node-js-api- announcement/)

</comment><comment><USER3> I like the concept</comment>


Show HN: 140verses - <AUTHOR> http://www.140verses.com

<comment><USER2> Fantastic! I originally thought you meant visual art but was pleased to see a surprisingly fluid poem (though it didnt make much sense).

I experimented with twitter as a random art generator a while ago, but its been down since they changed their API requirements and I haven't had the motivation to fix it yet. An old snapshot can be seen here: <http://binarymax.com/brownian_2.gif>

There really is a lot of great stuff you can do with the entropy that comes from twitter.

</comment><comment><USER3> One of the devs website is being flagged as distributing malware by Chrome on About page. You should remove that reference from the site until it gets resolved.

Cool idea btw.

<comment><AUTHOR> Thanks, fixing that!

</comment></comment><comment><USER4> Along the same lines: Pentametron: <http://pentametron.com/> It actually goes a little further by writing sonnets.

<comment><USER5> These two sites just made my day.

</comment></comment><comment><USER6> Cool idea btw "love", "coffee" example doesn't work

<comment><USER7> Worked for me, though I had to actually enter those words in the boxes...

</comment></comment><comment><USER7> Brilliant idea, though I'm getting a lot of "Sorry, we couldn't mash peoms for you, Perhaps you can try another time or two!"

<comment><USER7> Oh yeah, and that Error says Peoms rather than Poems...

<comment><AUTHOR> Gotcha!</comment></comment></comment>


Dispatch – the curated weekly newsletter for the Paramedic profession - <AUTHOR> https://dispatchnews.substack.com/ <comment><USER2> This could be an interesting HN submission, but not in this form. Here are some tips:

(1) Wait until you have an article that goes in-depth into some topic in this space, rather than just an intro, then submit that. People need something to sink their teeh into.

(2) Add a comment to the thread giving the backstory of how you came to work on this, and explaining what's different about it. That tends to seed discussion in a good direction.

(3) Have a username that represents you personally, rather than the project as an abstraction. On HN it's an antipattern to have your username be that of your company or project. It creates a feeling of using the site for promotion and of not really participating as a person. You don't have to use your real name, of course, just one to communicate that you're there as a person rather than as a brand. If you'd like to change your username, we can do that for you.

If you do those things, email us at email@example.com when your article is up and we can make sure it doesn't get flagged.</comment>


Rebuilding a 1981 Accord - <AUTHOR> http://imgur.com/a/uBCY8 <comment><USER2> Nicely done. I love the step by step photos of the project.

<comment><AUTHOR> It's not my work, but I was highly impressed with the build and the computer interface to the ignition timing.</comment></comment>