Ask HN: Does anyone still use IRC? - <AUTHOR> If so could you recommend any good Server/Channels. <comment><USER2> Whatever OSS projects you use have channels on freenode. Keep those open all day and lurk, you'll be amazed how much you learn from osmosis. If someone comes by with an easy question, answer it and you'll have social karma to get help from others when you need it.

<comment><USER3> You mean you actually keep a chat window open all day long? Isn't that ridiculously distracting?

I check my email twice a day, and HN the same. Anything else that could potentially distract me from what I'm doing gets turned off. I couldn't imagine giving random strangers the ability to intrude into my train of thought whenever they wanted.

<comment><USER4> I use an IRC bouncer to stay logged in to the IRC servers. I connect to the bouncer, which replays anything I missed on the IRC server while I'm not logged in to the bouncer.

<comment><USER5> Can you share the name of the bouncer? Sounds like nice functionality...

<comment><USER4> I use znc running on a slice. (<http://en.znc.in/wiki/ZNC>)

I've also heard of Bip (<http://bip.t1r.net/>), but I've never used it.

<comment><USER6> What exactly is "running on a slice"?

<comment><USER7> Slice = virtual server at <http://www.slicehost.com/>

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> There's about 190* of us HN'ers in the #startups channel right now. <http://ircstartups.pbworks.com/>

*Maybe we're not all HN'ers, but darn close.

<comment><USER9> how many are active? IRC is usually full of many decent sized rooms with limited chat and mostly inactive users

<comment><USER10> This is a _feature_. Most of the people on an IRC channel are lurkers. And the channels are very quiet so that people can hear you when you do speak. Believe me, I wish they were _quieter_ , because even as they are they tend to be too distracting for me to lurk there while working.

It does seem creepy. Our company's salespeople laugh at the programmers because we really love to gather in one physical room, then sit silently next to each other typing on laptops and communicating largely via Jabber chat and IRC, _even though we are two feet away from each other_. If you've never tried to concentrate on programming this seems like alien behavior, but it actually makes a lot of sense, for the same reason that it makes sense for all the Drupal devs to sit in one IRC channel but almost never speak.

If you crave more chat, join more channels in parallel.

<comment><USER11> The ability to send code snippets and URLs alone makes chatting this way worth it.

<comment><USER12> And the ability to store and refer to it later.

</comment></comment></comment></comment></comment><comment><USER13> Yep. Freenode IRC. I lurk on about 25 channels, ranging from the local Linux User's Group channel, to the #startups channel (which draws a lot of HN'ers) to the channels for several programming languages I use, etc.

IRC is definitely alive and well, especially on Freenode.

</comment><comment><USER4> IRC is an indispensable part of my work. The team I'm on uses a private server for team discussions (works great when someone is working remotely), and we also are in a few public channels on freenode. On freenode, I'm in #openstack to talk about cloud object storage infrastructure, #eventlet to learn more about it, and #cloudfiles for Rackspace product-specific questions. #openstack has been pretty active in the last couple of weeks (since we released the code) and has been pretty good for helping people who are learning about the systems. Internally, I can't imagine not using IRC. If I choose to work remotely, or if I'm working odd hours, it becomes a very important tool to keep up with what the other devs are doing.

To stay in the channel when I'm not online, I use an IRC bouncer (znc running on a slice, for me). When I log back in on my computer, I have the conversation I "missed" played back for me. I know other people that use irssi in a screen session to accomplish something similar.

<comment><USER14> we do the same thing. We have a few remote developers, so we set up our own private server.

I use XChat on Ubuntu, and it is perfect for my limited usage.

</comment></comment><comment><USER15> We use Freenode, #grooveshark is an open channel for random techie users who want to talk directly to developers (or each other), and people working on 3rd party extensions/add ons to Grooveshark.

We have other internal invite-only channels for coordinating with external developers on internal projects. We also have github spamming a channel now whenever changes to a repo that a lot of people are coordinating on get pushed.

The #redis channel saved our asses at least once when Pieter Noordhuis wrote a script to help us with a particular issue we were having. :) And the #facebook channel has been a useful resource as well.

I was wondering if anyone here uses IRCcat: <http://www.metabrew.com/article/how-we-use-irc-at-lastfm> I know flickr also uses it and it seems like it would be insanely useful to have notifications about all kinds of things going into various channels. I know flickr also uses it: <http://velocityconference.blip.tv/file/2284377/>

Haven't spent much time working on it but so far the bot gets kicked for flooding immediately. May only be a problem because we're using freenode for testing. Long term want to run our own IRC server, but for testing it was easier to just use freenode...

</comment><comment><USER16> imho great deal of IRC's greatness comes from irssi, an awesome client for it. It has a learning curve though, especially if you are not familiar with screen (or tmux). While you can run it locally, it really shines when you run it in a server inside a screen session, connected via ssh.

Each phone I have had in the past 5 years has had a ssh client available, so I could just attach to my session and have the exactly same client available wherever I go.

I use irc mainly to keep in touch with old friends. I'm also running a small non-profit, and we use irc extensively in internal communications. And occasionally I go to a projects channel on freenode if I happen to have a question or something, the people tend to be quite useful (as long as you ask smart questions and be polite). Lurked some time at ##c++, from which I feel that I learned a lot from.

<comment><USER17> which IRC server do you guys use ?

I have long wanted to set up an private IRC server for our workplace, but got lost in the zillions of complicated IRC server setups.

Do you use it over VPN .... or have you set it up to use some kind of authentication ? It would be great if you could help.

<comment><USER18> Jabber servers works really well for workplaces... they have chat rooms and private communications.

<comment><USER19> So does I.R.C.

</comment></comment></comment></comment><comment><USER20> We (developers and sysadmins) use it at the big corporation where I work. Keeping irssi open in screen (GNU screen) means I can connect remotely from anywhere with a single session, and it doesn't appear when I'm focused on something else in a different screen session. Each project gets its own channel. It's way better for collaboration than having to match schedules for each little question.

It's also very helpful for asking specific questions about OSS projects, as mentioned. The people actually developing projects hang out in IRC, not at StackOverflow.

</comment><comment><USER21> I hang out on a private IRC server with about 15 folks, mostly old friends of mine working in various parts of the industry. It's primarily a place to talk about geeky topics and complain about work.

<comment><USER22> Same here. We've had a private channel running for over a decade now and as people move to new jobs and shuffle around, the channel is always home.

We've even built a bot to capture URLs and tweet them to a private handle so people can check out sites after work or while away from the channel.

To the person asking if IRC is distracting, it's a LOT less distracting than the hipster that sits next to me and is constantly checking his smartphone every time an SMS comes in. Pavlov would roll over in his grave.

</comment></comment><comment><USER23> I telecommute, and most of my day-to-day conversations go over IRC. My IRC client has ~50 open windows at the moment, but less than half of them are actual Open Source channels, many of them are direct queries with the user. And thanks to screen, I never log off and don't miss discussions, someone trying to get in touch, commit and release messages to -dev channels, and so on.

If it suits one's style, it's an easy and nice way to stay linked to one or more communities.

</comment><comment><USER24> irc is still the only way to meet "the guy on the street" online. that's mainly why it still survives, despite its quirks.

i don't know what you mean by "good" server/channels, but let me say this:

back when irc was _the_ chatting medium, all the low-iq trolls that today are on the web were on irc. so the channels were ruled by a very strict dictatorship. you needed to be very lucky (or meet with real-world friends) to have a decent chat.

now that the trolls and other offenders largely moved out to the http space, i think irc has gotten its "underground" status back. this means you'll most probably find whatever you're after inside large networks. for example, if you want to know a quick fact about norway, go to #norway in undernet and shoot. somebody will reply. you need a python tip, #python in freenode is where some python gurus hang out.

i personally admire freenode for having managed to establish the "showing op status is not polite" netiquette. you can see channels with hundreds of users where not one op has shown up for ages. that was not easy to imagine not so long ago :)

</comment><comment><USER25> People with tiling window managers seem to IRC a lot.

<comment><USER26> A recommendation on a good console-based IRC client that doesn't use the alt- key for any shortcuts (interferes with my XMonad setup)?

<comment><USER18> Just curious, why not have the windows key be your primary meta for XMonad? In general I try and use that key for my WM shortcuts since most apps don't use it.

<comment><USER26> Because it's slightly less convenient for my thumb to reach, and just feels unnatural: it causes an ever-so-slight strain as I try to keep my fingers on the keyboard's home row. Yes, I can be pretty finicky sometimes. I know I'd probably get used to it if I tried, but it's rare for me to need alt for applications.

</comment></comment></comment></comment><comment><USER27> Answering the question directly, enough people still use IRC that axod has built a sustainable business from a web-based IRC client (mibbit.com).

<comment><USER28> That seems like a pretty indirect answer to me. ;)

</comment></comment><comment><USER29> At the company where I work (a web startup with 100 and some odd employees, half of which are the engineers) EVERYONE uses IRC. The engineers, yes, all the teams have their own channels, general channels, etc.

But so do the support staff, the business folks, marketing, etc. Its amazing.

<comment><USER30> Where do you work? That's awesome.

I've always thought IRC would be a great platform for internal communication.

How many times do we open up our typically single person-to-person IM clients to figure out where to eat lunch?

How distracting is your IM client compared to IRC?

I suspect some of the younger companies with money to burn use Campfire to fill these kinds of needs.

</comment></comment><comment><USER31> Of course. Mostly for private communication with friends and with BitlBee gateway. But it seems that many OSS project have freenode channels, that are very active, and not only that. There's very active #haskell channel on freenode, #xmonad, there are various fedora channels, there's an unofficial channel for students of the faculty where I study now, even channels like #lgbtreddit.

I don't see any decline of IRC use in any foreseeable future.

</comment><comment><USER32> Presently I'm in 12 channels on 7 servers. IRC fills a need for "semi- synchronous" communication in group form for me. part crowdsourcing, part water cooler gossip, it's entertaining and educational. It's real-time enough to satisfy the feeling that, as a teleworker, someone else is there, but not so real-time that I must devote my full attention (like a phone or video chat).

The big benefit to IRC for me is its ease of setup (apt-get install ircd- hybrid), end-to-end ownership of the communication protocol and infrastructure (ability to ensure privacy and anonymity, if necessary or expected), ease of extension and long-term stability. Or, put another way, "my server, my client, our channel, my friends, our bots." good times.

Disclaimer: I'm one of the (former?) long-time ircd-hybrid developers. I started a PhD thesis on education through IRC studying its social dynamics, but am considering dropping the effort.

</comment><comment><USER33> I dip my toes into IRC now and again, but keep getting frustrated by the learning curve. Yes, I get the basics of joining channels and chatting. I can see that there's a ton of efficiencies IRC power users have figured out that I haven't.

Any good resources out there for climbing the learning curve to journeyman IRCer?

</comment><comment><USER34> I have run my own irc server and been connected for over 10 years essentially in one way or another.

Most of my online friends gather there and idle / jump into chat when it's going on.

It's been one of the most pleasant constants in my digital life. An unmatched technology for keeping in touch with geographically challenged friends.

</comment><comment><USER35> irc.mozilla.org, it's a mainstay for most discussions and coordination for all our projects (IT, webdev, firefox, etc).

#webdev, #firefox, #amo (addons.mozilla.org), #labs

Over half our employees and all our volunteers are remote, so IRC is how we stay in touch for the most part.

</comment><comment><USER36> #kernel-panic on freenode for good Linux and tons of other good discussion. Been there for years. Active right now with 34 people. But pretty much every major FOSS project has an IRC channel, usually on freenode. Every programming language, framework, etc. is represented there. I get tons of good help there and help others as well. IRC is definitely very much alive. I rue the advent of modern one-on-one IM as it really ruins teamwork. A few months ago we threw up our own openfire jabber server and required everyone in the company to be present and it has really boosted awareness of what is going on and replaced the "hallway conversations" we were missing from our distributed development environment.

</comment><comment><USER37> There's a whole bunch of us active on #hackers-india (irc.oftc.net). Drop in sometime :)

</comment><comment><USER38> Yup. My company uses it amongst the devs to stay in touch. I also lurk on #nodejs.

</comment><comment><USER39> IRC is one of the primary communication tools for Wikipedia and its sister projects. The community is so distributed, coordinating meetings using UTC time becomes second nature.

There are dozens and dozens of channels. This page attempts to track just some of them.

<http://meta.wikimedia.org/wiki/IRC/Channels>

And even the non-technical staffers at the Wikimedia Foundation hold "office hours" on IRC from time to time.

<http://meta.wikimedia.org/wiki/IRC_office_hours>

</comment><comment> mike-cardwell I use bitlbee to gate all of my IM networks to a single IRC channel which I then point X-Chat at. I also ocassionaly jump onto freenode to get real time support for various open source packages.

<comment><USER40> Another user of Bitlbee here. It goes hand in hand with using irssi and a bouncer to maintain an always-on presence that I can take with me anywhere I go.

</comment></comment><comment><USER41> Hell yes. If you don't use IRC you're missing out.

I hang out wherever I'm learning stuff (#clojure atm), and keep logs of many channels so I can search them later.

</comment><comment><USER42> If you're looking for a good tech community / channel to hang out in, I can recommend two. These channels are best suited for programmer / coder types who enjoy hanging out and chatting about tech stuff.

irc.darkscience.org/#darkscience irc.oftc.net/#neverfear

And their respective websites:

<http://darkscience.org/> <http://neverfear.org/>

</comment><comment><USER43> We've had a lot of great support requests and chats with customers on IRC (Freenode, #newrelic).

Like others have mentioned, it's great to have a few rooms open for projects/services you care about, where you can check in every so often and read the backlog. While some rooms are archived, I've never read them that way, but keeping IRC idling in the background can be a great way to keep tabs on things.

</comment><comment><USER44> I hang out on 3 networks. I am on Freenode for a few different projects I support and associated chat channels that I've been at for years. I am on StarLink, a small network, for the two channels associated with Alpha & Omega Ministries. I am on EFNet for one specific channel of a website.

I also use a local bitlebee server for instant messaging.

All in irssi with screen on a local server in my house.

</comment><comment><USER45> Yeah, but I try to limit it to 10 channels for as to not get overwhelmed (I have a habit of wanting to check every time something new is said, plus I need room for bitlbee sessions).

I mostly idle on local user groups and language/framework-specific channels. I rarely say anything, but just keeping an eye on the conversations has taught me a lot of stuff I wouldn't have known otherwise.

</comment><comment><USER46> Another often overlooked use for IRC is as a side channel during events. In conferences, we'll all be in a room, on IRC commenting on talks. During the BP oil spill, a channel (#theoildrum) had 500+ people, including some with oilfield experience, discussing the live ROV feeds, as well as scanning to find lots of ROV feeds that were not visible to the general public.

</comment><comment><USER47> #startups on Freenode. I use IRC for collaborating with my colleagues (we're a distributed company). EFNet.

shameless plug:

Check out jIRCii. <http://jircii.dashnine.org>. It runs on OS X, Windows, and Linux. It's (in my opinion) the perfect blend of mIRC and BitchX. It's scriptable too with over 70 scripts contributed.

</comment><comment><USER48> Freenode, Notably #Startups but also the different subject areas (JavaScript, CSS, HTML, JavaScript, Jquery etc)

</comment><comment><USER49> I was pretty active in the suse project with bug testing and whatnot for a while, so that got me on freenode. I lurk about 15 channels there.

I'm on two private networks with about 5 channels between them and I lurk on Undernet out of nostalgia. I'll hit up Efnet if I'm looking for security related topics.

</comment><comment><USER50> I use IRC every day. I use it to work with others on opensource projects and many of my past interns and I use it to stay in touch daily, creating a very helpful network. My current interns can get help from not only ne, but people who used to work for me. It's a great networking tool.

</comment><comment><USER51> I'm in #slicehost on Freenode most of the day. If you want smart people + plenty of humor, come on over.

</comment><comment><USER52> #startups on freenode and a small private channel with 10 or so friends on IRCNet. LimeChat powered!

</comment><comment><USER53> At my previous company we had an IRC room for the devs (and whoever else wanted to join) which had a bot that reported on updates to certain feeds within the company, it was pretty useful and it's not distracting at all. I used irssi+screen, the bot was a Supybot.

</comment><comment><USER54> At my previous workplace, a web devshop, we had a company channel where everyone hanged, even the CEO (when he could). Plus project-channels where we could discuss details of each project, and where we invited clients and partners that used IRC.

</comment><comment><USER55> I just got on the #reddit channel @ irc.freenode.net the other day and learnt quite a bit (about profits in the medical marijuana industry) by just hanging out there. I actually got on IRC to see if there really was anyone who still used IRC.

</comment><comment><USER56> IRC is one of the best ways to communicate. There's a lot to learn, and a lot to be taught.

Sometimes the best experience can be had by just reading what other people write.

For example... Erica Sadun is in #iphonedev and #ipaddev on Freenode. Know who she is? :)

</comment><comment><USER57> I recommend running <http://www.bitlbee.org/> and using irssi not only for IRC but for IM as well. For IRC I get a lot out of #pocoo on Freenode and use a private company ircd.

</comment><comment><USER58> Yes, you can find me on #gllug (Greater London Linux User Group), #amahi (Home Server), #greyhole (folder duplication a la WHS but using Samba) all on irc.freenode.net

And of course irc.quakenet.org #|HSO| and #fidelitas :)

</comment><comment><USER59> I spend most of my time idling in #riak on irc.freenode.net

I use an IRC bouncer too.

</comment><comment><USER60> I use IRC all day long. Every day.

Freenode. If you're into perl, they have an IRC server of their own.

I have been working remotely for the past year or so and we use it to keep the team in touch with one another.

<comment><USER61> IRC network actually - irc.perl.org is 6 servers, four of them on Shadowcat's hardware, the others donated by individual members of the perl community.

<comment><USER62> Yet #perl6 is on freenode.

<comment><USER63> I assume there is also other perl activity on freenode because it comes high on the IRC stats at <http://langpop.com>

</comment></comment></comment></comment><comment><USER64> Every day at work. I can't recommend any channels to you though.

</comment><comment><USER65> Freenode is excellent. As far as channels, these are the ones that I particularly like:

      ##programming
      ##compsci (a bit quiet of a channel)
      #emacs
      #haskell

</comment><comment><USER66> IRCnet used to be fun to spend the teenage days. (late nineties and the beginning of century, hehe)

EFNet has tremendous communities devoted to music

Freenode is go-to for open-source software chatrooms.

</comment><comment><USER67> I lurk on Freenode on various project and language channels.

</comment><comment><USER68> Our company uses #disqus on freenode (private) for development and general chatter.

I'd like to change the channel for internal chat and use #disqus for public use though.

</comment><comment><USER69> irc is very helpful when interacting with people who don't speak English as their first language, or have a very thick, difficult-to-understand accent.

</comment><comment><USER70> Yes.

#rubymn (Minnesota ftw)

#railsbridge (if you're interested in ruby on rails; the <http://railsbridge.com> community)

</comment><comment><USER71> I am currently idling on:

freenode/gearman

freenode/memcached

freenode/drizzle

freenode/mongodb

freenode/mysql

freenode/phpc

freenode/hiphop-php

freenode/facebook

freenode/twitterapi

freenode/jquery

YMMV.

</comment><comment><USER72> I am always idle on IRC, still love it

</comment><comment><USER73> I run irssi on screen all the time

</comment><comment><USER74> Look at all these comments! Clearly the answer is "nope, nobody uses IRC anymore" ;)

</comment><comment><USER75> Totally. I help out on the #ubuntu channel, and get help on the #python channel.

I avoid #politics

</comment><comment><USER76> I open Colloquy to get help or to see what the cool guys are talking about.

</comment><comment><USER77> Of course.

</comment><comment><USER78> Yes, for instance, a lot of haskellers are on #haskell on freenode.

</comment><comment><USER79> Freenode baby Freenode

</comment><comment><USER80> On Freenode: #django, #startups, #python, etc.

</comment><comment><USER81> The habit was fading. Thanks for reminding.

</comment><comment><USER82> Not on iPhone they aren't :-(

<http://news.ycombinator.com/item?id=1561306>

<comment><USER83> Wrong link?

<comment><USER82> yes. I meant to post the one about the Rooms IRC client for iPhone getting approved for the AppStore then getting pulled minutes later. <http://news.ycombinator.com/item?id=1560753>

Ironic, and possible relevant.

Personally I do not use IRC. My work blocks it unless I want to court a reprimand by finding those few servers with non-standard port numbers.

At home, I'm too busy with wife and kids, and whatnot to spend any real time chatting. Most of my online activities are batched to only occur once or twice a day which pretty much precludes irc.

<comment><USER84> Uh, there's still Colloquy (<http://colloquy.mobi>), a port of the most- popular Mac client to iPhone. So it's not like Apple has banned IRC in general.

</comment></comment></comment></comment><comment><USER85> Does World of Warcraft count?:)

</comment><comment><USER86> Hell yeah. All day every day.

</comment><comment><USER87> freenode for channels about opensource projects undernet for goofing off

</comment><comment><USER88> #webdev on EFnet

</comment><comment><USER89> on freenode: #android-dev #webos-internals

</comment><comment><USER90> STILL use IRC? IRC is in heavier use today than ever before! Not only has the amount of networks (and servers per network) exploded the past 10 years, but so has also the global amount of users per network and IRC as a whole.

</comment><comment><USER91> I pretty much live on freenode in all the dev channels, but I keep the terminal on a fairly isolated place so that I don't get distracted. Keep a shell on my linode, /lastlog to check to see if anybody was lookin' for me.

</comment><comment><USER92> freenode #jquery is pretty much the place to be</comment>


How left wing is Facebook? - <AUTHOR> http://www.powerlineblog.com/archives/2016/09/how-left-wing-is-facebook.php <comment><USER2> At this point it's pretty obvious that this process is mostly run by people in terms of reviews. These people are probably under the age of 25, which means they are more likely to have a liberal viewpoint. All of that said, FB might be well served to hire a few people at/near senior citizen age to also do reviews, and have 3 reviews per post... Same for appeals.

This should lessen the influence of any one viewpoint. That said, I see plenty of promoted posts (mostly libertarian) in my feed, so I'm somewhat surprised that conservative posts are being suppressed in terms of promotion.</comment>


Satire, sewers and statesmen: why James Gillray was king of the cartoon - <AUTHOR> http://www.theguardian.com/artanddesign/2015/mar/21/satire-sewers-and-statesmen-james-gillray-king-of-cartoon

<comment> chestnut-tree We tend to think that as history progresses, more and more taboos are broken and we're freer today to publish or lampoon ideas, people and events than in the past. One thing that's surprised me about satirical images from the 18th century is how bold they sometimes are.

For example, here's an anonymous cartoon from 1740 mocking Thomas Walpole - <USER2> prominent British politician and banker.

[http://images.library.yale.edu/walpoleimages/dl/000000/lwlpr...](http://images.library.yale.edu/walpoleimages/dl/000000/lwlpr00981/lwlpr00981.jpg)

Here is a quote from a BBC documentary explaining the image: _" There was a new force at work in Georgian society: satire. One of the Georgian age's most notorious images is Walpole's huge naked bottom blocking the way into the Treasury. To get on in 18th century government, this is what you had to kiss"_ [1]

[1] [https://youtu.be/-EDmxfSydCw?t=50m18s](https://youtu.be/-EDmxfSydCw?t=50m18s)</comment>


BlackBerry to exit hardware business - <AUTHOR> http://www.cbc.ca/news/business/blackberry-hardware-loss-1.3781876 <comment><USER2> Comments moved to [https://news.ycombinator.com/item?id=12597109](https://news.ycombinator.com/item?id=12597109).

</comment><comment><USER3> The fact that this has no discussion after 2 hours just underscores how lost and forgotten Blackberry is.

It reminds me of the demise of Palm.

<comment><USER4> It's a duplicate. Bigger discussion over here: [https://news.ycombinator.com/item?id=12597109](https://news.ycombinator.com/item?id=12597109)

<comment><USER5> First comment to the bigger discussion was posted 2 hours after submission anyway.

<comment><USER3> I was hoping that this was a dupe and that the real discussion was happening elsewhere.

I'm not sure whether that's funny or just sad.

</comment></comment></comment></comment><comment><USER6> More lively discussion is here: [https://news.ycombinator.com/item?id=12597109](https://news.ycombinator.com/item?id=12597109)

<comment><USER3> Wait. John T Haller of PortableApps?

Damn dude I still remember browsing your original site years ago. I used Portable Firefox in grade school and it was a life saver.

<comment><USER6> Yes indeed! Still working on it though we've picked to be more cloud folder centric. Glad it helped you! The</comment></comment></comment>


Quickipedia – Instant Wikipedia summaries with the power of autocomplete - <AUTHOR> http://simplestepscode.com/quickipedia/

<comment><USER2> Nice job, I'm working on something similar, now focussing on the ranking, just a note: if I search 'picasso' I get a disambiguation page description, you should try my json-wikipedia [1], it adds also a field with the 'type' of the article, so it would be easy to remove the disambiguation/redirects ;)

[1] [https://github.com/diegoceccarelli/json- wikipedia](https://github.com/diegoceccarelli/json-wikipedia)</comment>


Ask HN: How? Safe harbor yet actively moderate, curate and censor content? - <AUTHOR> I can't really wrap my head around this.
IIRC Safe Harbor was made so ISPs and websites would not be legally liable for the content posted by users. Safe harbor guards these business from criminal liability.
If these ISPs and websites have nothing to worry about legally then why is there the need to moderate, curate or censor? It's not as if these platforms would ever lose enough customers that they would fail or go bankrupt.
Can someone explain this? <comment><USER2> I assume you are familiar with the chans (4chan, etc.)?

That's what no moderation looks like. What kind of ad supported business could you build if your content looked like that? How could you even build a mass audience?

> It's not as if these platforms would ever lose enough customers that they > would fail or go bankrupt.

That seems like a dubious proposition to me.</comment>


Application Monitoring Is The New Unit Testing - <AUTHOR> https://instrumentalapp.com/blog/post/91367016864/application-monitoring-unit-testing?utm_source=hacker%20news&utm_medium=social%20media&utm_content=application%20monitoring%20is%20the%20new%20unit%20testing&utm_campaign=application%20monitoring <comment><USER2> I think application monitoring might be coming up in the hype cycle, but it is not a great replacement for unit testing and selling it in that light is probably unwise.

Waiting 'til you ship to production to find a bug is a really, really, really bad idea no matter how fast you diagnose it. If your site is doing any serious scale, this could cost a year's worth of dev salary in the time it takes to make the fix.

Application monitoring is a useful tool, but the implications of overselling it will lead to some real problems for teams that aren't wise enough to see the forest for the trees.

<comment><USER3> The article title is misleading. Application monitoring is not a unit test replacement. You want both.

<comment><USER2> I agree.

</comment></comment></comment><comment><USER4> Says the company that is selling their application monitoring solution.</comment>


EU Patent Office Rejects 2 Patents with AI Listed as Inventor - <AUTHOR> https://www.techdirt.com/articles/20191221/00310443618/eu-patent-office-rejects-two-patent-applications-which-ai-was-designated-as-inventor.shtml <comment><USER2> Easy workaround -- let the AI come up with the idea and then put a human's name on it. Done and done.</comment>


The NYTimes.com loves GeoDjango - <AUTHOR> http://open.blogs.nytimes.com/2008/12/22/represent/#comments

<comment><USER2> I wonder how much effect ESRI's choice of Python as a scripting language for ArcGIS has had on the proliferation of Python based GIS tools.

</comment><comment><USER3> Is anyone out there using a Pylons stack for GIS? I'm starting a new geo- related project, and GeoDjango looks awesome, but I'm having trouble getting over the fact that Pylons is just more my style.</comment>


Ask HN: Is there a DNA/genealogy site you trust? - <AUTHOR> I'm really interested in 23 and me and the general building-a-family-tree-via-DNA service, but, like a lot of people, are concerned about my genetic info being later resold.<p>Is there a service HNers could recommend for the privacy conscious? <comment><USER2> No, stay away. Even if a company is trustworthy today, it doesn't mean that it won't update its Terms of Service tomorrow/be bought.

<comment><USER3> Really the only answer this thread needs. Companies can be bought and sold and the information they have can change hands and change retention and privacy policies. There's no trustable verification method for knowing, for absolute certain, that none of your data has been retained at your request. There simply is no reasonable way to trust these services.

<comment><USER4> Makes me wonder if there could be a business model, type (maybe nonprofit?), or license that user data would be destroyed in case of a change of ownership. Or maybe even a time-limit on data, like how messengers have self-destruct options.

</comment></comment></comment><comment><USER5> If you want to use it for family tree research you basically have to go to the companies that have the largest databases, because that's where you'll find familial matches (e.g. 23andme and AncestryDNA).

23andme has a better track record of refusing governmental requests than AncestryDNA.

In terms of genetic information sharing, none of the providers shares non- anonymized data. 23andme allows an opt-out for sharing anonymized data.

You can also do the tests pseudo-anonymously, while they ask you to provide your real name, etc. you don't actually have to do so. You can purchase a test kit at a retail store for cash and register online with a fake name if you wish to do so.

</comment><comment><USER6> I did 23andme earlier this year for the health information. After much deliberation about the privacy risks, I decided the tradeoff was worth it.

I looked at the family tree/genealogy data as well, and I was stunned how many of my close relatives were in there.

It does depends on exactly what your fears are about how the data will be used, but to some extent the ubiquity of these services means the privacy of your genetic information has already been violated regardless of wether you choose to use them yourself.

</comment><comment><USER7> My approach was to buy a temporary domain with a cash prepaid gift card, register it with a fake name from a VPN, set up email (also via VPN), do the DNA thing (via VPN) with a fake name and an email address at the new domain, download my data (VPN), then delete the account.

The only thing linking it to my existing identity is the actual DNA data itself, which is probably insufficient for insurance companies or data brokers to relate it to my existing records for discrimination purposes.

<comment><USER8> But if other members of your family submit their DNA with their real identities, then possibly you can then be identified like how they caught the Golden State Killer.

<comment><USER7> You're right. I should probably make a point not to go on a multidecade killing spree, given the circumstances.

<comment><USER8> You go to all the trouble to be anonymous and then have a flippant response to a real way your efforts could be undone. I mean, you shouldn't go on the killing spree regardless.

<comment><USER7> It's an attack that can't be mitigated; having controlled for all of the variables in my control, the only thing to do about those that are not is make jokes.

It also presumes they lied and didn't delete my data on request.

</comment></comment></comment></comment></comment><comment><USER9> I'm not sure. Supposedly 23andMe will delete your data completely upon your request. So you could sign up, print your reports, then request deletion.

<comment><USER10> Would they have shared your non-anonimized data with anyone by the time you request a delete? Perhaps through some unspecified 3rd party whose privacy policy you don't know?

<comment><USER9> This was a few years ago, so it could have changed. Supposedly they only shared 'anonymized' data for research purposes. You could opt-in (or maybe out) of genealogy connections. Of course, with enough attributes and cross references even anonymized data is seeming less and less safe.</comment></comment></comment>


Like Chrome but miss the Firefox AwesomeBar? Try Fauxbar - <AUTHOR> https://chrome.google.com/webstore/detail/hibkhcnpkakjniplpfblaoikiggkopka

<comment><AUTHOR> The Omnibox has some strange UI choices, and this little extension is about as good a workaround as you'll find right now until they implement a patch.</comment>


Every time I visit HN, I die a litlle bit inside. Should I go back to school? - <AUTHOR> Hi, I am a 30 year old (single) developer from India, had a chronic health condition due to which I spent half 20s running from hospital to hospital, just to stay alive, not cured but stable now. Working in bodyshop building CRUD web and android apps for banks for the past five years. Folks at my office see their work just as a means to a paycheck, no pride in their work. No design discussions, no adopting new frameworks, no hackathons, no functioal programming - nothing but bare minimum. Have to literlly fight to even have a discussion on writing automated tests. Every time I visit HN and see people do and discuss intricacy of architecture, programming practices, clean code, I die a litlle bit inside. How I wish I could work in a team like that.
I am worried to death (literally cannot sleep) that I have squandered my life away.
Have 2 options - take a big risk & go to US for higher education, study hard and get into a tech company.
OR continue at my current workplace, put in a ton of work learning the latest and greatest on MOOCs / Udacity /Pluralsight on my own, make a github profie and  and apply for jobs on 'Who's Hiring?' I am even willing to work for free for a couple of years if I can find a place in a team with good engineering practices doing something challenging, to become a better engineer.
So my question, Is it worth going back to school at 30, hoping for a better career ? Learn alogritms right this time, try to get into machine learning / AI ? <comment><USER2> There are literally no better companies in India you can work at? Your English skills seems pretty strong, and you have the right attitude. Maybe you should try to better your situation in India before taking a huge risk, especially with your medical issues.

</comment><comment><USER3> Just echoing what someone else already wrote: Your english is great and you have a great attitude, and you are ambitious and still young. Do not worry that you are wasting your life. It seems like you are passionate about coding, so you are in it for the long term. You should try to get a better job of course. If that means going out of the country to do it then take the chance while you're young. I don't think you need a masters degree to do that, although that may be what you want to do anyway. Best of luck to you.

</comment><comment><USER4> Could you become a solo freelancer and work from your current location?</comment>


Wanderland: A journey through Iran's wild west - <AUTHOR> https://www.1843magazine.com/journeys/wanderland-a-journey-through-irans-wild-west <comment><USER2> There are some incredible (in the literal sense of the word) geographic features in this part of the country.

I took my motorbike through this region a few years ago (riding Turkey / Iraq / Iran).

Copy paste mountains. Scarlet salt lakes. Stranded boats. Burnt plains. Ardent forests and mountains.

Some photos for the curious: [http://travel.ninjito.com/2014-08-18-Iran](http://travel.ninjito.com/2014-08-18-Iran)

[http://travel.ninjito.com/images/2011-06-18-RoadTrip/SLR/qx-...](http://travel.ninjito.com/images/2011-06-18-RoadTrip/SLR/qx- trip-40.jpg)

[http://travel.ninjito.com/images/2011-06-18-RoadTrip/SLR/qx-...](http://travel.ninjito.com/images/2011-06-18-RoadTrip/SLR/qx- trip-43.jpg)

<comment><USER3> I'm getting 404s on your links.

<comment><USER2> I think I (poorly) coded anti-hotlinking when I wrote my site.

Here are all the images in an imgur gallery: [https://imgur.com/a/aNdXGwu](https://imgur.com/a/aNdXGwu)

</comment></comment></comment><comment><USER4> In case the author sees this, a correction: the ziggurat is 105 meters on a side, not 105 square meters. This makes the ziggurat's base more accurately around 10 thousand square meters.

</comment><comment><USER5> If you consider coming to Iran, it is probably one of the cheapest countries to travel to, just remember change your money when you arrive not before. and don't come if you are form US of UK because of political issues

<comment><USER6> Have travelled there several times in the last years. People are lovely and so welcoming!

Last time (May '18) we travelled with an entire group (~30 people) all US citizens going there to learn more about the country their president was not so fond of.

That was just amazing.</comment></comment>


[UK] Channel 4 removes full-length programmes from YouTube - <AUTHOR> http://www.bbc.co.uk/newsbeat/25666997

<comment><USER2> Unfortunately their own player no longer works in modern linux distributions (Trick of installing HAL no longer works).

YouTube was the only way to watch 4od on linux.</comment>


Deep Learning Summit: Panel of Pioneers [video] - <AUTHOR> http://videos.re-work.co/events/32-deep-learning-summit-track-1-montreal-2017?utm_source=Newsletter+Signup&utm_campaign=12c311b81d-newsletter_oct_26&utm_medium=email&utm_term=0_547d24aafe-12c311b81d-522831557 <comment><AUTHOR> These being "Yoshua Bengio, Yann LeCun, and Geoffrey Hinton."</comment>


FDA approves first marijuana-derived drug and it may spark DEA rescheduling - <AUTHOR> https://arstechnica.com/science/2018/06/fda-approves-first-marijuana-derived-drug-and-it-may-spark-dea-rescheduling/ <comment><USER2> The current drug regulation regime strikes me as backwards--it seems that the more appropriate assumption should be to not regulate a substance, until it can be shown to be harmful (I'm not talking about label accuracy; I see that as more of a truth-in-labeling issue).

To me, cannabis illustrates how absurd the current drug regulation paradigm is. The idea that it would be classified as a Schedule I substance is ridiculous. But to get the FDA to change anything, you have to have people engaging in illegal behavior, to create a demand for specific legislative action about a specific substance, to drive a market for a derivative chemical, to petition to the FDA, to begrudgingly change the scheduling.

I've come to the conclusion that drug scheduling and the war on drugs is an economic failure, leads to police abuse and socioeconomic injustices, and acts to reinforce monopolies in the health care market, by ceding power over access to specific providers.

<comment><USER3> > it seems that the more appropriate assumption should be to not regulate a > substance, until it can be shown to be harmful

How are you going to stop people from dying when I sell arsenic as a baldness cure?

<comment><USER4> It is not just about harm. FDA rules on medical products also in fitness of purpose. If you can show that your arsenic preparation is effective and safe, it will be allowed on the market. Likewise MDMA preparations and cannabis preparations for specific purposes. (They cannot because it is schedule I. )

DEA governs the drug schedule, not FDA. That should be don from almost strictly criminal point of view, with major politicization.

<comment><USER5> > It is not just about harm. FDA rules on medical products also in fitness of > purpose.

This is because most drugs aren't really that good for you, however in the context of some illness, the disease may be worse than the unwanted effects of the drug.

A good example is methotrexate (MTX): wow, it really clears up its patients' arthritis, but it's so toxic that it's only administered to patients who are more at risk of dying of cancer than of the MTX.

I've appeared before the FDA (in pursuit of drug approval) and in my experience they're actually more than fair. The biggest problem is the assholes who try to game the system, which just gums up the works for everyone.

edit: I'm no longer in that biz so have nothing to gain by praising or condemning the FDA. The approval process is far from perfect but I honestly think it's pretty good overall.

</comment></comment></comment></comment><comment><USER6> Researchers having trouble due to Schedule 1 status in the states seems weird, when Canada just legalized recreational use. What has this war on drugs given America?

<comment><USER7> The honest answer is that it gives conservatives the ability to be racist and imprison minorities for minor offenses.

<comment><USER8> Taking HN threads into flamewar is not allowed here. We ban accounts that do this, so please don't. The effects on the community are destructive and basically always the same.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

<comment> King-Aaron It should be noted that the above comment is likely far less trying to incite a flame war, and more likely referencing the alleged reasoning for the 'war on drugs' was to directly marginalise African Americans and 'hippies'.

"The Nixon campaign in 1968, and the Nixon White House after that, had two enemies: the antiwar left and black people. You understand what I'm saying? We knew we couldn't make it illegal to be either against the war or black, but by getting the public to associate the hippies with marijuana and blacks with heroin, and then criminalizing both heavily, we could disrupt those communities. We could arrest their leaders, raid their homes, break up their meetings, and vilify them night after night on the evening news. Did we know we were lying about the drugs? Of course we did."

— John D. Ehrlichman, former Nixon aide, interview with Dan Baum

<comment><USER7> This is what i was going for, i should have cited my source

</comment></comment></comment></comment></comment><comment><USER9> THC is already an approved drug in the US under the name nabilone. It's schedule 2.

This newly approved drug is CBD. There is no need for the DEA to reschedule marijuana. They've already been ok with this contradictory scheduling for over a decade.

<comment><USER10> >THC is already an approved drug in the US under the name nabilone. It's schedule 2.

Nabilone is a THC _analog_ , with a slightly different molecular makeup than THC extracted from Cannabis. So no, THC is not schedule 2 under US law, as it is explicitly listed schedule I under the Controlled Substances Act.

Granted, the WHO recommends it be listed schedule III or IV, which I agree with...

<comment><USER9> Good point on nabilone being an analogue, not THC itself.

I guess my point is that the DEA doesn't require it's scheduling to be logical at all.

<comment><USER11> There is also GHB/sodium oxybate. It would be nice if this approval caused a major shift, but I tend to doubt it.

</comment></comment></comment></comment><comment><USER12> Most important quote:

> We'll continue to support rigorous scientific research on the potential > medical uses of marijuana-derived products and work with product developers > who are interested in bringing patients safe and effective, high-quality > products. But, at the same time, we are prepared to take action when we see > the illegal marketing of CBD-containing products with serious, unproven > medical claims. Marketing unapproved products, with uncertain dosages and > formulations, can keep patients from accessing appropriate, recognized > therapies to treat serious and even fatal diseases.

I can't believe the number of ridiculous statements I've seen and heard about medical marijuana. It's our modern-day equivalent of the witch doctor.

Looks like the FDA is finally going to allow legitimate research and medicine. [Edit] This will reign in all the unproven claims that marijuana is essentially a cure-all.

[Edit] I support complete legalization of marijuana for recreational use.

</comment><comment><USER13> Perhaps someone from the US could help me understand the federal vs state legality of cannabis in the USA?

Can a state override _any_ federal law?

Could federal-level law enforcement theoretically charge someone in a cannabis-legal state for drug offenses?

<comment><USER14> > Can a state override any federal law?

No, by the "Supremacy clause" of the Constitution.

> Could federal-level law enforcement theoretically charge someone in a > cannabis-legal state for drug offenses?

Yes. Because people interact much more with local police than federal police, it makes a big difference whether or not the local police enforce these laws. The Obama administration also chose not to enforce federal marijuana laws in states with less restrictive state laws. But both of these relate to enforcement rather than legality.

<comment><USER15> Actually, IIRC/IANAL, SINCE recreational use is legal in D.C., federally the government's hands are tied, because I believe what is legal in D.C. is legal in all federal jurisdictions (except where state laws says otherwise), so Federally, I believe Marijuana is no longer prosecutable per se, I could be wrong -- but states that still ban it can prosecute since it's their state law.

<comment><USER14> Sounds like a legal urban legend.

<comment><USER15> Maybe, I heard it on here, and they probably were 'IANAL' too.

</comment></comment></comment></comment></comment><comment><USER16> Marijuana is only schedule 1 because of Nixon attacks against his "enemies," hippies and blacks, as cooberated by Ehrlichman. And as the 1920's and Portugal prove, Prohibition always backfires and helps criminals.

</comment><comment><USER17> Wait sched 1? I can buy it at the vape shop nextdoor in detroit.

<comment><USER18> It's legal/decriminalized in some states/cities but that is in spite of federal law, not because of it. It's still illegal at the federal level.</comment></comment>


Streams: a new general purpose data structure in Redis - <AUTHOR> http://antirez.com/news/114 <comment><USER2> One thing I like about this post is the story of how the feature came to be: Someone who understood redis very well, thinking about the problem over literally years, eventually resulting in a more targetted and goal-driven thinking, and even that "specification remained just a specification for months, at the point that after some time I rewrote it almost from scratch in order to upgrade it with many hints that I accumulated talking with people about this upcoming addition to Redis."

I've been thinking about this sort of thing for a while, wanting to maybe call it "slow code" (like "slow food"). This is how actual quality software that will stand the test of time gets designed and made, _slowly_, _carefully_, _intentionally_, with thought and discussion and feedback and reconsideration. And always based on understanding the domain and the existing software you are building upon. Not jumping from problem to a PR to a merge. (And _usually_ by one person, sometimes a couple/several working together, _rarely_ by committee).

<comment><USER3> > This is how actual quality software that will stand the test of time gets > designed and made ..

No not really. It's called waterfall and we had it for decades.

And it was largely abandoned in favour of agile development because despite architects spending months designing something there was always something that was left out. Or the scope changed during those months. Or millions of other things that change whilst your hidden away from your customers/end users instead of delivering them new value every week.

For me personally the truth is somewhere between agile and waterfall. Some upfront design but not the slow code you refer to.

<comment><USER2> I disagree, I think "waterfall" (at least in stereotype) would be more like coming up with the specifications in advance and then _sticking to them_ regardless of new information from the world or from the experience of developing the software. That's not the story the OP told.

I also agree with the person who responded that there is a difference between building products, and building tools for building products -- building shared libraries/code/tools may require different approaches. One of the things I've noticed about shared code, is it's _much harder_ to fix rethought decisions or change paths. "Backwards incompatibility" doesn't even exist as a thing in your local app really, at least you can always at least theoretically global search and replace when you change APIs.

I don't think we've figured out any magic bullet process for developing software. It's definitely not the stereotype of "waterfall", but I don't think the stereotype of "agile" is it either -- at _least_ when it comes to shared library/tools. And I'm absolutely convinced that those shared tools which do well were almost always designed carefully and intentionally with understanding of the domain, not just slapped together with stimulus-response.

</comment></comment></comment><comment><USER4> I have a confusion about ID structure/format:

       The ID is composed of two parts: a millisecond time and a
       sequence number.  The number after the dot is the
       sequence number, and is used in order to distinguish
       entries added in the same millisecond.

Does this mean for example that 1506872463535.11 comes after 1506872463535.2 (because 11 > 2)? If so that means treating these as decimals (which will be easy to do inadvertently) will yield the wrong order (as would sorting them lexicographically). If so it seems like something other than a decimal point would be a better separator (colon perhaps).

<comment><USER5> Yes actually maybe it's a good idea to change the point with something else. Thanks for the hint.

<comment><USER6> Supporting `MAXSIZE` as well as `MAXLEN` on `XADD` would also handle a nice Kafka feature (the ability to define your log size in either number of messages or size on disk).

Something like: `XADD MAXSIZE ~ 2147484000 * foo bar` to cap the stream at 2GB + 1 node.

<comment><USER7> And if ids are timestamps, maybe we can define it as MAXTIME as well.

</comment></comment></comment></comment><comment><USER8> Projects tend to gain more and more functionality to match the new workloads they're being used to accomplish, and it must certainly be a difficult decision for project visionaries. Do I listen to my users and implement features that will solve their new woes, but in return accept increased complexity and higher learning barriers? Complexity sucks, but it's even harder to say no to users in pain.

I wonder if this opens such projects up to disruption, in the original sense of the word. I've seen the same teams forgoing apache for nginx, then forgo nginx for haproxy when it matches their needs. With additional layers of complexity, there may be accompanying opportunities for "simple but good" projects to gain traction.

<comment><USER9> It's also why software is a pop culture. Sophistication and completeness is seen as complexity and cruft by each successive generation, who start something new and simple.

I don't think it's very avoidable. Tech is genuinely getting incrementally better, but it's usually in a sawtooth pattern.

<comment><USER10> I'll agree that sophistication and completeness is often seen as complexity/cruft but it also _always_ comes with actual cruft as well since improvement is incremental and breaking APIs is annoying.

My favorite aspect of this cycle is when some features in the complex software become seen as so useful as to be required and standard, so when the new simpler version is created they have to figure out a novel way of providing that useful functionality in a simple and elegant way. And they do it, sometimes knowing they have made a significant advance and sometimes without knowing.

<comment><USER11> Do you have any examples of that second case?

Sounds too good to be true... but I'd _love_ to be proven wrong :)

</comment></comment></comment></comment><comment><USER12> I love redis, and this API looks amazingly simple. I'm sure I can think up a good use case for this, but the only problem I have with it is that time- series log data of this nature is increasingly becoming the defacto source of truth in the various models resembling some version or another of event- sourcing.

Obviously the general thinking is that event sourced time series data allows you to treat every other data source as derived state from the log data. If it gets written to the log, it's safe and that's the only real data that cannot be considered a redundant read layer. A common structure might look like:

1.) Event Sourced/Time Series Layer: Kafka/Kinesis>S3 takes in and saves log data

2.) Operational State Layer: RDBMS Constraints / Application Logic determine how operational state is derived from log data

3.) Indexing Layer: Query optimizations occur with redundant read layers in what is essentially all just various forms of indexing. This can be RDBMS index, ElasticSearch, MapReduce, Redis, etc.

Redis' place has historically been at #3, for many reasons. Whether an application has an event-sourced layer in which their operational state is derived from log data or is actually considered the primary source is something that is hugely variable. I would say that most applications do not make a distinction between #1 and #2, and just write state directly to an RDBMS. But while the operational state may or may not be considered redundant, depending on the application, the indexing layer is almost guaranteed to be a redundant layer. The redundancy of the index layer means that Redis operating purely in-memory allows the whole thing to be blown away with no consequence. To move it two steps down the data model to the defacto source of truth is a monumental shift in responsibility. Redis as an in memory caching layer has only ever had me have a cursory awareness of its capabilities in terms of saving to disk, but I would think that a fundamentally different use case like this will have me taking a serious look at where that functionality is at today.

All of this being said, there are plenty of use cases with kafka/kinesis which are done today which actually don't even save the log data at all, and just use them as an intermediary buffer to have multiple consumers on an event stream. There's also nothing stopping us from just having one of the consumers of this stream to be saving it to S3/Disk ourselves.

<comment><USER13> I would imagine this is best used for the "store a ton of events with some capped maximum size" Kafka use case (ie - realtime analytics, IoT data, etc). I just can't imagine using Redis as your source of truth for an event sourced system, especially without the partitioning and log compacting features that Kafka has. Still this seems like a pretty amazing feature to have in Redis, can't wait to start playing with it.

</comment></comment><comment><USER14> It's been a long time since I looked into this: is there now a way to configure a cluster of Redis instances such that you won't lose messages on node failure? If not, all the nice at-least-once delivery (or "effectively once" when you add message dedupe) you get with something like Kafka/Kinesis/GCP PubSub is gone.

If not, either people's messages don't matter /that/ much (which is fine, just not great for most of my usecases at the moment) or everyone's in for another round of "oh shit, where did the data go?"

Edit: Just in case we end up in CP vs AP datastore wars, please go read [https://martin.kleppmann.com/2015/05/11/please-stop- calling-...](https://martin.kleppmann.com/2015/05/11/please-stop-calling- databases-cp-or-ap.html)

At-least-once delivery requires neither CAP consistency (linearisability) nor CAP availability (any non-failed node must return a response in a non-infinite time), but is a very useful property!

<comment><USER5> Hello, the streams have basically the same characteristics as any other Redis data structure, that is, from the POV of a local node, you can configure strong persistence on disk, but on node failures, you have basically different tunable amount of _best effort_ consistency, it means that you cannot guarantee no messages are lost. So basically this means that you can:

1\. Use the default asynchronous replication, and live with the fact (if the use case permits this) that on failover, the message did not yet received the slave that will be promoted.

2\. Use WAIT to force synchronous replication to N slaves. This will not _still_ make Redis ensure you in mathematical terms that the failover will pick a slave that received the message, under complex partitions, but narrow the real world failure models leading to losing data to more "unlikely" cases. Yet you have just best effort consistency but with better real-world outcomes.

So Redis streams will be good choice if one of the above is acceptable.

<comment><USER15> What is the consistency model of redis?

It sounds like anything can be lost in redis during normal HA operations even with WAIT pushing to a majority of slaves. Is that right?

<comment><USER16> Eventually consistent unless you're only using a single node. I believe that Redis itself commits to disk at various checkpoints in time, so if a fail happens, you're really only guaranteed to fail over into a pool of data that's consistent up to the last checkpoint of the node you're moving to.

EDIT: And as antirez said above, you can WAIT to force synchronization to all nodes, so you would be pretty likely to fail over onto a node that has n-1 messages if it didn't sync in time. That still isn't guaranteed however.

</comment></comment></comment></comment><comment><USER17> The clear writing/documentation, concise API design, and clever implementation of antirez and the rest of the Redis team continues to amaze me. Redis is easily my most admired OSS project.

</comment><comment><USER18> Very cool! We've been doing time series with Redis using sorted sets, referencing items by timestamps and integer offsets, and using the clock shift workaround described in the article. Having this kind of thing consolidated down into a few Redis commands would be handy. The API looks clean, too.

</comment><comment><USER19> Two comments on effectively once stream processing.

1\. Consider adding an example for a stateful event stream processor client that saves the last read stream offset in redis, together with its current state and continues reading from that offset as an atomic operation. For example, a client that sums a stream of numbers, in order to have effectively once semantics would need to persist to redis the sum and offset together.

2\. Consider adding a stream read deduplication example to mitigate clients that reinserted the same event twice. It is not clear how the client should behave if it didn't get an ack and it resents an event. What is the correct resending semantics so the reader would effectively dedup? What is the right data structure used to dedup message ids without consuming too much memory, etc...?

</comment><comment><USER20> I've been using redis + resque[1] for a few side projects and I have to say I'm glad that streams are getting first class support in redis. I was always a little wary of hacking this sort of functionality on top of redis lists. It worked, but it sort of seemed a little bit fragile.

[1] [https://github.com/resque/resque](https://github.com/resque/resque)

</comment><comment><USER21> I'm very excited about this. I've been eying HTTP EventSource for a while now, but there hasn't been a good solution for the backend broker. Kafka is overkill and Amazon Kinesis' pricing isn't viable if you have lots of topics. This fills the need perfectly and Redis is already part of my stack.

<comment><USER22> Check out nchan.io, an nginx module that does everything you need to connected EventSource to redis backed pub/sub.

<comment><USER23> +1, good to see other developers out there using EventSource and nchan. We've been happily using both in production for almost a year now.

</comment></comment></comment><comment><USER24> Amazing ! nit: Change COUNT to LIMIT while it still time. Also can this primitive be used to replicate redis data instead of sentinel/cluster ?

</comment><comment><USER25> Under what circumstances would one prefer Redis streams over Kafka and vice versa?

<comment><USER26> I can think in some circumtances: 1 - You already have a Redis infrastructure and don't wanna or don't have resources to deploy a full Kafka infrastructure (3 kafka brokers + 3 zookeeper nodes)

2 - Kafka clients are not available (or are poorly available) for every programming language. Redis has a simpler protocol, so it has more/and better clients available and even if you use an exotic language, it is easy to write a client to it (well... easier than Kafka)

3 - Kafka AFAIK does not have any internal cache implementation, so every read is served from disk (+ page cache). This means that Redis Streams will (probably) perform much better for use cases when the consumers need to fetch data from old offsets.

edit: added reason number 3.

<comment><USER27> re: client support - I dunno, this seems like a pretty comprehensive list to me? I mean, there's even a rust client: [https://cwiki.apache.org/confluence/display/KAFKA/Clients](https://cwiki.apache.org/confluence/display/KAFKA/Clients)

<comment><USER26> Confluent only officially supports the Java client (and now has a Python, Go and .Net clients as well that I didn't know) and it is really recommended to use the client with the same version of broker due to protocol incompatibilities.

Most Kafka client implementations are open-source projects of their own, this is also true for most redis clients implementations, but again: Kafka protocol is much more complicated than Redis.

I haven't used Kafka with other languages besides Java or Scala, so I can't really say how mature are the other clients.

But my point about how easy is to implement a client for Redis if needed is still valid. =)

<comment><USER28> They also support a c reference implementation (which is how they get others).

</comment></comment></comment></comment></comment><comment><USER29> Slightly off-topic, but could the blog be adjust a little for mobile reading?

        <meta name="viewport" content="width=device-width, initial-scale=1">
        #content { max-width: 800px; } // replaces width: 800px

seems to do the job, and also behaves better in narrow desktop browser windows.

<comment><USER5> Thanks! I'll change it tonight.

</comment></comment><comment><USER30> Fantastic news, congrats Salvatore! Cannot _wait_ to replace some hacky Kafka uses with tried-and-true Redis4! :)

<comment><USER31> In what sense is Kafka (or your use of it) hacky? I have never used Kafka, but I have always thought of it as being more solidly engineered than Redis but also more complicated and perhaps tricky to deploy (based on blog posts I read).

<comment><USER32> In any context its used where the demand (by whatever measure you care to use: bandwidth, throughput, message durability, etc.) doesn't justify it or isn't a good use case of Kafka, for starters. That happens all the time, because every data and infrastructure engineer in the Bay Area wants to put Kafka on his resume.

<comment><USER15> But that doesn't explain why Kafka has any minimum the output required. Does it have usability issues?

A good tool should be able to be used at any scale.

<comment><USER30> Kafka has very poor tooling in my experience (a folder full of fairly buggy bash scripts...), and due to ZooKeeper requires a lot of operational care. For example, it's extremely easy to destroy a Kafka cluster by bringing a new, empty ZK server online with newer but incorrect data in its volume. ZK will happily trash the entire cluster thinking it has new instructions. So network isolation is key, which, while obvious, is another source of potential failure.

Kafka also has the JVM, which requires a lot of love to scale in my experience. I do not want my programmers messing around with GC options when writing to what should (to them) be exposed just like a regular file handle (except distributed across many systems). I strongly prefer to avoid Java applications at all costs - in my experience it takes years and years and years for Java based infrastructure to become relatively stable & reliable (see ElasticSearch 5.0, or ask anyone who has been oncall for a Tomcat based application). This is almost certainly personal bias, but it's my bias regardless.

Redis also has a _massive_ number of tooling / monitoring / ecosystem advantages, including hosted options, and can run on a single instance without configuration changes from the developers perspective.

I also have personal reasons to prefer Salvatore's work over the work of Confluent.

<comment><USER33> > For example, it's extremely easy to destroy a Kafka cluster by bringing a > new, empty ZK server online with newer but incorrect data in its volume. ZK > will happily trash the entire cluster thinking it has new instructions.

How does that happen? I mean a new, empty ZK server with never data than the rest of the cluster?

Also, please note that ZK is not meant to be a database, but a coordination service, it's guarantee is to have all nodes being always in consistent state and neither of its nodes allow to make any changes if there's no quorum. So if a new node somehow has more recent data with higher serial number it's expected that remaining nodes will sync to that.

<comment><USER30> Exactly right - in my case the situation was another team accidentally bringing a new ZK node with "bad" but "new" data online. Had there been network isolation, no issues. Had there been static cluster identifiers, also no issues. It was a messy environment, and it should have been prevented by operational diligence, but my point is redis is "harder to mess up". As on on- call engineer, I'll always go with simpler, foolproof tools. Another qibble is how gnarly the client-side driver for Kafka is...

I don't hate Kafka, I just don't like ZK and find redis has better tooling and a better track record at my shops :)

<comment><USER33> In order to connect a ZK host to the cluster its IP needs to be included in configuration of all the nodes.

It's hard to accidentally add node to a cluster. A person who can "accidentally" add a ZK node has enough permission to do a lot of more devastating things accidentally.

<comment><USER30> Yep. All it takes is service discovery and a not-totally-familiar with ZK jr. sysadmin.

This is all in service to my point about simplicity and safety.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER34> > However a special ID of "$" means: assume I've all the elements that there > are in the stream right now, so give me just starting from the next element > arriving.

I can already see lazy users just repeatedly reading $, and then dropping messages when they arrive faster than they read them.

Might it be safer to instead have command to ask what the latest ID in the stream is? You'd start off by using that to work out where the streams are, then construct an XREAD command to read from there. To construct your next XREAD, it should be easier to update the IDs from the ones you just read, rather than fetching the latest IDs again. Maybe.

</comment><comment><USER35> I've had a break from Antirez' blog for a while - it's fun to go back and see how good his English has gotten!

</comment><comment><USER36> Surely any discussion about the concept of logs and mention of Kafka should have a reference to this excellent article by Jay Kreps on LinkedIn:

[https://engineering.linkedin.com/distributed-systems/log- wha...](https://engineering.linkedin.com/distributed-systems/log-what-every- software-engineer-should-know-about-real-time-datas-unifying)

<comment><USER37> It [OP] was a bit cringe worthy, frankly. Obviously the Kafka papers about 'reconsidering the log' and 'unified view' and all that have been out there for years now.

The Kreps article is quite excellent and well worth the read.

</comment></comment><comment><USER38> _> A final important thing to note about XRANGE is that, given that we receive the IDs in the reply, and the immediately successive ID is trivially obtained just incrementing the sequence part of the ID, it is possible to use XRANGE to incrementally iterate the whole stream, receiving for every call the specified number of elements._

Love it. No need for an expensive SCAN command.

</comment><comment><USER19> The consumer groups proposal breaks the FIFO abstraction of a stream by allowing multiple clients to process a single stream.

Have you considered adding a semantic layer inside streams that allows each client to consume a substream? In effect the stream becomes multiplexed substreams.

If substreams makes the design too complex... have you considered server side stream 403 semantics? When a stream is manually deprecated it enters an immutable state and provides a redirect response with a link to another stream. This would allow multiplexing and demultiplexing streams without changing the client implementations too much.

For completeness I would state the obvious when fifo grouping is needed: 1\. Scaling stateful event processing by splitting streams and adding more clients (CPU limit) 2\. Scaling cross region replication by splitting streams and adding more tcp connections (network limit) 3\. Handling more throughput by splitting a stream into two redis nodes (disk I/O limit)

<comment><USER39> Don't use consumer groups and every client will get a complete copy of the stream. What is broken with that?

<comment><USER19> The use case I'm referring to is when the client must be sharded to avoid CPU or network or disk bottlenecks.

<comment><USER39> Then that's exactly what consumer groups help with but it sounds like you want partitioning then - which is exactly what Kafka does but with a little more automation.

Run multiple Redis instances and use a simple hash based on whatever key you want to route messages and get the throughput you need. It's probably never going to be part of the core Redis logic but should be possible as a module to do the routing when used in a Redis cluster.

<comment><USER19> What I'm referring to is changing the partitioning dynamically by splitting streams, not just redis nodes. Here is one implementation example [http://docs.aws.amazon.com/streams/latest/dev/kinesis- using-...](http://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk- java-resharding-split.html)

Doing it without server support is tricky.

</comment></comment></comment></comment></comment><comment><USER40> Is there a reason to choose millis as the granularity instead of micros or nanos? Is it because there's a stronger expectation of machines in a cluster agreeing on what milli it is "now" vs the other granularities?

I'm kind of thrown by the idea of putting the timestamp / stream-id in the XADD command, I would have thought the server would assign that, since one of the strengths of redis's single threaded nature is consistency: what's in redis is the truth. If you allow clients to specify timestamp, what happens when ntpd isn't running on some? I probably misread or misunderstood it.

Could you allow specifying `$` as the timestamp to tell the server you want it to use whatever it thinks the current time is as the timestamp / stream-id?

<comment><USER5> Hello, the stream implementation does not need for the different servers (for instance master and its slaves) to agree about the time. Simply the server that receives the XADD command will generate the ID (and the time part of the ID) to attach to the item. All the other participants in the replication will accept the same ID, because clients will use " _" to specify the ID, while the command is rewritten to slaves with a specific ID. Example, I run into the master:

        127.0.0.1:6379> xadd stream * a 1 b 2
        1506977609865.0

But this is replicated as (output of redis-cli --slave):

        "xadd","stream","1506977609865.0","a","1","b","2"

So XADD allows to specify an ID just for replication / AOF pruposes, not because clients should actually specify an ID normally. However of clients really want to do that, they could but at the risk of getting errors, for instance:

        127.0.0.1:6379> xadd stream 10.0 a 1 b 2
        (error) ERR The ID specified in XADD is smaller than the target stream top item

Redis will anyway not accept any ID which is smaller than the current top-item ID.

The reason why it was chosen to use milliseconds instead of nanoseconds is because, for most applications to query for sub-millisecond ranges is likely not useful, so to see even larger numbers in the ID maybe is just unpleasant if not useful, however we are still in time to change this if there are good motivations. But being the time the one produced by the local host, after a failover the IDs are generated by another host. Milliseconds can still more or less match with good time synchronization, but nanoseconds? So it's like if this additional precision will be just used to store non-valid info.

<comment><USER40> > Redis will anyway not accept any ID which is smaller than the current top- > item ID.

I was reading on mobile earlier and maybe missed this point, excellent. I also didn't realize that clients would use (star) to specify the ID and that the receiving server turns that into an actual ID before replicating it / AOFing it.

> But being the time the one produced by the local host, after a failover the > IDs are generated by another host. Milliseconds can still more or less match > with good time synchronization, but nanoseconds? So it's like if this > additional precision will be just used to store non-valid info.

I think most failovers necessarily take longer than a millisecond so any resolution smaller than millis would _probably_ be okay, but yeah this is not a compelling reason to switch to micros/nanos. My suggestion to switch to micros/nanos was more to try to reduce the number of collisions requiring the server to de-dup / assign sequential sub-epoch numbers to events arriving during the same server tick. I guess that's not a big issue though.

Thanks for the reply, Salvatore. Redis is one of my favorite codebases and projects.

</comment></comment></comment><comment><USER41> Redis Streams could be a nice real-time counterpart to [http://traildb.io](http://traildb.io): For instance, use Streams in Redis to record data in real-time and periodically store it in TrailDBs for long-term archival and analysis.

</comment><comment><USER42> This may actually solve an issue I was about to tackle, which is high-speed notification delivery. Currently I was going to do a nasty wrestle of PUB/SUB with Lists and blocking keys to try and get a cluster of message processing servers to digest notifications when they get added to the "queue". My purpose in this is notifications, as in actual push notifications for mobile/web/etc. This seems like it fits perfectly with what I had planned. Especially since this ensures message delivery instead of fire-and-forget as you mentioned. As a question though... is the XACK commands working in your current branch? Since that is key to my usage of ensuring message consumption.

<comment><USER42> Essentially, I was going down this route if anyone's curious....

[http://code.flickr.net/2012/12/12/highly-available-real- time...](http://code.flickr.net/2012/12/12/highly-available-real-time- notifications/)

</comment></comment><comment><USER29> Could the difference between `MAXLEN ~ 1000000` and `MAXLEN 100000` be handled internally, by marking the overflowing items as deleted until a whole block can be removed? Looks like this tombstone functionality is already planned, would make the API simpler.

<comment><USER40> I would suggest making the efficient behavior the default and let people use `= 1000000` when they know they really want the expensive but exact behavior.

</comment></comment><comment><USER43> Are there plans for a "unique count" over an XRANGE?

I currently use multiple Sorted Sets (one set every 5 minutes) and Union 30-60 worth to produce a "rolling window" of uniques.

I can see an alternative where I just request a unique count of elements within an XRANGE.

</comment><comment><USER44> What sort of compression do the blocks undergo? E.g. does periodicity of the timeseries help reduce the the space of the 64/128bit timestamp? Gorilla[1] style compression would be great, although it'd likely make sub block level range queries tough.

[1][http://www.vldb.org/pvldb/vol8/p1816-teller.pdf](http://www.vldb.org/pvldb/vol8/p1816-teller.pdf)

<comment><USER5> Hello, yes IDs are delta compressed so they use actually just a few bytes per entry (often just 2) instead of 16.

</comment></comment><comment><USER45> Mirror: [https://webcache.googleusercontent.com/search?q=cache:https:...](https://webcache.googleusercontent.com/search?q=cache:https://blog.even.com/anti- perks-4e2e49e80983)

</comment><comment><USER30> I have one follow up question - is TTL a planned feature? Being able to set a TTL on the _stream itself_ and -also- on the messages would be extremely nice. While MAXLEN prevents a queue from being extremely large, I also want to remove "stale" data after a configurable time period.

Use case: A log of network latencies, where a user might currently `XREAD` with a timestamp 10 minutes in the past, would be able to save on memory usage by expiring log entries > 10 minutes, and then being able to `XREAD STREAMS strm 0` and let Redis (and therefore the infrastructure, not my code, manage data retention).

Also, how does this work re: evictions? Say a node is at max memory, will entire _streams_ be evicted, or (I hope) the oldest messages in the LRUed or LFUed queues.

</comment><comment><USER46> there's video[1] at the end explaining this new feature

[1] [https://www.youtube.com/watch?v=ELDzy9lCFHQ](https://www.youtube.com/watch?v=ELDzy9lCFHQ)

</comment><comment><USER47> here is an approach i successfully used before: since timestamps are read from clock the epoch could be a persisted value and a few nibbles could be used for the instance id and a sequential number. with that you get 64bit numbers for easy use and computation without loosing the time information at the cost of a simple transformation function. it simplifies the interface much and generally makes things faster.

many clients will even not need that timestamp anyways.

</comment><comment><USER48> Why not just use sequence ID? I'm confused about why a timestamp is important. The sequence ID gives us ordering, is always guaranteed to be increasing.

<comment><USER5> Because with the way stream IDs are conceived you also get time-based range queries for free. With time series this is very important in many use cases.

<comment><USER48> I see, so this composite structure is in lieu of having two distinct fields exposed in the API?

</comment></comment></comment><comment><USER49> This looks very cool. Can't wait to try it out.

</comment><comment><USER50> After reading this I'm still not sure I understand. Time-series data I can see, but is there a use-case outside of this?

<comment><USER51> Poor-man's Kafka?

</comment></comment><comment><USER52> I am looking forwards to trying this out as an easy access cqrs / event sourcing entry point.

</comment><comment><USER53> The use of `+` and `-` in XRANGE seems inconsistent. Why not use `0` and `-1` like LRANGE?

<comment><USER5> Because they do not mean a position, but a special ID.

<comment><USER54> It seems that "$" is a special ID for the last message, as opposed to the last possible message.

I would humbly suggest that "^" would be a suitable symbol for the first message in a stream. ^ and $ are used in regex (and vim) in a similar way.

That way you could write "XREAD BLOCK 5000 STREAMS newstream ^" and get all the messages in a stream from the beginning, and then block until a new message comes in all with a single command. You would still be able to add a count if needed, to prevent client flooding.

<comment><USER5> Exactly, $ is the last message ID, + the greatest, - the smallest. I chose the dollar exactly because of regex assonance. However the corresponding ^ is kinda useless because with XREAD we specify the last ID we got, so it would result in not returning the first element of the stream. It means that it's more useful to specify just 0 in that case.

</comment></comment></comment></comment><comment><USER55> Funny, we also built a lot of our technology around the Streams concept.

[https://github.com/Qbix/architecture/wiki/Internet-2.0](https://github.com/Qbix/architecture/wiki/Internet-2.0)

</comment><comment><USER56> nice to know, they have added a new data structure !</comment>


Tell HN: whatbrowser.org (Google) Is Down - <AUTHOR> I noticed that whatbrowser.org has been returning a 404 for the past few days.<p>Anyone else notice? I use this site for support to help people identify their browser. <comment><USER2> Will [https://www.whatsmybrowser.org/](https://www.whatsmybrowser.org/) help? Seems to do the trick for me.

</comment><comment><USER3> It had a notice that it was going to be shut down: [https://web.archive.org/web/20181031191521/http://whatbrowse...](https://web.archive.org/web/20181031191521/http://whatbrowser.org/)

<comment><USER4> Examining the page source, there's this line:

var browserVersions = [{'Firefox': '27', 'Chrome': '44', 'Chrome OS': '34', 'Opera': '30', 'Edge': '12', 'Safari': '8', 'IE': '11'}];

Firefox 27 was released on February 4, 2014, and Chrome 44 on July 21, 2015. Since the snapshot was in October 2018, the page apparently hadn't been updated for at least 3 years.

Edit: Here is a report of the bug in the wild: [https://webcompat.com/issues/17258](https://webcompat.com/issues/17258)

</comment></comment><comment><USER5> Google has shut down too many projects lately.

So much so that I'm now nervous using Photos, Keep for notes and even Drive (it was replaced with One--although I can't figure out what changed.

Does anyone feel the same?

<comment><USER6> Yes, I'm nervous too, although to be fair when they do shut down a service, they generally let us know and allow us to download our data. I'm not aware of any service they did not do that, but there have been so many so maybe. Without that history I would not use any of their services.

<comment><USER5> Sure, that's true but it's still a pain to start using something and they shut it down after a couple of years.

I bet organizations have had a bad time with Google Inbox's shutdown, for instance.

</comment></comment></comment><comment><USER7> [https://supportdetails.com/](https://supportdetails.com/) is another option that's been around for over 10 years

<comment><AUTHOR> I'm kind of hoping that someone at Google will notice this and get it back up and running.

Google has made it near impossible for members of the public to contact them.</comment></comment>


Finding Chicago's Best Pizzas - <AUTHOR> https://stevedolinsky.com/ultimate-chicago-pizza-quest <comment><USER2> Take the pizza fights to /r/chicago, please. They love arguing about it.</comment>


Show HN: Recursive DNS Server Fingerprint - <AUTHOR> https://recdnsfp.github.io/ <comment><USER2> It's nice to see a country as large as Brazil having 24% of its domains with DNSSEC records (973k out of 3.9m domains). I expected it to be close to zero.

This is very interesting work. I wish there was an easy way to see DNSSEC statistics for each ccTLD side-by-side with the fingerprint report.

<comment><USER3> brazil is the only country other than US that understood internet control.

they have their own, centralized registrar, and they use proper tlds such as gov, jus (justice) etc with their own tls system (which sometimes update faster than browsers can keep up so you have to add root signatures manually to your systems)

<comment><USER4> I was actually surprised when Kaspersky announced NIC.br was compromised and many banking sites where hijacked. If I remember correctly, they (NIC.br) identified a vulnerability but then denied Kaspersky claims.

[https://www.wired.com/2017/04/hackers-hijacked-banks- entire-...](https://www.wired.com/2017/04/hackers-hijacked-banks-entire- online-operation/)

<comment><USER3> Did anyone ever release any more info on this?

so far, all points to the bank falling for a scam and releasing credentials to nic.br

</comment></comment></comment></comment><comment><USER5> This is a very cool use of RIPE Atlas!

Note that it's not going to flag many of the censorship apparatus, because they will inject replies only for queries matching their denied patterns.

Reversing that list in a useful way remains tricky, to say the least.

</comment><comment><USER6> Seems odd that their tests have a drastically different number of probes from different source countries. total_probes ought to be exactly the same from every source, for a more rigorous experiment.

<comment><USER7> It sounds like they used all the probes available. Many of the countries simply have very few RIPE Atlas probes available; I don't think it's reasonable to only select 5 probes from the US, because that's how many are installed in Vietnam; if you did, you're unlikely to pick any of the probes that showed this behavior.

Instead, it's better to report the total and suspicious numbers, and take the percentages with a grain of salt on low total probes.

<comment><USER6> I thought the point was that they queried many available DNS servers, not that they did so from as many different locations as possible. Even if they only have a dozen sources in a given country, can't they still query all the DNS servers they know of from there?

<comment><USER7> > We used all RIPE Atlas probes (~9000 probes) to send DNS queries to 220.127.116.11. > Each probe issued several queries, a single query covered one of the > features described above (e.g. DNSSEC validation, IPv6 only-domain > reachability, NXDOMAIN redirection, …).

My understanding was that they did the same queries from as many network locations as possible, and looked for unexpected results.

Querying more known public dns IPs would provide better confidence that a given probe was attached to a network that hijacked DNS, but still wouldn't tell you very much about the internet in a country with a low probe count.</comment></comment></comment></comment>


Friendsheet - A better Facebook photo browser - <AUTHOR> http://www.friendsheet.com

<comment><USER2> Shameless Plug: <http://news.ycombinator.com/item?id=3438290>

I did a Facebook photo browser too, just two weeks ago. Zach's is definitely much more polished and professional, but here are some features of mine.

\- It's single page app. All Ajax, no page reloads. HIstory supported.

\- Slightly more photos(users wall)

\- source is on Github: <https://github.com/nambrot/Facebook-Photo-Browser>

\- accompanied by a blog post on architecture: <http://nambrot.com/blog/4>

The demo runs here: <http://fbphotobrowser.herokuapp.com/> Deployed on Heroku, backend Node.js, frontend: Backbone

Feedback is appreciated

<comment><AUTHOR> pretty cool, i'll probably do some ajaxy page transitions at some point.. just hacked this together. Friendsheet is all html / JS.. hosted 100% on S3

</comment></comment><comment><USER3> Looks really good, and it's a very nice way to view the photos. I like that you also implemented a way to like and comment on the photos. Very engaging and fun.

I also like the design a lot.

Doesn't Facebook have a policy against "alternative ways to view Facebook" type applications? Didn't I see a conversation on HN the other day about a guy who got blacklisted for life for making an ads free version of Facebook or something like that?

Anyways, good job

<comment><AUTHOR> thanks! definitely inspired by fb, but with some streamlined tweaks.

I don't think facebook policy directly states that, it's more about not misleading users into thinking something is made by facebook. And definitely anything that circumvents ads is against policy.

</comment></comment><comment><USER4> Can someone educated me why this is so "awesome"?

the first problem I've found is that it has no indication or hint how to navigate prev/next in zoomed in mode.

<comment><AUTHOR> right left arrows... and soon swipe on ipad

I'll definitely add a tutorial / help section for ya

</comment></comment><comment><USER5> Zach did it again. Love the UI, so much faster to use than the Facebook newsfeed!

</comment><comment><USER6> Looks really nice. Looks inspired by Pinterest I guess..

<comment><AUTHOR> yes, pinterest combined with timeline

</comment></comment><comment><USER7> Screenshots?

I don't have a Facebook account..

<comment><AUTHOR> <http://cl.ly/2I2Y062q000Q1w0e3z1P>

</comment></comment><comment><USER8> Looks sweet! Bookmarked.

</comment><comment><USER9> Love it.

</comment><comment><USER10> looks really cool!</comment>


Craigslist personals associated with 16 percent boost in HIV infections - <AUTHOR> http://arstechnica.com/science/2015/02/craigslist-personals-associated-with-16-percent-boost-in-hiv-infections/ <comment><USER2> Related article: "Cheap Smartphone Dongle Diagnoses HIV And Syphilis In 15 Minutes".

There's startup potential there.

<comment><USER3> "Vertical integration."

</comment></comment><comment><USER4> Actually I like the critical causation and correlation discussion under that subject. But I would like to propose that we are with any other suggested social causality as critical as we are here.

It might seem convenient to use that argument when it comes to defending something dear to us such as Craigslist. But when it comes to more sensitive social and economic issues I wish we would see the same skepticism.

</comment><comment><USER5> There is a gaping hole in this study which seems to be making the rounds on the major news outlets: why did the authors not control for the arrival of other personal dating sites?

I don't doubt the correlation but to pin this on craigslist specifically seems dubious at best.

<comment><AUTHOR> They looked at Craigslist in terms of its rollout of local sites for the various communities:

 _> Conveniently, the spread of Craigslist provides a natural experiment: local sites were rolled out gradually over time, allowing the comparison of infection rates before and after their arrival..._

 _> The data suggests that there was no rise in infection rate in advance of Craigslist opening shop, and there appears to be a bit of lag after a local site opens. This makes sense given that each city will take some time before users discover and start using the service, and the authors looked into this in more detail by tracking the number of ads present in each local site. Within a year of the opening of a local Craigslist, as activity picked up, HIV infections began to rise. The effect remained after the authors controlled for a variety of other demographic factors in each area._

So if the data says you can consistently expect a spike in infection rates a year after a local CL opens, and that effect is consistent across communities, it seems unlikely that other sites would be affecting that -- unless they were rolling out their own local sites in the same communities as CL was, at roughly the same time.

</comment></comment><comment><USER6> 16% boost in [new] HIV infections, or a 0.7% increase in HIV cases.

Of course the title makes it sound 23 times more severe.

</comment><comment><USER7> In related news: sex associated with 100% boost in sexually transmitted diseases.

</comment><comment><USER8> This kind of article repulses me. Making the link between Craiglist and HIV rate doesn't make sense, correlation is not causality.

<comment> barry-cotter You can never absolutely prove causality. But as is pointed out in the article Craigslist became available in different cities at different times. The pattern of a lag followed by an increase in STDs was found in many cities.

Whether one is repulsed by the implications of a study has no effects on its truth value.

<comment><USER9> Something else could have caused both the STD increase and the appearance of a local craigslist presence. Like, say, a certain critical mass of population density.

<comment><USER10> Exactly. Craigslist isn't randomly opening their local sites. They are doing so in response to changes in communities following some semi-strategic growth strategy. Its not a natural experiment as claimed by the authors at all. Its like looking at the effects of google fiber on the cities it launches in without taking into account that those cities were selected by google for a reason. Quasi-experimental methods would be helpful here like a propensity score matching to match the cities to others that are like them and make pseudo-controls.Even then you can't consider this remotely a natural experiment.</comment></comment></comment></comment>