Whatever happened to the IPv4 address crisis? - <AUTHOR> http://www.networkworld.com/news/2014/021714-ipv4-278692.html <comment><USER2> This article is focused on the US, a country which was never really going to feel the brunt of the IPv4 crunch.

For an example of a _real_ victim, look at Qatar, a country which only has a single IP address _for the entire country_ (everyone sits behind a NAT): [https://en.wikinews.org/wiki/Qatari_proxy_IP_address_tempora...](https://en.wikinews.org/wiki/Qatari_proxy_IP_address_temporarily_blocked_on_Wikipedia) [0]

Whenever someone from Qatar decides to vandalize Wikipedia, Wikipedia is forced (temporarily) to block the entire country from accessing Wikipedia. This has an adverse impact on the rest of the country.

Non-Qatari Wikipedia users also suffer, because Wikipedia makes those blocks very temporary (since they are effectively shutting off an entire country), which makes it easy for those vandals to regain access to Wikipedia quickly.

[0] This sad state of affairs is not _solely_ due to IPv4 (incompetent/apathetic network administrators are also at fault), but it's a contributing factor.

<comment><USER3> I can assure you the US is feeling the brunt of the IPv4 crunch. I'm building something that needs massive amounts of public address space and I simply can't get IPv4 addresses for it. I've asked multiple providers. You are LUCKY if you can get a /28 at this point. Hoarding. Rolling with IPv6 instead.

I get you have an interest in the Qatar/Wikipedia situation, but it seems to be unlikely due to IP address resource management issues. Probably more like tyrannical control, if anything.

<comment><USER4> Then there are guys like me. I have a /24 routed to my home (pre-ARIN legacy space registered in the early 90's!)

<comment><USER3> Interested in running a micro cloud? See comment above.

</comment></comment></comment></comment><comment><USER5> This is an extremely US-centric article. ARIN was never in as dire straits as the other RIRs. In Europe and Asia the situation is much worse. For example, lack of IPv4 addresses delayed DigitalOcean's growth in Amsterdam, and carrier-grade NAT is already being used by some consumer ISPs in Europe and Asia.

<comment><USER6> The article also ignores the fact that running dual-stack (like Comcast) requires just as many IPv4 addresses as before. Even after ISPs get IPv6 working flawlessly, their customers still need to access IPv4 websites, and that means CGNAT or some messy kind of tunneling will still be necessary.

<comment><USER7> Then don't run dual stack. Run native IPv6 and NAT64/DNS64. Or in DO's case make IPv4 access optional for smaller droplets and charge $1/month extra for it. In reality things like DB servers or backend app servers don't need public IPv4 addresses, and this would speed up IPv6 adoption considerably.

<comment><USER8> Customer applications which require IPv4 and haven't been upgraded to support IPv6 still need a v4 address, and customers probably have to support their own clients that only use v4. You still have to ship the customer both L3 protocols. So the carrier would at least need to ship a NAT'd v4 address; v6-only would basically frustrate/anger/alienate a whole lot of customers, which is dumb from a making-money-with-my-company perspective.

In reality you can't just decide for your customers that they do or don't need something - you have to ask them what they need (if you want to continue having customers). Nobody's going to re-engineer all their shit to support your wonky network if they can just go to another company that provides them what they need.

<comment><USER7> No. You provide the best service to your customers for the best price. For DO's case, default to IPv6 and charge extra for each IPv4 address used. Currently, they charge $5/month for their cheapest droplet. Change that to $4/month and charge an extra $1/month for an IPv4 address. This way if my setup is more complex than a single droplet running everything, I can save some money on VPS's that don't need public IPv4 addresses (the database servers, application servers, etc.)

For Comcast and the like, once again give me the option to either do NAT64 or a full dual stack. As a regular consumer I probably won't care. As a gamer or a developer I might.

The IPv6 transition is going to happen sooner or later. Either you are going to make it painless for your customers by providing IPv6 early and using strategies to make the transition to IPv6-only smoother or you are going to make your customers suffer.

<comment><USER9> It won't help. If they charge $4+$1, it would not be seen as "they just changed the price structure". It would be seen as "they are charging for something that used to be free". Especially as marketing folks won't miss the chance of advertising "our package is now $4 _" small print "(_ additional charges apply)". And most customers will be royally pissed at that, both by the fact that they got advertised $4 but have to pay $5 and by the fact something that used to be free now isn't. The fact that the whole package costs the same won't matter, people don't think in those terms. And then it would get the neutrality angle - they now charge extra for using specific protocols! we warned about this and now it happens! - and the whole thing would become a huge mess.

<comment><USER7> It's already happening. Most providers charge $1-2/month for every additional IPv4 address AND you have to justify why you need it. EC2's Elastic IP's also don't get added to EC2 instances automatically; you have to add them manually and newly opened accounts are limited to I believe 10 IPv4 addresses for all EC2 instances. That's what IPv4 exhaustion looks like today.

This has nothing to do with net neutrality. It's progress of technology. Your ISP doesn't give you IPX access do they? Instead this is a natural progression of technology. Want to use old tech? Pay a premium!

<comment><USER9> That's _additional_ IPs. But one IP has been given for free (well, not in all setups, but in common packages). Unlike IPX. This may be progress for you, but for those who still need their IP setups working it's nothing but trouble. That's why it doesn't change - because people hate to change already working setups.

<comment><USER7> How much more common than EC2 can you get?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> It's here. IP addresses are costing more and providers are less generous with them. It used to be common to get a handful with a dedicated server, now you get one or two.

I have a friend who runs a small low-cost minecraft hosting. He stopped giving his customers dedicated IPs at all. They get a range of ports and a hostname with appropriate srv records added.

That's the other result, technical work-arounds. You can point to a particular service on a particular port with an srv record, host multiple websites on one IP, even SSL-enabled ones with SNI, etc.

<comment><USER11> This is basically how it should work. Prices rise until alternatives become worth thinking about.

<comment><USER12> that's the economic way of thinking. the engineering way would be to fix the artificial scarcity by implementing a technically superior solution.

<comment><USER13> There is always an endless line of other things to deal with in both the economic and engineering realms. Squeaky wheels tend to get the attention in both. When the IP situation causes enough discomfort, there will be an avalanche transition, and probably not a moment sooner.

<comment><USER12> i agree completely, to my great distaste.

</comment></comment></comment></comment></comment><comment><USER14> I have noticed a change in approach, even if unconscious. Instead of predicting doom, they have started to celebrate small victories (like Google IPv6 traffic passing 3%). I think that's natural when the size of this undertaking is so great.

Unfortunately IPv6 adoption is not a matter of just providing access lanes to this wonderful new technology. IP permeates too much of the infrastructure, tooling, etc. How could it not? Some companies might find the cost/ROI of working around IPv4 limited address space to be less than migrating to IPv6.

<comment><USER15> There are also tons of little things. Here in Indonesia, I know my ISP (FirstMedia) runs IPv6 internally on some things. Their web sites are available over IPv6. But I can't connect from home over IPv6. Supposedly the consumer hardware on my side supports IPv6 but it doesn't do so very well and I haven't bothered trying directly through the cable modem yet.

</comment></comment><comment><USER16> [http://cr.yp.to/djbdns/ipv6mess.html](http://cr.yp.to/djbdns/ipv6mess.html) (circa 2003)

<comment><USER17> That was always a question I had in the back of my mind. We have this huge address space. Why not reserve a single prefix that means "look at the lower 32-bits of this address and use it as an IPv4 address"?

<comment><USER18> Every IPv4 packet needs a source address. If the client is IPv6, what is the IPv4 source address?

You would want to assign an IPv4 address to the client transparently; and, since they're scarce, share it with several clients. That's what NAT64 does.

<comment><USER19> Why not just have the first 32 bits of ipv6 as equivelent to ipv4

so the ipv6 address of 188.8.131.52 is 18.104.22.168:0

<comment><USER20> What does that gain you over the mapped suffix, where you embed the IPv4 address at the end? (which is used by NAT64, etc.)

<comment><USER19> because as I understand routing is still done from the MSB. So you leave the IPv4 existing infrastructure in place with no modification, but all those /32 addresses now become useful and all the existing IPv4 infrastructure and routing paths still work.

Rather than having to replace every single piece of hardware, software, nameserver and routing stack that works on IPv4. I could plug my IPv6 router into any ISP that gives a v4 address and have as many v6 addresses as could ever be needed.

It makes no sense to me why you would route IPv4 addresses on v6 LSB, it completely ignores current internet infrastructure and routing.

<comment><USER20> What you propose already exists and is called Teredo,6rd or other tunneling protocols (6to4/6rd is probably the best fit with getting a /48 per IPv4 address). Except they again map the IPv4 space into the suffix of the IPv6 address (or do no mapping depending on the protocol).

But you don't want to do that forever as you are now paying for a IPv4 header PLUS some more headers instead of just one IPv6 header if you have native IPv6.

<comment><USER19> No, I'm not talking about tunneling, I'm saying natively route at the v4 level, and the header will be less, because +160bit addresses will only be used when required.

so your routing table holds "66/4 port 1" ,"* port 2"

instead of hundreds of millions of entries to get the same thing by having the "66" >64 bits deep into the address (or worse _:_ :66:* port 1, which breaks everything - hell how is this even done now?).

My point is, if the v4 address was in the MSB as standard, IPv6 would be working in virtually every single IPv6 device already.

As it is, we are all still using workarounds (and VPNs).

<comment><USER20> > No, I'm not talking about tunneling, I'm saying natively route at the v4 > level, and the header will be less, because 128bit addresses will only be > used when required.

There are multiple issues with this. The first and probably most important one is that it doesn't address routing table fragmentation, which is pretty much solved with IPv6, because most ISPs will end up announcing on the order of 1 or 2 prefixes instead of dozens, which can't ever be aggregated (like is the case in the IPv4 world right now and will only get worse).

The second one is, that it doesn't gain you much in terms of deployment over IPv6 + tunnels.

> so your routing table holds "66/4 port 1" ,"* port 2" > instead of hundreds > of millions of entries to get the same thing by having the "66" >64 bits > deep into the address (or worse ::66:* port 1, which breaks everything - > hell how is this even done now?).

Ok.. I have no idea what you are talking about here (mainly your notation is leaving me confused)..

> My point is, if the v4 address was in the MSB as standard, IPv6 would be > working in virtually every single IPv6 device already.

Even in the presence of NAT?

<comment><USER19> the routing table is the same as now, just that the IPv4 address becomes the network which sub routes. IPv4 hardware doesn't need to care about sub routing.

my point is with that notation, right now we have say a google address of 220.127.116.11

How does IPv6 handle retaining all the existing work and man hours that has gone in to making packets go to the 66.249.73/24 network, as quickly as possible, from anywhere in the world.

It seems to me it expects every administrator from top to bottom to start from scratch, then everyone is scratching their heads as to why that hasn't happened.

>Even in the presence of NAT?

No, and this is a good thing, only IPv6 devices which have their own IPv4 address/network can issue IPv6 addresses on that network. This is a good thing. e.g. I get 66.249.73 to be 421D:4900:0:0 in hex.

In what universe does this need to be ditched and started from scratch, and making something:something:0042:1D49 re doing - by hand - every NS, routing table etc

Where did these hundreds of millions of higher level routing tables suddenly come from?

It keep being phrased as "what is the source address of an IPv6 host on an IPv4 network". It seems to me the answer should simply be "the first 32 bits of the IPv6 address" \- and it seems stupid it's not structured like this.

<comment><USER20> > No, and this is a good thing, only IPv6 devices which have their own IPv4 > address/network can issue IPv6 addresses on that network.

So you will still need tunneling to make it work. As you will still have to run CGN to get all your customers online. Unless of course you do want to change the network infrastructure. At which point your solution gets much worse than plain and simple IPv6.

> It keep being phrased as "what is the source address of an IPv6 host on an > IPv4 network". It seems to me the answer should simply be "the first 32 bits > of the IPv6 address" \- and it seems stupid it's not structured like this.

Sure.. that solves the routing problem (except for IPv4 routing table explosion), but doesn't solve the problem that IPv4-only hosts still can't talk to IPv5-hosts. You send [IPv4][IPv5] packet to IPv4 host. Huh? What's that IPv5 thing? Or the other way around.. IPv4 host sees AAA record (with IPv5) in DNS..

Your proposal solves nothing that can't also be done with IPv6 and tunnels if you are really keen on keeping your 5 old router running a few more months before throwing it away (which you can't do anyway, as your limited FIB size will force you to buy new hardware anyway, due to IPv4 route table fragmentation).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> > The day of reckoning still looms – it's just been pushed out as the major > Internet players have developed ingenious ways to stretch those available > numbers.

To me, this indicates something either broken about IPv6 or a lessened severity of the IPv4 problem: If it's better to apply bandaids to IPv4 than to roll out IPv6, then either IPv6 is not easy and flexible enough to be a viable alternative, or the problems faced by IPv4 are not as intractable as was suggested.

<comment><USER22> It's both. IPv6 is not an easy migration, and people already have decades of experience squeezing the most they can out of IPv4, so the short-term solutions have just been to just squeeze IPv4 a little harder until everyone working on IPv6 gets it fully operational.

</comment></comment><comment><USER23> This is an interesting article, but it contains some rather surprising innumeracy in its cavalier comparison of 2^128 to the number of grains of sand in the earths crust. 2^128 is an enormous number, roughly 3.4e38:

                                   2**128 = 3.4 * 10**38
      grains of sand to one mile down [1] = 5.1 * 10**26
     stars in the observable universe [2] = 7.0 * 10**22
             estimated grains of sand [2] = 7.5 * 10**18

This means that every star can have 100 planets (equals 7e24 planets) each with 50 trillion IPv6 addresses.

[1] [[http://www.teracomtraining.com/tutorials/teracom-tutorial- IP...](http://www.teracomtraining.com/tutorials/teracom-tutorial-IPv6.htm)]

[2] [[http://www.npr.org/blogs/krulwich/2012/09/17/161096233/which...](http://www.npr.org/blogs/krulwich/2012/09/17/161096233/which- is-greater-the-number-of-sand-grains-on-earth-or-stars-in-the-sky)]

</comment><comment><USER24> The main problem the 20 years ago with the take off of the internet as a mass networking standard it was blindingly obvious that ipv6 was deeply flawed - IPv6 should have been taken out behind the woodshed back then and ipv7 or 8 done properly.

When I looked at it 19 or the 20 involved in the RFC for ipv6 where from academia plus one guy from bell labs.

Migration and Interpenetration should have been the highest priority in the design on a replacement for ip4

<comment><USER25> Maybe. Or maybe we will suffer tremendous amount of pain for some period of years after which we will have a much brighter future as opposed to an ipv7 that made too many concessions and left us with a mess that would be politically impossible to _ever_ fix.

<comment><USER24> mm yes or maybe know when to let a flawed standard die when it is over taken by events.

Even 20 years ago the Internet was obviously not going to be able go on as before where you could make major changes over a long weekend during the summer at the few core university's that where the the internet.

That is why ipv6 should have been replaced.

<comment><USER26> Replaced with what? IPv6 has problems, sure, but the main problem is that IPv4 and IPv6 aren't compatible with each other. You're going to have that problem with any other replacement for IPv4 as well.

<comment><USER24> Because they designed ipv6 as a alternative to the IPv4 address space, rather than an extension to the IPv4 address space.

</comment></comment></comment></comment></comment><comment><USER27> The problem is the majority of the companies who are stalling the IPv6 upgrade are in the US; which as chimeracoder stated is not going to feel the crunch as bad as other countries. People are very short sighted for one; and for two are afraid of 'breaking' what works.

I have even setup organizations with native IPv6 addresses (no tunnel) to watch them fear and lament it.

There's a thousand excuses and people need to look upon this as an opportunity; to up their skill set and mentor a new generation.

</comment><comment><USER28> Here's a report you can see the current projects with a bit of history: [http://www.potaroo.net/tools/ipv4/index.html](http://www.potaroo.net/tools/ipv4/index.html)

The potaroo site by Geoff Huston has been running for over a decade tracking address consumption.

Some history for ARIN consumption predictions: Feb 2014 predicts Mar 2015.

Oct 2013 predicts Jan 2015 [0].

Apr 2013 predicts Apr 2014 [1].

Nov 2012 predicts Sept 2013 [2].

Sep 2012 - RIPE out of addresses.

Apr 2011 - APNIC out of addresses.

Feb 2011 - IANA out of addresses.

Dec 2011 predicts July 2013 [3].

July 2011 predicts Nov 2013 [4].

Prior to this it's simply about IANA calculations, though with some algebra some dates could be extracted.

As well, here's a Cisco article from 2005 describing some of the painful parts of trying to predict the address consumption (where they guess 2016 in 2005): [http://www.cisco.com/web/about/ac123/ac147/archived_issues/i...](http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_8-3/ipv4.html)

[0] [http://web.archive.org/web/20111227105916/http://www.potaroo...](http://web.archive.org/web/20111227105916/http://www.potaroo.net/tools/ipv4/index.html) [1] [http://web.archive.org/web/20111227105916/http://www.potaroo...](http://web.archive.org/web/20111227105916/http://www.potaroo.net/tools/ipv4/index.html) [2] [http://web.archive.org/web/20121122120407/http://www.potaroo...](http://web.archive.org/web/20121122120407/http://www.potaroo.net/tools/ipv4/index.html) [3] [http://web.archive.org/web/20111227105916/http://www.potaroo...](http://web.archive.org/web/20111227105916/http://www.potaroo.net/tools/ipv4/index.html) [4] [http://web.archive.org/web/20110709090704/http://www.potaroo...](http://web.archive.org/web/20110709090704/http://www.potaroo.net/tools/ipv4/index.html)

<comment><USER15> Trying to plot this it means that the crunch will probably really hit increasingly hard from about 2015 through 2016. You go from predicting two years out to one year out (two years later).

These aren't brick walls but the shortage is already being felt. I know that at Efficito, one of the reasons we switched hosting providers was that we couldn't get ipv6 connections working flawlessly to our backup connections before (meaning more ipv4 space required). I don't think we will ever hit exhaustion per se.

</comment></comment><comment><USER29> Truth is NAT works just fine for the vast majority of cases, and makes a layered (IE not-eggs-all-in-one-basket) approach to security much simpler.

The real problem is routing table size with BGP. As we continue to divide the internet into smaller routable blocks, this is requiring an exponential amount of memory in BGP routers. Currently, the global BGP table requires around 256mb of RAM. IPv6 makes this problem 4 times worse.

IPv6 is a failure, we don't actually _need_ everything to have a publicly routable address. There were only two real problems with IPv4: wasted space on legacy headers nobody uses, and NAT traversal. IETF thumbed their noses as NAT (not-invented-here syndrome) and instead of solving real problems using a pave-the-cowpaths-approach, they opted to design something that nobody has a real use for.

Anyway, I'm hoping a set of brilliant engineers comes forward to invent IPv5, where we still use 32 bit public address to be backward compatible with today's routing equipment, but uses some brilliant hack re-using unused IPv4 headers to allow direct address through a NAT.

Flame away.

<comment><USER28> Not a flame - your perspective is very typical for people that don't have a lot of experience with networking past the host or server level. (Very little experience with networking in the core, provider, or putting together network services architecture).

1\. In theory the routing table with IPv6 can be smaller. The address design should be hierarchical, which means you should be able to have much fewer routes. It's too early to tell if this is actually true or not, but the addresses themselves are 4x larger - which isn't going to be the determining factor in routing table size.

2\. Not everything needs to be publically routable, true. IPv6 has the idea of link local and autonomous system local addressing which IPv4 doesn't have. The RFC 1918 block was used instead. But think for a second - there's only 4 billion addresses (less when you count bogons and multicast ranges), and it's only a matter of time until those are taken up. So we can choose to do it now, 2 years from now, or 5 years from now, but devices are growing faster than ever and it's only a function of time.

3\. NAT is not a security feature, is not good for the internet, and the sunk costs spent building an ALG for every protocol to work around it is a significant development sinkhole. It's a workaround often masqueraded as security, and does cause many application problems. It's just not normally the application developers that have to fix those problems - it's the network and security teams.

4\. IPv6 was created in the late 90's. People have been waiting for brilliance to supercede IPv6 for a while. I'll admit it's not the easiest, but there are a certain set of problems you have when you expand the address space.

5\. I'm familiar with all the IPv4 headers, and nearly all of them are used. ID is used for packet identification, particularly through network services, DSCP is used heavily, DF and other flags are used - they're just obscure. If you look at IPv6 those same headers are basically recreated, though with slightly different names. The ones that aren't included are addressable through the extension headers.

So, yeah. That's another perspective that may help you understand why IPv6 is a bit of a quagmire. The faster people understand this, the sooner we get to a place where the chicken-egg problem fades away.

<comment><USER30> I only care about one point. That NAT is not a security feature.

The original reason that I began using NAT was so that my ISP couldn't charge me per device. You just plugged in a NAT enabled router, and ran everything behind it. That became so ubiquitous that ISPs gave up on trying.

My concern about IPv6 is that ISPs will want to go back to charging per device. I didn't like that then, and I don't want it now.

<comment><USER19> From a host perspective it's a great security feature. you have a local address means your host cannot be contacted from the outside world. You want your host to have an IPv6 address, VPN into an IPv6 provider.

The fact that demand for this is so low, just goes to show it's not needed at the moment.

In fact, I can't think of a single reason "why" IPv6 would be needed.

I definately don't want all my devices to have a web reachable address, far from it, total security nightmare.

one entry point - a VPN on IPv4 is just great thanks, secure and easy to manage. want to access my other devices, jump on the VPN.

I that sense, you can describe IPv6 security, as configuring your VPN with no password and letting anyone connect to it.

The other way to look at it, is the successsor to IPv4 is called tor.

<comment><USER26> > one entry point - a VPN on IPv4 is just great thanks, secure and easy to > manage. want to access my other devices, jump on the VPN.

There are 4 billion IPv4 addresses and 7 billion people on the planet. Before we even get into business use of IPv4 for servers and such we don't have enough addresses to do what you want.

</comment></comment></comment></comment></comment><comment><USER31> More than likely the people in charge will not act pre-emptively by upgrading to IPv6 during the normal upgrade/replacement cycle of their network hardware. Instead, they will wait until there's a real crisis so they can ask the government to fund their next hardware upgrade.

</comment><comment><USER32> Not a comment on the article, but IPv6 adoption relies on significant upgrades of existing hardware. Think of the size of the lookup table that a new bit of hardware has to be able to look up against and store compared to IPv4. Significantly more processing power is required, particularly if the hardware is a device that does inspection of some sort, even if basic! It isn't just a case of switching end machines to use IPv6.

<comment><USER33> IPv6 makes the global routing table much smaller (because there's enough space for the subnets to be logical parts of the network, rather than squeezing as many addresses in as possible), so the core routers for the IPv6 internet ought to need less hardware than the IPv4 ones.

<comment><USER32> Ah thanks! Does this mean that the end routers can be simpler too, or do they still need significant processing power?

<comment><USER34> One big advantage of IPv6 is that routers will not have to deal with fragmentation. At higher debits, it should make a pretty important difference. [http://en.wikipedia.org/wiki/IPv6_packet#Fragmentation](http://en.wikipedia.org/wiki/IPv6_packet#Fragmentation)

IPv6 also deals better with packet headers, so it's easier to calculate for routers. I suspect however this will make more of a difference for high end routers with lots of traffic.

As others have mentioned, the dual-stack transition does require more resources. Then again, you can implement NAT64 (or 464xlat), which seem to become more realistic/robust/more widespread solutions (running dual-stack also means managing two layers of routes, two layers of firewalls, etc).

(Disclaimer: this is not my day job, I just find it lots of fun to play with on my local network and the Montreal local city mesh.)

</comment></comment></comment></comment><comment><USER35> Doesn't the US defence department hold a ridiculous amount of the address space? What would it take for them to give some of that up?

<comment><USER33> The growth in internet-connected devices is exponential. By the time the last blocks were issued they were going out at 1 class A (i.e. 1/256 of the entire IPv4 address space) every _month_. So it's not worth the effort to recover existing addresses.

<comment><USER36> iirc they were using more than a /8 every month at the end of apnics lot (also cidr is a more useful notation)

</comment></comment></comment><comment><USER37> A peak into the future of peak-oil. Let's see how this plays out and learn from it.

<comment><USER38> If only the available IPv4 address space were growing as rapidly as the available oil...

1960s: Peak oil 1995 @12.5 billion barrels/year.

Today: Peak oil 2035, current production is over 2.5X the previously predicted peak.

There's also an increasing expectation that rather than "peak oil" being a supply-side constraint, it will be a demand-side constraint as more efficient use and alternative energy sources become available - that is, we will never "run out".

[http://www.sciencedaily.com/releases/2013/07/130710114436.ht...](http://www.sciencedaily.com/releases/2013/07/130710114436.htm)

<comment><USER12> peak oil is really about EROI; we'll never run out, but there are reservoirs that aren't going to be produced because it doesn't make economic/energetic sense to do so.

</comment></comment></comment><comment><USER39> still wonder how much of the internet is not IPV6 compatible.

<comment><USER15> Quite a bit actually. Our experience at Efficito may be interesting.

We previously had backups located in Denver and production servers located in Europe, both through the same hosting provider (FDC). We never could get IPv6 working flawlessly (high packet loss, etc) intercontinentally and the problems weren't on our side.

We moved to Hetzner and have our systems spread throughout their datacenters (backups, and some backend systems on one side, customer data on the other) and have had absolutely no problems with ipv6.

What this tells me is that somewhere between the Czech Republic and Colorado there are routers which although they sort of support ipv6 don't do so in a usable way.

<comment><USER40> Were they using HE as an ISP? I ask because HE makes a big deal of IPv6, and we regularly get packet loss between LA and Denver on IPv4. Way overloaded. FDC says they use Cogent -- Cogent used to be not-so-great (they're better nowadays).

<comment><USER15> We didn't look into it too closely. mtr would have told us but we didn't go into it. They did tell us that if we upgraded our servers and switched data centers, we would get better peering. But we opted at that point to move off and go with Hetzner instead because we weren't at a point at the time when their higher end servers would have made sense.

</comment></comment></comment></comment><comment><USER41> nat</comment>


The sound of mspaint.exe when interpreted as PCM audio data - <AUTHOR> http://soundcloud.com/r2bl3nd/windows-7-x64-ms-paint-exe <comment><USER2> Back in the days when dinosaurs roamed the earth, my roommate was working on a commercial app that was a music composition and sequencing tool. One night I heard some unusual music coming from his room, and a few moments later he ran out all excited. It turns out that he'd had a bug in his code, and the music we'd been hearing was the program playing itself: interpreting its own code as music data.

What was surprising, though, was how coherent it actually sounded as music. There was, for example, a perceptible beat. We always wondered whether this was due to the structure of the executable code, or because of the way the program interpreted its data. Unfortunately he was never able to reproduce the bug in quite the same way.

[1] "RealTime" for the Atari ST, see <http://tamw.atari- users.net/realtime.htm>

<comment><USER3> I had a really, really odd experience when I was very young -- probably no older than 6 or so. I was on a 386 running MS DOS 5 and at the time, I was hooked on a game called "Pilgrim's Quest". I was also in the habit of breaking edit.com out on everything I could get my fingers on, and binaries were no exception. I had no idea what I was doing (otherwise I would've been using debug.com), but I would delete things, type random characters, etc. One day, I took edit.com to the binary for Pilgrim's Quest, and proceeded to mess with it, rendering it useless. Except, it wasn't useless. When I ran it, the display was messed up, but pressing keys caused notes to play. To this day, I can't figure out why this would've happened -- I know of no functionality in the game that played single notes (although, to be fair, I don't remember if it played a note and then stopped or continued playing it indefinitely -- this was a long time ago), and I can't see how one could stumble upon that.

I really wish I knew what the hell I did, as that was the first time I ever made the computer do something it wasn't intended to do, even if it was not my intention.

</comment></comment><comment><USER4> I used to listen to RAM dumps from an HDTV tuner card I was reverse engineering to try to find its audio buffers. Naturally the image buffers always sounded interesting. I eventually wrote a rudimentary ALSA driver for it.

You can get sounds that sound like an alien machine by taking some super-noisy image or executable data and slowing it down several times using a high- quality resampling algorithm.

<comment><USER5> > I used to listen to RAM dumps from an HDTV tuner card

This is pure gold.

</comment></comment><comment><USER6> This is bizarre. It sounds ridiculously close to actual records that I own and have played out.

For example:

<http://www.youtube.com/watch?v=InUiVFLTGdo>

<http://www.youtube.com/watch?v=tjBOqPDXEog>

Fun fact: the second one was entirely made on a gameboy.

Expect mspaint.exe to have been remixed and to have found its way onto london's pirate airwaves by the end of the week.

</comment><comment><USER7> I'm reminded of one of my favourite pieces of "music", the Symphony For Dot Matrix Printers. Basically, a couple Canadians acquired a crop of old printers, wired them up to some equally old computers, and figured out how to feed them all sorts of interesting printing instructions. Listening very carefully to the resulting sounds, they figured out how _exactly_ to place microphones to capture all the fascinating sonic nuances, organizing the printers and their noises into various instrumental roles. They composed a surprisingly musical work for the printers to play, as though they were an orchestra, which was performed live and recorded. The recording is rather difficult to track down, but is quite a treat for enthusiasts of real "computer music".

They sound like printers. But they sound like printers, re-imagined by people who hear music in everything.

For anyone interested, here is their very, very old website: <http://www.theuser.org/dotmatrix/en/intro.html>

There was a symphony #2, but the first is the better of the two in my opinion.

</comment><comment><USER3> This reminded me of an interesting bit of research by Inigo Quilez into using sounds from gm.dls (the Windows General MIDI sample et) to generate textures: <http://www.iquilezles.org/www/articles/gmdlsgfx/gmdlsgfx.htm>

Pretty interesting what you can get out of it.

</comment><comment><USER8> I actually stumbled across this while experimenting with libsndfile(www.mega- nerd.com) I built a .NET wrapper around it for converting telephony .vox files to a more user friendly format. In the process of beta testing I found you can feed any bits you want to it and it will spit out a "wav" or other of its supported formats to be played by any media player. I guess I will have to revisit that project for musical inspiration...

</comment><comment><USER9> I remember being able to distinguish bitmaps in Apple II games saved to tape (floppy disk drives were too expensive for me at that time). This brings me back memories.

There was also a radio program that transmitted audio on one channel and computer programs on the other one, broadcast from the local university radio on, IIRC, Saturday afternoons. Those programs were for MSXs, so I never loaded them.

</comment><comment><USER10> It sounds like there is a lot of repetition in the EXE file.

<comment><USER11> I was surprised by that. I wonder what portions are actual machine code, and what parts are embedded data blobs ("resources" in Windows terminology). I also can't explain the crescendo at the start.

<comment><AUTHOR> I'm pretty sure that is a fade-in added by the uploader.

</comment></comment></comment><comment><USER12> Another interesting thing to do is play back image data as audio. I used to use this for wild sound effects in tracks I was making about 15 years ago. Never did get to the point where I could play around in photoshop to deliberately create sounds, but I suppose one could if you made a study of it.

</comment><comment><USER13> Anyone care to suggest the reason behind the four notes that are repeated over and over (occasionally with slightly different filters)?

<comment><USER14> I assumed they were UI elements. I haven't actually seen MS Paint in ten or more years, but I'm guessing it has many widgets that are square buttons with something inside, so the image data would appear as repeating patterns of identical length and have at least somewhat similar content (because everything is in a bordered box, and there's probably an "up" and "depressed" version of each, leading to nearly identical sounds being side-by-side in the data).

That's just a guess though. One would probably need to disassemble it and roam around a bit to be sure.

</comment></comment><comment><USER15> A whole new genre of music concrete[1] has just been born.

<http://en.wikipedia.org/wiki/Musique_concr%C3%A8te>

</comment><comment><USER16> Now where's the music video where this is synced to the same data interpreted as 100x100px frames of RGB image data?

</comment><comment><USER17> Used to do this "back in the day" (circa 2k2) for getting nice audio glitches back when I was producing significantly noisier music.

<http://sickmode.org/scrap/mercykilling.mp3>

perhaps it will inspire a new generation of noisemakers ;)

<comment><USER18> Woho, this is great!

Do you got any more of this stuff lying around? Dont have the time to go trough all the stuff in scrap.

<comment><USER17> Thanks for the feedback!

The only other similar-ish thing I have lying around from that era would be <http://sickmode.org/scrap/organ_grinder.mp3>

I've mellowed out quite a bit since then ;)

</comment></comment></comment><comment><USER19> Funny. I used to do this all the time in Fasttracker2. Most of the time you can use noise like this as high-hat. But sometimes you discover 'gem' that just sound great without tweaking.

</comment><comment><USER20> The "tune" on soundcloud.com is mspaint.exe as 8bit signed PCM, but modified - try importing it in f.e. Audacity and you will notice that quite some editing (fade ins/outs, time stretching etc.) has been applied to the soundcloud.com version.

<comment><AUTHOR> The creator states:

 _This works in Audacity, and probably any other audio program that can import raw PCM data. I imported it as 22050hz 8-bit stereo audio in Adobe Audition, but it seemed to sound mostly the same in Audacity._ _All I did to the audio was master it slightly to make it sound less harsh to the ears, as well as remove a long section of noise._

Did you import it with a 22050hz sample rate?

</comment></comment><comment><USER21> I remember <http://dumpanalysis.org> does something like this.

</comment><comment><USER22> It sounds interesting, but what amazed me most is soundcloud working on iPhone.

</comment><comment><USER23> Apparently, Microsoft is better at music than software :D

</comment><comment><USER24> If you play it backwards, it says Paul is dead...

</comment><comment><USER25> Microsoft Underground Chiptune Resistance!

</comment><comment><USER26> Sooo geeky :)</comment>


Ideal Progamming Fonts - <AUTHOR> http://www.codinghorror.com/blog/2004/12/progamming-fonts.html

<comment><USER2> Inconsolata isn't mentioned but is great too.

</comment><comment><USER3> I've taken the liberty of removing the inconsequential from that article, but it seems to have left me with only the word 'monospaced'.</comment>


Cellphones, Though Banned, Thrive in U.S. Prisons - <AUTHOR> http://www.nytimes.com/2011/01/03/us/03prisoners.html?src=twr <comment><USER2> One of the factors that created this market is the unreasonable phone systems currently in place at correctional facilities. They are excessively overpriced and that cost is largely passed on to friends/family on the outside. In the long run it's actually cheaper to buy one of these expensive contraband cell phones and add money to them as needed.

Some people complain that inmates shouldn't be allowed to call anyone, but it's by maintaining these social connections to the outside world that gives them a greater chance of rehabilitation. You keep chipping away at their threads of humanity and you will create some very broken men; who will eventually share that street with you one day.

<comment><USER3> Prisons do often leave prisoners more likely to commit crime than they were before they were incarcerated (there was a great article on The Atlantic last year on how ankle monitors are changing this, by keeping many non-violent prisoners from ever entering prison).

However, allowing cellphone usage inside of a prison allows many convicts to run criminal networks from inside of prison walls. This makes it difficult for law enforcement to make a dent in organized crime, as incarcerating a gang's leaders doesn't prevent them from organizing further criminal activity.

Non-violent criminals could arguably be given cell phones, but the potential for violent criminals to organize violent crime means that allowing violent criminals to use cell phones in prison is a dangerous proposition.

<comment><USER4> It doesn't necessarily have to be cellphones. The reason they are so popular in prison is because of the outlandish fees to make a collect call, and in some cases, those phones are monitored. If you're choices are a collect call with $5 connection fee and $2/min and a $300 cellphone loaded with hours of calltime than it doesn't take a genius to see that the cellphone is going to win whether its for the mob boss or incarcerated shoplifter.

A sensible middle ground would be to make collect calls affordable, or even free from a selected list, so that cost works against smuggling in cellphones. Until then, people are going to find more elaborate ways to reach outside. And they've got lots of time on their hands to do it.

</comment></comment></comment><comment><USER5> _The solution, Mr. Ozmint and others say, is to simply jam cellphone signals in prisons._

That may solve that one problem, however the article glosses over a much larger problem -- the inmate bought his phone from a prison guard. There is evidently corruption inside the system and as a result there's no telling what else that guard (or guards) are transporting into the prison for inmates.

<comment><USER6> I heard it said once that "You can get anything on the inside that you can get on the outside -- except a woman". (This was in an article/interview about rampant drug use inside prisons.)

</comment></comment><comment><USER7> In a sane prison system, they'd probably support having inmates playing Farmville rather than other things they could be doing. Sure, they _could_ theoretically be doing drug deals or other criminal enterprises, but the Gotti-types already do that type of thing, and 99% of people in jail just aren't important enough for that.

But politicians get elected and "prison contractors" get rich by making prison conditions worse and less effective.

<comment><USER8> If prison conditions are too good then prison becomes less of a punishment. Not to start a ideological argument here but there is a case to be made for prison to be unattractive, especially for people who would like to play farmville all day.

</comment></comment><comment><USER9> _"It is impossible to have enough staff to watch the two million people we have locked up in the country at this time," he said. "In a perfect world, yes, we would find all the phones. But this isn't a perfect world."_

Yes, but wouldn't these cellphones be perfect targets for DoD developed Electronic intelligence equipment? We should be able to localize cellphones within prison and kill access from the phone's MAC address. Better yet, the authorities could get a warrant (since the cellphone within prison boundaries is a Federal offense now) and start monitoring the communications.

I'd rather the Homeland Security folks would spend multi-millions on such a program instead of spending it on Rapiscan.

<comment><USER10> re: "We should be able to localize cellphones within prison and kill access from the phone's MAC address."

If you read through to the end of the article, you'll learn that Mississippi has implemented an interception system that achieves that kind of result.

</comment></comment><comment><USER11> _"Almost everybody has a phone"_ (from prisoner in article)

If that is true, then the guards are participating either actively in giving over the phones or passively by ignoring the phones.

There are handguns as small as a cellphone and you don't see "everyone has a handgun" in prison. So work that one backwards.

Guards must think there is no harm so they ignore, or they are paid off somehow.

Fire all the guards, get new guards or switch them from another prison to get rid of their payoff familiars.

<comment><USER12> Guards won't give prisoners guns because that would present a real and direct threat to themselves. Is the difference between prisoners having guns and prisoners having access to telephones really that blurred to you?

<comment><USER11> Hence my point, it's the guards participating, they are breaking laws and should be prosecuted.

Put a few guards behind bars and maybe the others will stop allowing cellphones?

You'd think seeing how horrible prison is would encourage the guards to follow the law themselves.

<comment><USER13> > Put a few guards behind bars and maybe the others will stop allowing > cellphones? > > You'd think seeing how horrible prison is would encourage > the guards to follow the law themselves.

It didn't work for the prisoners (many of whom are return customers), why would it work for the guards?

</comment></comment></comment></comment><comment><USER14> If they can't keep phones away from prisoners--a population that has been legally stripped of many of the freedoms and protections that we on the outside enjoy, how can they expect us to believe that they can keep bombs off of airplanes?

The prison population is less than the number of people that fly in two days, so the TSA has a much bigger population to deal with, and the TSA doesn't have 24/7 access to flyers, nor the right or ability to control who they talk to, and so on.

</comment><comment><USER3> This great documentary by Vanguard makes it clear that a lot of cellphone smuggling is actually done by visitors and prisoners using body cavities: <http://www.hulu.com/watch/109763/vanguard-prison-contraband>

It's not just guard bribes that are the problem... as long as the incentive exists to smuggle items in, people will do so (both guards and visitors). Managed Access (as in Mississippi) would kill that incentive, which would eliminate the smuggling (of cell phones, at least).

</comment><comment><USER15> If everyone agrees that cell phones in prisons are a bad thing, when why not use jammers to stop the signal? A small cell phone jammer can be gotten for ~$130, and it probably wouldn't take many to blanket a prison. This would lead to issues with communication between the inside and outside, of course, but a mixture of land lines and VOIP would make it pretty easy for guards to continue as normal.

<comment><USER7> From the article, at least some people think it won't work: "It's illegal, plain and simple," said Chris Guttman-McCabe, vice president of regulatory affairs for CTIA-The Wireless Association. He cited the Communications Act of 1934, which prohibits the blocking of radio signals — or, in this case, cellphone signals — from authorized users.

<comment><USER16> The problem with this argument is that prisons are a different place legally than the rest of the world. In addition to the the limited rights of the prisoners, they also impose limited rights on the people working there. If the prison authorities and whatever legislature has authority over the prison decide that there are no "authorized users" within a prison then there is no violation of the act.

</comment></comment></comment><comment><USER17> How do they charge them?

<comment><USER18> They've probably got the self-charging models.

</comment></comment><comment><USER19> There are too many people in US Prisons, that's why they can't keep tabs on everyone.

</comment><comment><USER20> I think allowing all inmates to have cell phones and having a system to monitor all of their calls to stay in contact with those who can help them re- integrate with society is a terrific idea.

However it will never work because the smartest criminals will use it to escape, terrorize with gangs out the outside, coordinate drug inflow and all sorts of other problems. The prison barely has enough staff to keep the place under control.</comment>


Ask HN: Is anyone else not able to connect to github? - <AUTHOR>

At 10:46 I get (7) couldn't connect to host if I run curl. Browser is no good. Tried on my cell phone and couldn't connect either.<p>The status page (https://status.github.com/) says everything is fine. <comment><USER2> Yes, I'm unable to connect. According to a quick Twitter search it appears it's down for everyone.</comment>


Why Using Bell Curve in Performance Evaluation Is Bad for Your Organization - <AUTHOR> http://www.anwarbosbool.com/2016/02/why-using-bell-curve-in-performance-evaluation-is-bad-for-your-organization/ <comment><USER2> This post does not explain why using a bell curve in employee performance evaluation is "bad" Just what statistics to use when analyzing performance requires thought, a model, and a knowledge of statistics.</comment>


Show HN: Material Bread – A React Native Material 2.0 Component Library - <AUTHOR> https://material-bread.org/ <comment><USER2> Love it when projects give a bunch of examples and all live too. The last time I looked into an RN library you basically had to test all the components yourself to get an idea of what they looked like and how they worked.</comment>


A secret tunnel that runs underneath the phone companies and emerges in paradise - <AUTHOR> https://boingboing.net/2016/09/22/i-have-found-a-secret-tunnel-t.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+boingboing%2FiBag+%28Boing+Boing%29 <comment><USER2> It's a shame that this title wasn't more descriptive. Here is the gist, which may interest many more HN readers than the current 17 point rating (buried on page 3) suggests:

"Calyx is a famous, heroic, radical ISP that has been involved in groundbreaking litigation -- they were the first company to ever get a secret Patriot Act warrant unsealed, fighting for 11 years to overturn the gag order. ... For $500, Calyx will send you a little wifi hotspot with a Sprint SIM in it that comes with a year's worth unlimited, anonymous, unshaped, unfiltered 4G/LTE bandwidth on Sprint's network. Unlimited as in, I downloaded 60GB with mine and it didn't break a sweat. ... Subsequent years are $400 (because you don't get another wifi gadget, just the service)."

EDIT: Also some good info in Calyx's "Frequently Asked Questions about our 4G / LTE Membership Benefit": [https://www.calyxinstitute.org/member/4G-LTE- FAQ](https://www.calyxinstitute.org/member/4G-LTE-FAQ)

</comment><comment><USER3> Sharing this further than BoingBoing may cause an early termination of the program if the influx of users is too high.

I had bought into a similar "unlimited" 4G/LTE hotspot program that used Verizon's network about a year ago [0], then news about it spread like wildfire and it only lasted less than two months after that. The wording in the Terms of Service of that program is very similar to this program.

I probably won't buy into this program, but for the sake of others, please don't let this go "viral".

Hopefully Calyx limits the number of people who can get this so only X number can join per month as to not attract scrutiny from Sprint.

[0] [https://www.rvmobileinternet.com/millenicom-reborn- unlimited...](https://www.rvmobileinternet.com/millenicom-reborn-unlimited- verizon-data-plans-for-47-75month-from-omnilynx-wireless/)

</comment><comment><USER4> Has anyone tried this? Sounds great. Fan of Calyx, would be happy to give them money if they have good reviews.

</comment><comment><USER5> This should be higher on the list.

</comment><comment><USER6> Only in the US though.</comment>


TikTok set to become a standalone US company to satisfy White House - <AUTHOR> https://www.ft.com/content/58eb7c26-2154-477f-af19-19157ae29261 <comment><AUTHOR> [https://archive.is/DFrZV](https://archive.is/DFrZV)</comment>


Evernote Tops Ten Million Users « Evernote Blogcast - <AUTHOR> http://blog.evernote.com/2011/06/06/evernote-tops-ten-million-users/ <comment><USER2> Wow, they're doing very well. As a former delicious user who changed over permanently, and to a premium account, this past December, I wonder how much their growth was affected by Yahoo!'s claim of sunsetting delicious.</comment>


Securing access to Wikimedia sites with HTTPS by default - <AUTHOR> https://blog.wikimedia.org/2015/06/12/securing-wikimedia-sites-with-https/ <comment><USER2> User generated content sites are some of the most important that need to implement HTTPS.

Consider reddit for example. The entire site is HTTP. That means your ISP can pull up a list of your entire reddit browsing history. They can see every post you read, every comment you wrote, etc. Collectively this data could form quite an accurate picture of a person's mental makeup. Is that the kind of data you want floating around?

HTTPS encrypts query strings so upstream providers cannot see which specific pages you browse.

Of course, HTTPS still shows domain names... but it's a start. Next step is DNSSEC for everybody!

p.s. Anyone interested going down the rabbithole exploring mass tracking of individuals, should google "entity mapping llc" and poke around the results...

<comment><USER3> [https://www.reddit.com/prefs/security/](https://www.reddit.com/prefs/security/)

You can force it to https only for your user account.

<comment><USER4> Thanks for that tip. I just enabled it for my account. I wonder why they don't redirect to https by default.

<comment><USER5> from what I remember reading is that when they first started with HTTPS, they weren't able to easily deal with the added overhead. They switched to cloudflare and are working on rolling out HTTPS across the site. One of the changes involves removing the reddit bar.

</comment></comment></comment></comment><comment><USER6> This is a great move and has inspired me to think of how I can do this for the sites I run.

The problem I have is that I run forums which accept user generated content, and the links in the content are then parsed and embeds are put in.

For example, YouTube links have the embedded video put below to the link.

YouTube support https, but a lot of smaller sites that offer really useful tools don't yet support https. An example is something like [http://www.bikely.com](http://www.bikely.com) which does not support https at all, and yet the embedded maps are common enough on the cycling forums I run.

I cannot just proxy via a https domain as these sites usually have JavaScript that require permission to talk to their own domains.

I've decided to start emailing all of the 3rd parties to ask that they add https to their site.

Almost 40% of my traffic is now over https, and it's harder to increase that as every time I try to I receive support complaints about mixed-content warnings, missing content, etc.

If you also run a site that has embeds in user generated content, please consider emailing the 3rd parties and explaining why they should move to https.

There are only a few things holding back a lot of sites and it's no longer the cost of a cert and SSL termination:

1) Embedded widgets that are http only today

2) Advert widgets/scripts that are http only today

Those things hold the vast majority of news sites back too. I even checked the Guardian yesterday and no https available. The only news site that was https was The Verge, but I'm not sure tech news is really news.

<comment><USER7> Exactly. The problem is that News sites rely on advertising.

Due to the general low technical quality of adv companies/companies' suppliers/customers etc you will find often find an image/flash/js served over plain HTTP or with a bad certificate.

That single HTTP bit will immediately trigger the terroristic browser behaviours we learned to accept and love, including but not limited to: red signs, technical jargon pop ups, fully red pages with jargon disclaimers of all sorts and "getme outta here" buttons.

Funnily enough, I remember reading a technical blog post written by Guardian devs themselves, citing this very problem.

<comment><USER8> The browser isn't exactly wrong for triggering those 'terroristic' behaviors. If there is one single http bit being included and rendered on the https web page then the entire security of the page is rendered useless if there is a MITM attacker between you and the source. That single http bit can include whatever the attacker needs in javascript to redesign the page to steal content and provide tampered content.

</comment></comment></comment><comment><USER9> I've had a few conversations with other advocacy nonprofits who are not using https, even when they know they are being targeted for surveillance. It seems the biggest reason they don't switch is because their IT staff would bitch about the work and they don't want to sour the relationship between management and IT. Typical underfunded IT bureaucracy and politics.

What I've started to do is offer to play bad cop, and come in as a consultant and take the heat from IT when I propose the work to switch to https. That way, IT bitches about the consultant and not management.

<comment><USER10> Very cool. Well done you. How do we scale you? :)

</comment></comment><comment><USER11> On Wikipedia, one of the issues is that a lot of the behind-the-scenes JavaScript gadgets (for routine administration/maintenance tasks) are written by community members.

When HTTPS was first rolled out on Wikipedia (before that we had the joy that was a single secure server for all language sites - secure.wikimedia.org), a lot of these scripts wouldn't work because they hard-coded protocols and so on. Fixing all those edge cases was quite difficult. The scripts are often abandonware but essential for admins to do their jobs.

One of the downsides of community projects like Wikipedia...

</comment><comment><USER12> Not perfect, but wikimedia scores a respectable A on ssllabs:

[https://dev.ssllabs.com/ssltest/analyze.html?d=wikimedia.org...](https://dev.ssllabs.com/ssltest/analyze.html?d=wikimedia.org&s=188.8.131.52)

Same IP as en.wikipedia.org, so does that mean they're using SNI to serve different certs? [http://caniuse.com/#search=sni](http://caniuse.com/#search=sni)

<comment><USER13> Nope, it's a wildcard cert -- if you view the cert contents you see that the subject includes

CN = *.wikipedia.org

<comment><USER14> Actually, it's a bit more complicated than that. We don't just serve _.wikipedia.org from our frontends, but also e.g._.wikisource.org etc.

So yes, we are using SNI but for wildcart certs :)

</comment></comment></comment><comment><USER15> This is an interesting site: [https://httpswatch.com/global](https://httpswatch.com/global)

<comment><USER16> You should also check out the Trustworthy Internet project report on SSL/TLS deployment quality.[0] The big increase in B and C grades this month (compared to last month) is due to Qualys's SSL Labs tester becoming more strict about DH parameter size (less than 2048 bits caps your grade at B; see also Logjam), lack of TLS 1.2 support (caps your grade at C), and RC4 deployment with TLS 1.1 and 1.2 (caps your grade at C, RC4 deployed for TLS 1.0 still only caps you at B).[1]

[0] [https://www.trustworthyinternet.org/ssl- pulse/](https://www.trustworthyinternet.org/ssl-pulse/) [1] [https://community.qualys.com/blogs/securitylabs/2015/05/21/s...](https://community.qualys.com/blogs/securitylabs/2015/05/21/ssl- labs-117-obsolete-crypto-rc4-and-logjam)

</comment></comment><comment><USER17> This seems good, but too limited. Why not make it so that wikipedia can only be served over HTTPS?

I remember when the UK censored wikipedia because one page had an image they considered child porn and they transparently redirected the domain to point to a proxy. If they only serve it over HTTP this should hopefully break such a scheme - which isn't the perfect solution (that would be the very painful death of those involved with the censorship), but at least it makes it clear what is happening.

<comment><USER18> > This seems good, but too limited. Why not make it so that wikipedia can only > be served over HTTPS?

That's what they're doing. From the second paragraph: "We will also use HTTP Strict Transport Security (HSTS)"

<comment><USER19>
      * HSTS requires HTTPS connections
      * Internet.org (currently) requires HTTP only [1]
      * But Wikipedia is a member of Internet.org [2]

I'm curious how Wikipedia will handle this. Perhaps allowing Facebook to MITM?

Or maybe the security mentioned in the original article isn't intended for those who can't afford it[3].

      [1] https://developers.facebook.com/docs/internet-org/platform-technical-guidelines
      [2] https://internet.org/press/introducing-the-internet-dot-org-app
      [3] https://en.wikipedia.org/wiki/Wikipedia_Zero

<comment><USER20> HSTS only works after you connect over HTTPS. If you can only access a site over HTTP, HSTS doesn't do anything. So maybe people hitting WP from Internet.org can never get it over HTTPS in the first place?

</comment></comment></comment></comment><comment><USER21> This is all great, and I can see the benefit of having HTTPS available for all the sites.

That being said, various documentation has started serving docs on HTTPS only which means I can not access it from work.

There are a lot of scenarios where having HTTPS will just impede people doing their work.

And finally, how are we to trust that for example Version or Thawte are not influenced by the likes of NSA and make possible for them to decript our traffic with ease?

<comment><USER22> >how are we to trust that for example Version or Thawte are not influenced by the likes of NSA and make possible for them to decript our traffic with ease

Abandon all hope that HTTPS will safeguard you from the NSA or any major foreign intelligence agency.

<comment><USER21> And with that I absolutely agree.

So what is the level of paranoia that SSL is useful for? Since this is what the article says:

> Encryption makes it more difficult for governments and other third parties > to monitor your traffic. It also makes it harder for Internet Service > Providers (ISPs) to censor access to specific Wikipedia articles and other > information.

And we agree it doesn't really help with government surveillance?

Do ISPs randomly censor access? Or do they again do it on government requests. Cos if government finds that your site needs censorship why would they not just block the whole site? Another thing that is harder for ISPs to do with SSL is caching.

Maybe I'm not brightest child on the block so I'm still struggling to figure out what is a benefit of having HTTPS everywhere.

And having the likse of Google punishing non SSL sites just makes this fad worse. I don't need SSL on StackOverflow, Django or Python documentation. Does anyone?

<comment><USER23> Yes, I believe I do. It is no ones business but mine and the site's what I am reading or contributing.

<comment><USER21> Fair point, I am not pretending that everyone will have same requirements and oppinions. But even with SSL at least the domain is still visible. And in some cases there are ways to infer what URL you actually visited.

I also see companies using MITM successfully in a way that unless you check the cert your self it seems legit. I still use HTTPS when I go to Google but I can see the cert is spoofed.

And what about the people that don't care and are effectively prohibited from using a public data site at all since the site decided to use HTTPS only? Do way say we don't care about them? Since few years back we wanted our sites to be available to everyone, on old browser new browsers, mobiles and so on.

And having people smarter then me (like Roy Fielding) agreeing this does not do much for privacy rather content confidentiality (and actually making communication less private) is not making me any more convinced.

Bottom line, and I don't expect everyone to agree, is that I am all for using SSL even by default, but for public data I would still want to have access to it over plain HTTP.

I want/need that choice, otherwise we are hindering corporation employees and people living in the countries in which governments do massive surveillance. I think it is important for people to realise that SSL is not the ultimate solution for data integrity and specially privacy as it is often posed to be.

Thanks to all expressing your views in comments.

</comment></comment></comment></comment></comment><comment><USER24> This is good.

</comment><comment><USER25> If you go to the Turkish Wikipedia you will see that at the top there is banner. It says that some of the pages, such as vagina and human penis are blocket in Turkey. This is a great move, I mean the SSL. But Turkey will block the entire domain then. Then people will go to buy VPN from foreign countries etc. Bad for Turkey. Fuck those who are ass hairs!

<comment><USER25> I don't understand why you downvote me.</comment></comment>


On the Rise of FinTechs – Credit Scoring Using Digital Footprints - <AUTHOR> https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3163781 <comment><USER2> There was another article recently which argued that because there is a strong correlation between race and default rates, if you apply machine learning to a dataset, the algorithm will find a way to extract what is basically a proxy for the race from the data.

So basically any sort of ML applied to credit data will run afoul of the equal credit opportunity act.

The article also made the point that basically all ML credit scoring startups are illegal because of this, but they get away just because they are small and not on the radar.

<comment><USER3> I make credit scoring models for a living and I can tell you that in most countries you won't find "proxies" or any other strong variables just by applying "machine learning".

Usually when you try neural networks in this segment you end up with exactly the same variables and outcome as you would with a normal logistic regression with 10x the complications and a lot less stability of the model.

There simply is not enough input parameters that are significant to the outcome.

But it might be in the US where the field is less regulated and you are allowed to collect all kinds of information on the person that you could proxy something like ethnicity although I have not found ethnicity significant in any of my sets. We do get a some variables even if we are not allowed to use them. Again this might be different in the US.

<comment><USER2> Quote from the article:

> You can load in tons and tons of demographic data, and it's disturbing when > you see percent black in a zip code and percent Hispanic in a zip code be > more important than borrower debt-to-income ratio when you run a credit > model.

[https://thenextweb.com/syndication/2017/12/29/future- fintech...](https://thenextweb.com/syndication/2017/12/29/future-fintech- racist-according-anonymous-data-scientist/)

<comment><USER3> If you account for most other significant things like income, education, social status, job etc you will find that ethnicity is not significant.

The fact that you can sometimes use ethnicity as a proxy for social status and other things just show the discrimination that happens in some places. But when you set all other factors equal someone from Asia, Africa, Northern Europe will have the same default rate. At least in the (European) countries I've run models in.

<comment><USER4> People with the exact same FICO score have different default rates, if you manage to bin them by race: Asians > Caucasians > Latin > Black.

> We do get a some variables even if we are not allowed to use them.

Cool, so you had access to an ethnicity variable to measure its proxy power and significance? I feel this is important and very rare outside of Europe.

</comment></comment></comment></comment></comment><comment><USER5> The book Weapons of Math Destruction talks all about this. I've come to believe that pure risk shouldn't be the only factor in a person's interest rate.

That obviously makes a ton of sense from a business standpoint. You want to contain losses for risky borrowers but compete with other lenders for low risk borrowers.

But socially, this is perverse. People tend to be risky because they are already poor. So now money costs more for those who have the least of it. This is one of the feedback loops that makes poverty (and affluence, for that matter) so sticky.

I had this realization in my personal experience when I was able to refinance almost $100k in student loans at a crazy low interest rate. My household's finances are in great shape as my wife and I enter our prime earning years. But for us, such an opportunity is a gift, on top of an already sweet situation. The savings could be a game changer for a family whose finances are more marginal.

<comment><USER4> You can overdo fairness and cause troubles to the poor. Very concretely: Giving someone a loan, despite their credit score being marginal, will severely mess up their credit score forever, if they can't repay you.

If you are poor, then don't create more debt! It should be hard to rack up such a debt, not easy and accessible. No amount of credit is going to increase your social status, because you have to pay it back with your own current/near-future money.

If we want social justice for the poor through access to more money, then capitalism is not a good way to go. The state should become a credit provider.

<comment><USER6> How would someone's credit score be messed up forever? Debts are cleared completely after 7 years and have less weight after about half way through that time. Bankruptcy is along the same lines.

So your example isn't correct.

<comment><USER4> You still owe any debts after 7 years (up to 15 years in some states). Zombie debt is not "completely clear". But ok, read "mess up your credit score for 7 years" instead. Then point remains: Marginal high-risk credit underwriting is not only dangerous to the institution, but also to the receivers of the loans (and the economy in general). It is socially perverse to hook lower-income people to consumer credit. To have middle-income people lose their house.

</comment></comment></comment></comment><comment><USER7> Problem with utilizing datapoints like digital footprints is that it will run afoul equal credit opportunity act. ECOA was designed to stop banks from redlining neighbourhoods which usually punished minorities. With digital footprints, they'll be in theory redlining their digital including sites, products purchased etc.

</comment><comment><USER8> The attributes, from the paper:

The device type (for example, tablet or mobile), the operating system (for example, iOS or Android), the channel through which a customer comes to the website (for example, search engine or price comparison site), a do not track dummy equal to one if a customer uses settings that do not allow tracking device, operating system and channel information, the time of day of the purchase (for example, morning, afternoon, evening, or night), the email service provider (for example, gmail or yahoo), two pieces of information about the email address chosen by the user (includes first and/or last name and includes a number), a lower case dummy if a user consistently uses lower case when writing, and a dummy for a typing error when entering the email address.

<comment><USER5> I feel like _none_ of these things should be considered relevant to access to credit.

<comment><USER4> Let's play a game. You are in charge of a large pile of cash and want to make it grow by giving loans. Each day, two people apply, and you can give out one loan (you will have to rank the applicants). When people de-fraud you, you lose all of the loan. When people don't or can't pay you back, you lose all of the loan. When people pay back the loan, you make a little money.

Day1: User Agent: iPhone latest vs. Windows XP Day2: Referral: Facebook friend vs. search "cheapest loans" Day3: Time of interaction: 21:30 vs. 04:30 Day4: Email: firstname.lastname@example.com vs. email@example.com Day5: Funnel: Someone who spend 10 seconds vs. someone who spend 10 minutes, made a mistake in the name, entered an email address, then deleted it, and entered another email address at a different provider.

Now if your feeling does not point you to the first applicant every day, you look at the data for guidance. You find that the number of fraudsters and non- pay's is statistically significantly higher for people with the second set of characteristics.

The alternative is to use third-party data providers. That's another can of worms. Or flip a coin and start gambling proper.

<comment><USER9> That game is common in credit scoring classes for fresh analysts. The class is split into small teams. Each team is given ten anonymized, but real, credit applications and corresponding credit bureau pulls. Five from customers who subsequently defaulted, and other five from customers who paid the loan back. Each team tries to guess which are which. The team that makes the most correct guesses wins.

Then the results are compared with FICO score, and usually FICO is clearly better. Even with people with banking experience on the teams it's very rare to see humans beat the model, partly because humans tend to base their decisions on irrelevant details and their own biases.

<comment><USER4> Let's change the game. You can allocate an investment to either of two banks. When building credit scoring models, one bank has access to just FICO scores, the other bank also has access to FICO scores in addition to behavioral and signature data. Which bank do you allocate your cash to?

Now change the game so FICO is unavailable: For instance, when micro-lending to third-world country entrepreneurs. Do you still feel these digital signatures are irrelevant to making better credit risk decisions?

<comment><USER10> > Do you still feel these digital signatures are irrelevant to making better > credit risk decisions?

Yes, absent a credible theory as to why a particular characteristic could reasonably be linked to loan performance.

Otherwise, it's too likely the model could fall prey to confusing correlation with causation.

<comment><USER4> Let's say you add these digital signature variables to your credit risk scoring model anyway. The model then falls prey to confusing correlation with causation. What happens to the performance of the model?

<comment><USER10> I have no idea, as merely _adding_ them may have no effect at all.

However, depending on them exclusively (or in substantial/majority part), which I believe is the main premise, the eventual performance will depend entirely on if the the _actual_ causal relationship which created the correlation holds true. If it doesn't, the model would no longer be predictive.

[https://en.wikipedia.org/wiki/Confounding](https://en.wikipedia.org/wiki/Confounding)

</comment></comment></comment></comment></comment></comment></comment></comment><comment> yjftsjthsd-h "We analyze the information content of the digital footprint – information that people leave online simply by accessing or registering on a website – for predicting consumer default."

Wonderful. Gameable and dystopian all at the same time.

<comment> Spartan-S63 Feels like the open market is coming up with a solution similar to China's "social credit" scoring system.

<comment><USER11> The only thing stopping it in the US is the equal credit act. Most of these "novel" credit scoring solutions are just attempts to work around the race and other prohibitions in credit scoring. The good news is that these things get shut down quickly with enough complaints. This study points out that the digital tracking is likely a violation.

<comment><USER12> If that is true ("equal credit act is the only thing stopping it in the US"), then race is the strongest factor[1] for credit scoring, so I'm not sure how China does it (Chinese population is racially homogenous)?

[1] [https://en.wikipedia.org/wiki/Factor_analysis](https://en.wikipedia.org/wiki/Factor_analysis)

<comment><USER13> The Chinese recognize race within its borders[1], additionally it may be of interest to the scorer to know who is Hui, Tibetan, or Uyghur.[2]. I don't support this I'm just noting that China is not racially homogenous.

[1] [https://en.m.wikipedia.org/wiki/Five_Races_Under_One_Union](https://en.m.wikipedia.org/wiki/Five_Races_Under_One_Union) [2] [https://en.m.wikipedia.org/wiki/Ethnic_issues_in_China](https://en.m.wikipedia.org/wiki/Ethnic_issues_in_China)

</comment></comment></comment></comment></comment><comment><USER9> The baseline (FICO) has AUC of 68.3% which looks low. This may be because the analysis is performed not on the entire through-the-door population, but only on the customers that passed the creditworthiness check (which is using FICO).

In such situations it is customary to do some kind of reject inference or testing below the cutoff, as well as swap-in and swap-out analysis. It does not look like they did any of that.

</comment><comment><USER14> I lead the Data Science team at Oakam, a London-based fintech company founded in 2006.

If you find the article interesting, you may also be interested in Oakam's work using alternative data to predict credit default, which was covered recently in The Economist:

[https://www.economist.com/special- report/2018/05/03/mobile-f...](https://www.economist.com/special- report/2018/05/03/mobile-financial-services-are-cornering-the-market)

[https://www.economist.com/special- report/2018/05/03/mobile-f...](https://www.economist.com/special- report/2018/05/03/mobile-financial-services-are-cornering-the-market)

If you're a Data Scientist looking to work in this area, or just looking for a new challenge, please contact me (personal email in my profile) so we can have a chat!

We are also hiring software engineers (stack is React Native for iOS/Android, and mostly C# for everything else).</comment>


Show HN: our November Startup Month submission, collabacode - <AUTHOR> http://www.collabacode.com Long time lurker, first time poster.
We want to make writing software for the Web easier. collabacode represents our proof of concept.
I find it highly doubtful that collabacode would be where it is without Hacker News. As a show of thanks I'd like you to be the first ones to see it.
Thanks HN! <comment><AUTHOR> Long time lurker, first time poster.

This is a proof of concept version of our new collaborative development environment for the Web called collabacode.

We've only been working on this for 7 weeks and would appreciate some feedback.

Thanks HN!

</comment><comment><USER2> Nice app, the homepage made me almost click back. You have to redesign it.

I think a good audience would be the people who are learning to program. I gave up learning Java programming because it took hours to get things up and running. So a web ide would have fixed that.

You have to encourage users to go full screen. I clicked f11 by accident, and it was much nicer.

I would also like it if I could read Github files here.

Great app overall.</comment>


Tears of Steel, the Blender Foundation's fourth short Open Movie is out. - <AUTHOR> http://mango.blender.org/

<comment><USER2> And the IRC channel is very active now, #TearsOfSteel on freenode.</comment>


DeepOSM: Detect roads and features in imagery with neural nets using OpenStreetMap - <AUTHOR> https://github.com/trailbehind/DeepOSM <comment><USER2> oddly enough a relevant arXiv paper was released today: [https://arxiv.org/abs/1605.08323](https://arxiv.org/abs/1605.08323)

Also how does it compare with [https://github.com/mitmul/ssai- cnn](https://github.com/mitmul/ssai-cnn)

<comment><USER3> Great find, I'll add that to the references on DeepOSM.

Looking at that repo, the main differences are:

* that code applies to a specific (massachusetts) data set, while DeepOSM uses nationwide USA NAIPs trained against OSM extracts

* they seem to have implemented the full Mnih CNN - DeepOSM just uses a single layer as is

* Mnih didn't touch on using the infrared band

In general, that project seems more mature in replicating Mnih than DeepOSM. Our next goal is a website to show errors in OSM data to be corrected, rather than finish making the net deeper.

edit: added links to paper and code [https://github.com/trailbehind/DeepOSM/commit/8053057635a1a0...](https://github.com/trailbehind/DeepOSM/commit/8053057635a1a066ef5aecc8f70b6e2b0c7115d6)

<comment><USER4> When you say:

> Our next goal is a website to show errors in OSM data to be corrected

Do you mean feeding the data to one of the existing projects that crowdsource fixes based on lists of potential errors, or a new custom site?

<comment><USER3> Custom site to start at least, I don't consider any of this really production ready yet.

Here's the app work: [https://github.com/trailbehind/DeepOSM/compare/feature/deepo...](https://github.com/trailbehind/DeepOSM/compare/feature/deeposm.org?expand=1)

<comment><USER4> It might be worth considering integrating with

[https://github.com/osmlab/to-fix](https://github.com/osmlab/to-fix)

or

[https://github.com/maproulette/maproulette2/](https://github.com/maproulette/maproulette2/)

to help with evaluating/fixing of the errors you find.

</comment></comment></comment></comment></comment><comment><USER5> Offtopic, but does someone know a good reference and explanation for OSM tags? A lot of meaning is derived from tags, since the underlaying data structure is just nodes, ways and relations. And since the data is crowd sourced use of tags could be inconsistent, data could overlap and repeat itself. How someone makes a complex project like [1] using data like this?

[1] [http://project-osrm.org/](http://project-osrm.org/)

<comment><USER6> Taginfo[1] provides statistics on tags in use and there's a huge amount of documentation for tags (differences between tags, how to use them in certain situations, etc) in the wiki.

As an example, the page for highway tags[2] is quite rich.

[1] [https://taginfo.openstreetmap.org](https://taginfo.openstreetmap.org) [2] [http://wiki.openstreetmap.org/wiki/Key:highway](http://wiki.openstreetmap.org/wiki/Key:highway)

</comment></comment><comment><USER7> The author mentions difficulties with locating the full text of some papers. This seems like a good time to point out that you can take some DOI URLs, such as [http://dx.doi.org/10.1145/2424321.2424336](http://dx.doi.org/10.1145/2424321.2424336) , and turn them into freely-accessible DOAI URLs, such as [http://doai.io/10.1145/2424321.2424336](http://doai.io/10.1145/2424321.2424336) .

Also, see [https://www.reddit.com/r/scholar](https://www.reddit.com/r/scholar) .

</comment><comment><USER8> Clearly this is super cool. It is also a legal licensing nightmare since it takes "derived product" to a new level...

<comment><USER9> The license of the OSM database allows for derivative works as long as they are under the same copyleft license. No nightmare at all.

<comment><USER8> The nightmare comes when you mix it with proprietary datasets that you license from somebody else. Many companies right now are doing that and it is dubious grounds to stand on.</comment></comment></comment>


POLL: Does being over 40 help or hinder a startup Founder? - <AUTHOR>

<comment><USER2> If you are a rockstar and don't plan on living past 41 then it's a hindrance. Otherwise it's a personalized situation which we really can't answer.

</comment><comment><AUTHOR>
       Hinder

</comment><comment><AUTHOR>
       Help

</comment><comment><AUTHOR> If I add comment does this bump this up?</comment>


Building the BBC's Olympic site - <AUTHOR> http://www.bbc.co.uk/blogs/bbcinternet/2012/08/building_olympic_website.html

<comment><USER2> I was really blown away by how awesome the BBC's coverage was. You could watch every event, live and delayed, and they would even bookmark the live stream as you watched it (and preserved for replays) with event markers. This was particularly useful for track and field where you might want to skip around on various qualifiers and other events.

We never had a single issue with buffering or playback, and it was truly better than TV. It definitely felt like the future. I was really sorry our American friends in the USA couldn't experience the coverage like we had in the UK (without a VPN).

<comment><AUTHOR> I agree. When I first thought about watching the Olympics online I was just expecting to see a version of iplayer with the usual BBC1 and 2 coverage. What they pulled off was incredible. The people behind it should be very proud for actually delivering something that pushes beyond just replicating the TV experience.

My only issue was with Flash. I'm not into bashing Flash just for the sake of it, but my only frustrations were caused by Flash just not living up to the rest of the delivery experience. I think it was the hardware acceleration issue that is already mentioned in their FAQ. Geting the video to play was sometimes a case of throw a 6 to start.

</comment></comment><comment><USER3> For those of you interested in hearing more about this, there's a great episode of the podcast Framerate where they talk to Andy Armstrong, who was behind all of the streaming of the BBC olympics.

There's some really interesting tidbits in there and he offers a candid perspective of how it went and the future of streaming TV, etc.

<http://twit.tv/show/frame-rate/89>

</comment><comment><USER4> The full diagram is available here: [http://www.bbc.co.uk/blogs/bbcinternet/2012/08/21/2012_Archi...](http://www.bbc.co.uk/blogs/bbcinternet/2012/08/21/2012_Architecture_Overview_Diagram_20120727.pdf)

</comment><comment><USER5> The interesting part to me was the fairly heavy use of RDF triple stores. The BBC have used these to good effect before on the world cup site and waxed lyrical at the time about what a difference it made modelling the data in this way. AFAIK they use BigOwlim from Ontotext as their store, run in cluster mode.

</comment><comment><USER6> I see they are talking about this at Velocity Europe [http://velocityconf.com/velocityeu2012/public/schedule/detai...](http://velocityconf.com/velocityeu2012/public/schedule/detail/25792)

</comment><comment><USER7> PHP - another victory.

</comment><comment><USER8> The Paralympics coverage is abysmal by comparison. It's really sad.</comment>


Introducing Pixel, our new phone made by Google - <AUTHOR> https://blog.google/products/pixel/introducing-pixel-our-new-phone-made-google/ <comment><USER2> Unrelated comment but this is the first time I've seen a TLD used like this. Owning a TLD will be the new .com won't it.

<comment><USER3> That is interesting. I think custom TLD's cost a huge amount of money though (like $200k).</comment></comment>


Did Google just admit Apple's Siri is the future of search? - <AUTHOR> http://stritar.net/Post/Did-Google-Just-Admit-Apple-s-Siri-Is-The-Future-Of-Search.aspx

<comment><USER3> (ignoring the translation to a representation a machine can understand) asking when the best time to sow crops requires domain knowledge and lots of analysis, asking to be called upon a certain event is simply implementing a rule.

Sure, something like Siri could be incredibly useful in the future if we can architect it properly, but comparing the current "personal assistant" to a knowledge engine that can answer any possible question is like comparing a Wright Brothers plane to a modern fighter jet.

TLDR: author goes far overboard with his conclusion, nothing new ...

<comment><AUTHOR> I think the biggest challenge for software is understanding the question, not actually giving results to a query.

Do you remember Watson and Jeopardy? All errors it made came from not knowing what it was asked.

Have you tried Siri? It really can give quite surprising responses to complex questions.

</comment></comment><comment><USER4> Google Voice Search has been around well before Siri, it's not news that Google is moving in that direction and voice search has been available on my phones for a long time now (not only from Google). Just because Apple markets yet another technology as new and unprecedented doesn't mean it is, sorry.

Just for you to know: [https://market.android.com/search?q=voice+search&so=1...](https://market.android.com/search?q=voice+search&so=1&c=apps) Most of the apps there are older than Siri.

Also: <http://www.youtube.com/watch?v=gGbYVvU0Z5s> from what i can tell (i neither use voice search nor have an iphone), this looks essentially the same ..

<comment><USER5> Siri _isn't_ voice search. It's capable of it "Google [insert keywords here]", but that's not its main selling point, nor its main use case.

Siri is unprecedented (in a mass market product, anyways) because it combines NLP with actually _doing_ things, and moves away from the awkward "memorize these bunch of keywords in this order" voice control systems of the past.

"Schedule my dentist appointment next Monday at 1." is a much better way of input than the previous status quo of "Calendar. Add Event. Description. Dentist Appointment. Date. December 12. Time. One P.M."

It's weird that people consistently harps on Apple for "marketing yet another technology as new and unprecedented" when it isn't - and then bring up a completely different product to "prove" their point.

<comment><USER4> Can you point to me how exactly Siri works?

I bet it's merely comprehending key words and putting together a search or action that is most likely. Siri doesn't really understand language (else you can point me to proof). In the first case it's not much more then Google Voice Search. I can also write texts, call people, search internet for information on my mobile. Call it what you want, it's Voice Search.

The difference here is: Apples marketing.

<comment><USER5> [sigh] This is somewhat typically "engineer". Because the two products are built upon similar base technologies, they are the "same thing", and any differences are nothing more than "marketing".

Siri doesn't "understand" language, it _is_ , to the best of my knowledge based on a probabilistic solution. That's really where the similarities end.

The innovation here isn't the NLP part, or the probabilistic context determination part. It's the part that ties it into actual concrete actions - it books appointments for me, manages my wakeup times, smartly geofences reminders, etc etc, without me having to touch any UI.

It is in some ways a good reflection of Apple's product philosophy - <USER2> existing individual bits and link them into a compelling end-to-end solution. Music store? Link to your music library management. Then link it to the playback device itself. Voice recognition? Link it to contextual "understanding". Then link it to actually perform device actions. Each piece is somewhat unremarkable on its own, but string it together right and customers respond.

Sure, you can use Siri to do much of the same things Google Voice Search does - directions, showtimes, text search, etc, but that's not really the part that people are clamoring about.

There's a certain segment of our community that seem to be able to stare into every single Apple product, every single record-breaking profit quarter, every single raving review, and then be able to say with a perfect deadpan: "it's all just marketing". I suspect this demographic has a high intersection with the "it's just a big iPod" iPad crowd.

<comment><USER4> No, you don't get it.

One question: Do you think a Google Search only searches for those keywords without taking other factors in consideration? Based on what i can find (checked the source of some plugins of Siri Proxy) Siri is basically just Voice recognition, not "magic". You can easily fake "intelligence" when binding keywords to actions. Basically google search does the same. What's the difference? The marketing.

I'd say those two and Wolfram Alpha are prime examples of this.

A nice example i can show you with Wolfram Alpha. As a marketer i would tell you how supernice and innovative Wolfram Alpha is and that you can talk to it and then show you this:

[http://www.wolframalpha.com/input/?i=How+is+the+weather+in+B...](http://www.wolframalpha.com/input/?i=How+is+the+weather+in+Berlin+right+now%3F)

Wow, right? Alphas UNDERSTANDS you! It's awesome, buy it! But:

[http://www.wolframalpha.com/input/?i=Don%27t+tell+me+the+wea...](http://www.wolframalpha.com/input/?i=Don%27t+tell+me+the+weather+of+Berlin+right+now)

Now, that's simple negation that is not understood. It's easy to fix that. It's still not "intelligence" or whatever Apple is calling Siri. It's simple keyword based approach. Unless someone proves to me that the iPhone has the computing power of really understanding humans, it's old technology in a new (and great) design with massive marketing. As always.

Once you see that, you'll see that voice searching for "weather Berlin" and asking "How is the weather in Berlin?" is basically the same. The first one is just faster to execute (type or say). Only with the second example you can go all crazy and tell people that you reinvented the whole business.

edit:

That said, what is wrong with this link is not how Siri works or who is best or whatever. It's the notion of "Apple was right all along, Google not!!" which is just stupid seeing for how long Google was investing in Voice Recognition. It never was a matter of who is wrong, both companies invested in this technology. Why does it alyways come down to "Apple is right" or "Apple has invented X and Company Y just copies it!", it's ridiculous, like the patent law. Let those companies just build and sell their stuff but don't do all this "Apple was first!" shit, it rarely is. Most products of most companies are really genuine!

<comment><USER5> No, _you_ don't get it. I never claimed that Siri was "magic", nor does anyone besides the most lay of users. I'll easily concede that they're both based on very similar technologies - oh wait, if you actually read my original post, I already did.

The user-facing implementations are remarkably different - and the extension that Siri made to the original concept (that Apple then acquired) are both novel and innovative - something you seem rather happy to dismiss as mere "marketing".

> _"Basically google search does the same. What's the difference? The > marketing."_

Can Google Voice Search respond to "wake me up for my morning meeting"? Or "add a dentist appointment tomorrow at 3?". _That_ is the novel part of Siri, not the voice recognition part - which as you correctly point out was already a solved problem. The novel bit was taking this gooey, nice NLP stuff that we have now, and plugging it into places where traditionally horrible voice control software has been (or places where the shittiness of voice control prevented a real implementation).

But of course, if it's not algorithmically innovative, it's not innovative, period, right?

> _"Wow, right? Alphas UNDERSTANDS you! It's awesome, buy it! But:"_

Why are you stuck on this? I never claimed Siri "understands" you, nor does anyone on HN. We _know_ it's a probabilistic solution...

> _"The first one is just faster to execute (type or say). Only with the > second example you can go all crazy and tell people that you reinvented the > whole business."_

Ah yes, another "engineer" moment. Because we know users are just _dying_ to say fewer words to their device. Oh drats, if only there was a way to shave fractions of a second off voice dictation! I will gladly interrupt my natural mode of sentence synthesis in exchange for an abrupt series of disconnected keywords!

If you look at Google's search logs, you will see that a large segment of the population are _still_ typing full sentences into search engines, despite the fact that it probably _worsens_ their results. This is how "normal" people approach technology, and this is how normal people compose thoughts and sentences.

How dare Apple! How dare they make technology wrap around people's default mode of thinking instead of forcing them to adapt to the input mode that's optimal for the underlying technology! The nerve!

This is why Apple succeeds, and this is why the seemingly infinite horde of Apple-haters don't get it. All of the cool algorithms in the world won't save you from bad usability, or incorrectly exposing your interface to users. Siri took a technology that already existed and extended it into a space that nobody was in at the time - one that users find very compelling, and in some ways solved a problem users didn't know they had. To then turn around and blithely and smugly dismiss this as mere "marketing" is absurd, especially if your only support is that it's "built on the same technology". Need I remind you that we had touch screens for _years_ before the iPhone rolled around, and that we were building PCs long before Windows hit the market. Yet the effect of those two products on the market has been undeniable - even though both were just extensions from existing ideas.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> Please take a look at an older Siri video, which shows its capabilities of understanding. Perhaps it will help you understand this goes beyond voice control and asking simple questions such as "Where is" or "How much is". <http://mashable.com/2011/10/03/iphone-5-assistant/></comment>