The largest payment platform on Earth can reach 2 billion people - <AUTHOR> http://qz.com/5178

<comment><USER2> I must have missed the explanation on the chart showing Mobile Penetration by country. Can someone explain how a country like Vietnam has 173% penetration? Does a penetration greater than 100% mean many people have two cell phones?

<comment><USER3> I've heard (but can't find a link ATM) that it's not uncommon for people in developing countries have multiple phones or multiple SIM cards and will call on one or the other depending on that provider's current rates.

<comment><USER4> Not only that, but I know also many freelancers that have one personal phone/sim and one business phone/sim.

</comment></comment></comment><comment><USER5> Is it the largest payment platform in the world or the largest advertising platform in the world that just happens to do payments?

</comment><comment><USER6> This actually stemmed from an idea Nathan Eagle had a few years ago trying to Mechanical Turk the developing world. The idea being to send micro-tasks (often translation) to people via SMS, then paying them upon successful completion.

That never took off with much traction that I know of, primarily because it was hard to find micro-tasks that were worth the money.

There is this very weird legal distinction in Africa between airtime and actual money. Mobile money systems like mPesa are more and more prevalent, but if you stop and think for a minute everybody has been loading their phones with 'cash' for time eternal. (and been able to transfer it about)

It is only the addition of being able to remove the money from the network as cash that makes mobile money different. But as pointed in the FA, people spend so much money on airtime that for small values of x mobile money and airtime are equivalent.

In any case, smart move by Nathan, it is much easier making deals with carriers to buy large swaths of their airtime than integrate with their totally varied mobile money systems.

<comment><USER7> I think what nicpottier writes above is the insight to take away from JANA, basically communications budget is a fungible commodity for a lot of people. They really want their communication, there is a way to 'barter' communication credits for work. This works really well because there are more than enough ways to 'spend' mobile credits. Given a currency 'mobile minutes' an exchange rate to the local currency, and a market maker (the telcos) to enable both the currency exchange and the ability to transact and its a pretty cool opportunity.

</comment></comment><comment><USER8> The worlds largest payment platform has reached way more than 2 billion.. It's called the U.S. Dollar. I've never been in a Country where it has not been accepted.

<comment><USER9> I'm an American that travels extensively, as well as lives abroad.

The only places outside the US that accept US dollars are places located near the US border, or heavily trafficked tourist areas.

<comment><USER10> Lots of people in Latin America prefer to work with dollars when they can. Look at what the government of Argentina has been up to recently in terms of limiting the population's access to US dollars if you don't believe this.

People in high-inflation places can benefit from holding dollars instead of their local currency - and one easy way to get ahold of them is just to accept them as payment for goods or services.</comment></comment></comment>


Ask HN: What's your favorite ML dataset? - <AUTHOR> I am compiling a list of interesting free datasets for the ML community. What are your favorite datasets to work on?
If you'd like to get emailed the complete list please fill out this google form :)
https://goo.gl/forms/kHOzX35QRte3b7MS2 <comment><AUTHOR> Will post link to all results here once compiled and cleaned up. Or fill out the form and you'll get sent the results via email</comment>


Winamp returns in 2019 - <AUTHOR> https://techcrunch.com/2018/10/15/winamp-returns-in-2019-to-whip-the-llamas-ass-harder-than-ever/ <comment><USER2> I still use Winamp to this day, even if its place was somewhat diminished by streaming services. I was kind of sad that the Mac version wasn't the same as the Windows one. Frankly, sometimes you don't need libraries, syncing, subscriptions or stores. Sometimes you just want to click on a music file and play it without having to deal with all that. That's why Winamp was such a well-loved piece of software: just pop the classic skin on, close the library and use it to listen to audio.</comment>


The problem with deploying iPads in schools - <AUTHOR> http://speirs.org/blog/2010/8/9/the-ipad-project-day-three-end-of-an-era-and-drm-hell.html <comment><USER2> The California junior college system had a lot more funding than the CSU system. My first two years of college were at JC. All the programming classes had a computer at every desk. I thought this made sense, it was a programming class after all.

My first programming class after I transfered to CSU was done in a normal classroom with only a chalk board. The professor would write code in chalk on the board, and I would copy notes onto paper. It seemed dumb, but I learned a lot more in the chalk board classes than I did in the computer lab classes.

I think technology in schools can be good, but it can very easily make things worse too. Right now education really wants to embrace technology, but I have a feeling that they will over embrace it and do stupid things like giving every student an ipad. But maybe I'm wrong. Maybe wide spread adoption will allow for helpful innovation.

</comment><comment><USER3> Ironically, today Apple announced the App Store Volume Purchase program, targeted specifically at education institutions. Apple don't provide much information precisely how it works from the point of view of a school, but from the 5 sentence blurb on the developer connection site it seems volume purchases and discounts on volume purchases are in the mix.

<comment><USER4> Apple does this with portables too. It is generally in groups of 10+.

</comment></comment><comment><USER5> An "App Store for Education" should be in the works. I don't if there's an opportunity here for an entrepreneur or whether Apple will do this but it's not just about iPads. Students are going to be increasingly learning on all sorts of mobile devices and schools are going to want to have a way to centralize purchasing and control distribution. In addition, discovery on the current App Store is focused on the mass market consumer, not the needs of a student, teacher or school administrator.

I'd be very interested to hear a discussion around the feasibility of building a software platform specifically focused on distributing educational software to mobile devices in schools.

</comment><comment><USER6> Android has an "allow other markets" checkbox, which would enable his preferred solution. But I guess it had to be Apple for some reason.

What school gives iPads to all it's pupils, and why???

<comment><USER7> That doesn't solve the problem (unless you're advocating piracy) - they're looking to distribute paid apps from the App Store on many devices. It does strike me that they're willing to pay for the iPads at $499 a pop but don't want to buy more than one copy of an app.

<comment><USER6> So in theory you can sync to as many iPads as you want? Didn't know that.

</comment></comment></comment><comment><USER8> It would benefit Apple greatly if they were to distribute classroom apps for free- at least the basic stuff needed for student/teacher interaction. Second, they should promote wiki-style development of textbooks and associated materials. The textbook market is a huge sham and a leech upon our education system. Apple could make tons while simultaneously saving schools tons if they could free them from the textbook companies.

</comment><comment><USER9> It's an ostentatiousness toy; I'm geniunely interested how it could function in any educational environment?

<comment><USER10> Can't you say that about virtually any computer?

First thing to note in the school context is that the iPad is small and light, meaning it's easy for pupils to carry, and the battery lasts 10-15 hours, meaning they can carry it about all day. It's also a far superior reading device to anything else on the market... I mean do you genuinely not think it would be useful in education?

<comment><USER11> You consider the iPad to be a "far superior reading device" to, say, a Kindle or Nook? I do think the iPad (or any quality tablet with educational software available) could be very useful in schools, but I think it's a far cry from the "far superior reading device to anything else on the market" you claim it is.

<comment><USER10> I don't think it's far superior for the purpose of simply reading text, I was alluding to all the other capabilities it adds to the bargain. Colour, video, pinch-to-zoom etc.

</comment></comment></comment></comment><comment><AUTHOR> Related article from Ars Technica: [http://arstechnica.com/apple/news/2010/08/new-school- program...](http://arstechnica.com/apple/news/2010/08/new-school-program-for- ios-apps-doesnt-address-all-it-needs.ars)

</comment><comment><USER12> c'mon. if you want child to learn computers - install *nix and ask them to work in it.

but, ahh.. these government budgets - that they HAVE to spend..</comment>


The centennial of America's greatest sports scandal - <AUTHOR> https://www.city-journal.org/black-sox-1919-world-series <comment><USER2> The article rightfully mentions Dickey Kerr as a hero of the series by winning two complete game starts.

Curiously, Shoeless Joe Jackson set a World Series hit record that was unbroken until 1964 and had the highest series batting average of either team. He also hit the only home run of the 1919 World Series.</comment>


Artificial AI-Our experiance with AWS Mechanical Turk - <AUTHOR> http://shout.setfive.com/2008/11/20/artificial-artificial-intelligence-our-experiance-with-mechanical-turk/ A quick write up of our experience using Amazon's mechanical Turk service. <comment><AUTHOR> sorry for not making it clear - the bitmaps all ready had text inside of them so we were basically looking for OCR.

</comment><comment><USER2> (Assuming the original author is either the poster, or will be reading this...)

I don't quite understand the task; "transcribe these [image] labels"? Do the image bitmaps already have textual labels inside the picture, so you essentially need OCR?</comment>


Chasing the DAO Attacker's Wake – A second exploit - <AUTHOR> http://pdaian.com/blog/chasing-the-dao-attackers-wake/ <comment><USER2> As the article briefly suggests, this problem can be completely avoided if every method has no more than one external call and always puts it last.

I'm messing around with contracts to do about a dozen different things, and it turns out to be easy to meet this restriction, as long as I'm willing to design the UI accordingly. (E.g. don't send money to lots of users in one step, just update their balances and make them call a withdraw function.)

<comment><USER3> It seems like there are quite a few pitfalls to avoid just in writing these contracts, and they aren't obvious things in many cases. This seems _incredibly dangerous_ , given what is at stake when a bug is included in a contract.

Is there any form of auditing that contracts can go through to not have these pitfalls?

I wonder how many of them are still undiscovered...

<comment><USER2> There's both an active developer community willing to help for free, and professional auditors you can pay. There are also people talking about a NASA- style bugs/best practices database at some central location.

People are realizing that this needs to be developed like safety-critical software, not like fast-moving startup websites. But I would say it's not as scary as, say, developing software for aerospace or medical applications.

And what really helps is that you can do a lot with very little code. I think all of my contracts are under five pages. I don't think we're ready for big complicated contracts, but there's a lot we can do with simple little contracts.

<comment><USER4> Five pages? I thought these things were meant to be shorter than plain-English contracts. I draft and review contracts daily and five pages is long for basic things (employment, NDAs, IP transfer etc).

<comment><USER2> It's software, not really comparable to an English-language contract. Most software is much bigger than that.

Five pages is an outside estimate, for something like a currency exchange. My crowdfunding contract is about two pages, and the vault is one.

<comment><USER5> > It's software, not really comparable to an English-language contract.

If the point is to be an alternative to normal, "dumb" contracts, than that is exactly what it needs to be comparable to.

<comment><USER4> He is correct in stating that it is code v. english, apples v. oranges. But I was under the impression that the whole point of using code was to explain something in more concrete and definitive terms. If doing so takes more words, more abstract word, with more oppertunity for bugs and loopholes, then what is the point?

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> So one contract can call a function in another contract, in the middle of their transaction:

        if (_recipient.call.value(_amount)()) { // vulnerable

That's inherently a scary feature, and it was exploited.

Etherium's "Solidity" suffers from the desire to have it be a general-purpose computer. It should have been if-then rules, or a logic tree, or a decision logic table - a finite representation of business logic subject to exhaustive case analysis. Contract executions should have been atomic transactions - either everything commits, or everything gets rolled back. Stack overflow and running-out-of-gas terminations are inherently trouble.

Etherium needs to go back into the shop for a redesign. At least we found out early that this isn't going to work.

<comment><USER7> > At least we found out early that this isn't going to work.

$US150M later isn't early

<comment><USER8> That's called a bug bounty.

<comment><USER9> I came up with "self-distributing bug bounty piñata" in the last DAO-failure discussion [https://news.ycombinator.com/item?id=11921677](https://news.ycombinator.com/item?id=11921677)

<comment><USER8> Hehe, that's even better. But it is the essence of the matter. If you put $100M in a bucket you've just incentivized the worlds bad guys to audit your production in order to figure out how to open the bucket to get to the loot.

And this goes for all of commerce, which is an important realization: security is not an all-or-nothing proposition, it is an economic affair. Whatever you wish to secure, you can expect to expend a reasonably constant fraction of that on the act of securing it. So if you wish to secure $100M you will have to do a much better job than if you wish to secure $10M or $1M, just like you're going to spend more money on the lock for a $1k bike than a $25 junker.

When DOA ended up being over-subscribed they should have _immediately_ yanked the emergency brake because whatever plans they had would end up being moot by virtue of storing more money in the bucket than they originally planned.

Their security measures would _by definition_ not have been up to the task because they had planned to secure a much smaller amount. Either that or they were ridiculously over-confident.

</comment></comment></comment></comment></comment><comment><USER10> "This impacts all contracts on Ethereum, not just the DAO. This is an issue with Ethereum's JavaScript-like programming language (Solidity)."

No it doesn't, no it's not. It affects contracts that use this functionality with arbitrary untrusted contracts (which is a bad idea), and most contracts just don't use this functionality at all.

<comment><USER11> Yeah, all this shadenfreude from people who (wrongly) predicted ethereum would never work, and who want to feel vindicated, is predictable but frustrating.

<comment><USER12> Not at all.

When I discovered ethereum a couple of years ago I thought it was a good idea. Partly because I was turning similar ideas in my head at the time.

But what I read now sounds like amateurs designed & implemented it. Sorry.

<comment><USER13> And if masters had invented it, no one would be using it.

</comment></comment></comment></comment><comment><USER14> Isn't this just a normal data race with stateful actors? Something modified the data while A is calling B (doesn't have to be B, could be any X) in between a call and its response.

The answer to this is that mutation needs to be kept under control. Immutable data structures in mutable buckets. Locking around the data that mustn't change. Deep copy all state before calling out. And so forth.

This is basically an understood problem and I'm a bit appalled that Ethereum didn't build protection against it into their design.

<comment><USER2> It's not quite a data race, since there's nothing parallel at all. Transactions are put in a particular order and run one at a time. So it actually does have to be B that does the modification.

One easy solution is to have just one external call per method, and always put it at the end of the method. Another is to use a single mutex for all public functions of the contract, so any callbacks fail.

</comment></comment><comment><USER15> I don't understand "you can't assume anything about the state of the contract." Surely the contract's state is only modified by the contract's own code, which means you can assume that the state is not altered arbitrarily but only according to the rules set up by the contract. Yes, you need to be careful about external calls that make altering calls to you.

<comment><USER16> The problem pointed out in this article isn't that it is impossible to write correct contracts. The problem is that it means that it is _superhumanly difficult_ to write correct contracts, using the current feature set and infrastructure. This is especially true in what is theoretically an actively hostile environment, which the DAO hack proves is also _actually_ an actively hostile environment.

It is theoretically possible to call out to another contract that has the ability to make further calls safely... but how can you be _sure_ you've done it safely?

This isn't a unique problem. It arises in imperative-based programming all the time. But what in Javascript may be a bit of a coding error that means the next page of search results didn't load this time, in Solidity it means you may lose your shirt.

Also, I'd suggest that after-the-fact "static analysis" tools don't help. It's basically the same situation that real-world computer security is in, and right now, that world is _massively_ advantage attacker, with generally far less direct incentive to attack than a blockchain functioning directly as money does. If the static analysis tool is open enough for "everyone" to use, the attackers get to use it too, and they can use it on all contracts at once because they're all open, and they have all the motivation to do so before the proper contract owners have time to fix their contracts. Every time the static analysis tools release a new check, it's an advantage-attacker race between exploiting the new check and fixing the contracts. The attackers are going to win big, repeatedly.

A Turing-complete blockchain needs to come out of the gate nearly 100% correct on this front, just as it needs to come out of the gate nearly 100% correct on the encryption security.

<comment><USER15> I'm not sure that it is super humanly difficult. It seems likely that programming patterns, such as not using I trusted external sends at all, will emerge that any contract will need to abide by to be trusted.

If what you say is true then Ethereum is DOA, which seems unlikely to me. (I don't own any ETH.) I totally agree that it would be great to have more language based security and I've started dabbling with some stuff myself (playing with verified byte code compilation in Agda) but my basic attitude is that worse is better, these hacks are unfortunate but educational, and now we're going to see way more care in contract construction and let's hope it works out.

It might be that Solidity is going to be replaced, and it's super interesting what will come. A bunch of people are insisting that some variant of functional programming with types is necessary for correct contracts but I'm skeptical until I see an actual proposal...

<comment><USER16> And there are still people running around who think that writing code based on threads and semaphores isn't superhumanly difficult and people just need to Try Harder (TM).

At least when they're wrong, they usually don't lose millions of dollars. (But only... _usually_.)

I think some people are mentally modeling Ethereum as a sort of hobby thing in their head when they say things like that. I mean, who cares if my hobby static-rendering blog platform has a few bugs in it, right? But Ethereum aspires to be the foundation of an economy. Real foundations of economies have _billions of dollars_ worth of effort poured into them. While a small set of particularly brilliant people may produce something that can run in this environment, Ethereum isn't what it would look like if they did.

I also think that people don't realize that the usual fate of a cryptocurrency is to die. Bitcoin isn't the first cryptocurrency by a long shot. Cryptocurrencies in general are not strong, robust things. I think something _like_ Ethereum is quite likely, but there's no particular reason to believe we aren't three or four more major fully independent stabs at the problem away from a stable solution. Ethereum makes Bitcoin look like a homework problem in comparison; it would almost be beyond belief that we'd get it right the first time.

<comment><USER15> I think humans are capable of writing correct programs even in machine code, it's just tricky and requires some expertise.

I also think the EVM itself allows for other paradigms than Solidity that might make it easy to create certain kinds of contracts.

And I think worse is better is very powerful, so it wouldn't surprise me if Ethereum sticks along for a long time, like Bitcoin seems to be doing despite its flaws.

</comment></comment></comment></comment></comment><comment><USER17> "that impacts all of Ethereum"

Well, all of Ethereum smart contracts which are written in Solidity and which use a "call construct" in a certain way.

</comment><comment><USER18> With all this fuss over ethereum security flaws, can someone explain to me the practical use case for these smart contracts? I just don't understand where the benefit comes in. It seems like any type of contract that would be useful requires a human to qualify the meaning of the terms since these contracts cannot autonomously measure the state of the world.

Even something as trivial as betting on sports requires deference to a trusted authority to say "this team won". What advantages does an ethereum smart contract provide over, say, some kind of trusted bitcoin escrow site?

<comment><USER2> Contracts I'm working on include variants of anti-theft vaulting, blind auctions, crowdfunding, person-to-person gambling, currencies, exchanges, and a simple implementation of Bitcoin's Lightning. None of them require external data.

For those contracts that do, at least you're reducing the third-party trust to "provide accurate data" instead of "hold my funds without stealing them." To reduce trust in a single entity you can use various schemes that allow multiple independent parties to provide the data.

<comment><USER18> > _anti-theft vaulting_

Can you go into more details about this?

> _blind auctions_

Or this? How does the contract ensure that the winner of the auction receives what they have purchased?

> _crowdfunding_

In what respects beyond simply collecting funds? How does the contract judge that the terms of the funding are adhered to or ensure that the crowdfunders receive whatever it is they are entitled to as funders?

> _person-to-person gambling_

Any examples? In my post I gave the example of betting on a sports game but this requires a trusted 3rd party to officiate the outcome of the game.

> _currencies, exchanges_

These are vague descriptions and these things have already been done by bitcoin.

I'm not trying to nitpick, but I'm looking for specific practical examples that demonstrates the potential utility of ethereum.

<comment><USER2> So this gets into the difference between the rhetoric about smart contracts, and what they actually do. Obviously a smart contract can't ensure that an auction winner or crowdfunder gets the item. At best there could be some kind of reputation system. But to enforce something in the real world, you'd have to get an actual court involved. The smart contract removes the need to have an auctioneer running the auction and handling funds, that's all.

But that still means that, for example, when I'm trying to implement a new variant on crowdfunding, I can write the whole backend in several pages of code and just publish it, instead of renting servers, administering databases, signing up with a payment provider and having to comply with its terms of service.

An example of a vault is this scheme: [http://hackingdistributed.com/2016/02/26/how-to-implement- se...](http://hackingdistributed.com/2016/02/26/how-to-implement-secure- bitcoin-vaults/)

...which some researchers proposed for Bitcoin, saying it'd be "easy" to implement with a hard fork adding a new opcode. I implemented the same scheme in a smart contract and it took 20 minutes.

Sports betting needs third-party data, but casino games don't.

<comment><USER18> Thanks for the reply. I think I understand what you're getting at.

</comment></comment></comment></comment></comment><comment><USER19> Interesting Ethereum uses the term "function" however in programming we understand a _pure_ function to be an operation that always returns the same result given the same inputs.

I don't know anything about Ethereum, but if functions rely on external shared state then they aren't pure functions, are they?

<comment><USER20> Well, solidity is kinda javascript-style syntax and that's the keyword from javascript.

On a pedantic level, you're absolutely correct, but real-world usage generally makes 'function' equivalent to 'subroutine', hence why we have the specific term _pure_ function to disambiguate.

</comment></comment><comment><USER21> What if the DAO invested in short-selling Ethereum? 🤔🤑

<comment><USER22> There _was_ a 3K BTC worth short position right before the attack.

<comment><USER23> No, there were 3K BTC worth of short ~positions~ right before the attack.

</comment></comment></comment><comment><USER24> Any code that calls arbitrary callbacks is prone to failure. The context in which a callback is called matters.

Calling a callback at the end of a method avoids interfering with the caller's state but ignores the state of the caller's caller.

An ethereum contract will only be tractably analyzable if it avoids dynamically invoking arbitrary callbacks.

</comment><comment><USER25> None of these "exploits" really sound much more dramatic than using a database without transactions

<comment><USER26> Actually it's much worse, it's the equivalent of calling a function that ends up doing something totally different than you thought it would. Very reminiscent of a web bug that has since been blocked by newer browsers: parsing JSON data that ends up using a constructor redefinition exploit to execute arbitrary JS.

Pretty much this: [http://www.thespanner.co.uk/2011/05/30/json- hijacking/](http://www.thespanner.co.uk/2011/05/30/json-hijacking/)

</comment></comment><comment><USER27> Given that trust is the single most critical aspect of the entire platform, why doesn't it use languages (for both the platform, and the contracts), programs in which can be easily formally verified, and for which tools for such verification already exists?

</comment><comment><USER28> Seems like this is what mutexes were invented for

<comment><USER14> With a mutex, an A -> B -> A call would deadlock if the mutex is the stop-and- wait or spin-and-retry sort. It would have to be a succeed-or-exception mutex, which is not a common design.

<comment><USER15> EVM doesn't allow the implementation of a blocking mutex and a "spin-and- retry" would simply run out of gas.

</comment></comment></comment><comment><USER29> Solidity is to Solid as Oracle is to Unbreakable.

These really are solved problems.

</comment><comment><USER30> The backers of the DAO selflessly created a $150 million bug bounty for Ethereum and their Smart Contract - Good for them.

<comment><USER31> Yeah, but they seem to be awfully reluctant to pay out on it.

</comment></comment><comment><USER32> I'd be interested in knowing how this has been dealt with (or not) before. Was the problem recognized when designing E?

</comment><comment><USER33> Will the attacker be able to spend any of the funds?

</comment><comment><USER34> So what they should have implemented is communicating sequential processes / actors instead of arbitrary calls.

The more I read about ethereum's technical details, the more it sounds like a joke to me.

<comment><USER10> It's a code mistake by who did the DAO code. (DAO != Ethereum) It's more the equivalent of using eval in your nodejs code to evaluate some parameter received from a user. It's a stupid code mistake, but you wouldn't say the internet is a bad idea because some website got hacked.

<comment><USER35> _It 's a stupid code mistake, but you wouldn't say the internet is a bad idea because some website got hacked._

But if you're setting up a situation where server after server is constantly compromised because of hard to discern reasons, then yes, that is a bad idea. Having an internet is something as obviously good as electrical wiring and highway bridges. Having a badly implemented internet is as obviously bad as faulty wiring and badly engineered highway bridges.

This stuff is making Ethereum sound like it was written by people as unqualified to make such a system as a dev who would use eval in nodejs code on a user supplied string is to implement an ecommerce site.

<comment><USER10> Essentially this is the equivalent of a dev mistakely using an eval in the wrong place in a nodejs code. But all these articles are making it sound like the Ethereum team created the DAO and this issues affect Ethrereum as whole, which just isn't true.

<comment><USER36> The way I see it (continuing the simile): if you're writing something as critical as monetary transactions, _everywhere_ is the wrong place to use eval in nodejs code. And when you're writing a dedicated language for monetary transactions, _including eval in your API_ is just as wrong. The last part is why this issue affects all of ETH.

Solidify is a new language, specifically designed for Ethereum. If the response to contract issues like this is "but that's a known failure mode, don't do that", then that signals a problem with the language, not just the contract code.

<comment><USER35> _If the response to contract issues like this is "but that's a known failure mode, don't do that", then that signals a problem with the language, not just the contract code._

It also signals a problem with the culture, with respect to the perceived amount of rigor required by the problem domain.

</comment></comment></comment></comment></comment></comment><comment><USER37> Every article I see on etherium mentions the DAO. Not one of them explains what it means.

Gotta love unexplained acronyms!

<comment><USER38> It means "Decentralized Autonomous Organization".

It's usually used as a common term ("I'm creating a DAO"), but these guys called theirs "The DAO". This attacker exploited "The DAO".

<comment><USER39> Ok, but you realize that merely listing the words in the acronym does nothing to explain what it actually is. The whole ethereum "ecosystem" is a radical and new construct that very few people are familiar with.

Given that, I would even say that the name "Decentralized Autonomous Organization" is a deliberate, almost satirical, obfuscation.

</comment></comment></comment><comment><USER40> This reads a little sensational. The original source [1] referred to in the article looks at the issue in more detail.

[1] [http://pdaian.com/blog/chasing-the-dao-attackers- wake/](http://pdaian.com/blog/chasing-the-dao-attackers-wake/)

<comment><AUTHOR> Agreed the original article does a great job. However, without the "sensational" title and simpler explanation people were not paying proper attention to this.

The original article discussed a bunch of other things as well -- everyone should read it to understand the details.

<comment><USER22> By people you mean traders. This sensationalism is utterly unnecessary since the non-traders involved with Ethereum read the original article and are capable of understanding it.

</comment></comment></comment><comment><USER41> Nearly a year ago I made the comment --

"Who are these people[1] and what credentials do they have to build and run a cryptocurrency platform?" [1] [https://angel.co/ethereum-1](https://angel.co/ethereum-1)

Today, not a single cryptographer, mathematician, or anyone with software security experience is listed on the Ethereum Foundation member list[2]. I am not attacking any individuals by themselves, but as a group this says a lot.

For the record, what I see listed online today is a: consultant, magazine founder, a programmer, and a lawyer. On the advisory board a sales person, a psychologist, a leveraged debt consultant. On the "special advisors" list, which I have no idea what this means, maybe someone who answered a phone call a few times, one person with a bunch of business credentials, something "fintech" related with little explanation, and a tech CEO.

Maybe the Ethereum Foundation doesn't need anyone with deep experience in cryptography and security?

[2] [https://www.ethereum.org/foundation](https://www.ethereum.org/foundation)

<comment><USER42> This is unfair to banish a company for not having enough "security experience". It could happen to anyone (i mean making such a design mistake), plus they afaik had some external audits.

"Aha, no security background!" is wrong.

<comment><USER43> It is absolutely fair to banish a company for security credentials if it's dangling $150 million out in the open air.

<comment><USER44> > _What are you talking about?_

Please don't be uncivil. It's damaging to collegial conversation, and it's against the rules ([https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)). Please edit such stuff out of your comments here.

<comment><USER43> edited.

<comment><USER44> Appreciated!</comment></comment></comment></comment></comment></comment>


MySQL is to SQL like MongoDB to NoSQL - <AUTHOR> http://use-the-index-luke.com/blog/2013-10-01/mysql-is-to-sql-like-mongodb-to-nosql <comment><USER2> While a lot of people people have disagreed with this guy in the past, I find it very hard to disagree with him in most of his posts.

My exposure, as far as SQL goes, has been MsSQL, PostgreSQL and MySQL. With absolutely loads of MsSQL, and pretty hairy data structures at that (they have an aversion to the polyglot approach where I work) - SQL is one of the most enjoyable things that you can do once you get past the elementary "SELECT WHERE" (and stop using graphical development tools).

I have been saying what Markus said for a long time, MySQL is why RDBMS has a bad name and why the "not using SQL" movement exists. I really hope the MariaDB team spend their time where it is needed most (a join engine that doesn't suck), if they haven't already.

I mean, at the end of the day you have MySQL that can't even do hash joins, and then you have PostgreSQL with GEQO: [http://www.postgresql.org/docs/9.0/static/geqo-pg- intro.html](http://www.postgresql.org/docs/9.0/static/geqo-pg-intro.html)

<comment><USER3> You should know that the fork of MySQL, MariaDB, addressed hash joins well over a year ago: [https://mariadb.com/kb/en/block-based-join- algorithms/#block...](https://mariadb.com/kb/en/block-based-join- algorithms/#block-hash-join)

<comment><USER4> Unfortunately thats only a year ago. I believe MySQL added transactions over a decade ago and you still run into people who insist MySQL doesn't have transactions, because five years ago they read something written seven years ago that was years out of date at that time.

You just don't see behavior like that outside the database community. Hey did you know that BIND doesn't support IPv6? Yeah version 4.9.3 doesn't support ipv6. Don't bother me with facts like version 9.0 was released more than thirteen years ago in 2000.

So once every installed and reasonably security patched mysql server in the world supports hash joins, the world has at minimum a decade of people who have no idea what a hash join is, and no idea how or why to use it, none the less demanding that mysql doesn't support it.

(Bitter? Yeah maybe a little. You can get good advice about OS, or compilers, off the internet. But overall the internet is useless for database advice, a couple stars don't outshine the greater cesspool)

<comment><USER5> I have long said: "MySQL is the PHP of databases."

In my experience, the database is capable, but the database itself has some quirks (whether these are due to the database itself or the developers I do not know). Some things I do know for sure:

\- I worked on a product which supported MsSQL, Oracle, Postgres, and MySQL. Only MySQL compared character columns in a case insensitive way which caused us some extra work, but we did fix this using the proper collation.

\- I worked on a Django project. When using MySQL, a Django ORM query would pull the entire result set and hold it in memory while you iterated over it, sometimes consuming gigs of ram if we weren't careful. Iterating over a result set from Postgres would pull (presumably) only one result at a time. Whatever the case, it didn't cause memory problems like MySQL. This is not MySQL's fault, but again, it was the exceptionally problematic database.

\- I'm currently working with a MySQL database which is quite troublesome because foreign-keys are not enforced. See a non-null foreign-key? You might expect it to refer to an existing record, but you'd be wrong. Again, failure of the developers, not the database.

\- I also am currently working with a MsSQL database. We had to add a column to a MsSQL table; no locking, no problem. Then I had to add a column to a MySQL table, which locked the entire table for the length of the operation. MySQL was, again, problematic where another database was not.

There are other problems I have only heard of, but I will not mention those. All the above are personal experiences where MySQL was causing trouble where other databases were not. It parallels my experience with PHP, both are great tools if used right, but they seem to be misused more often then other technologies.

<comment><USER6> MySQL has been my go-to database for years because it was fast, reliable and effective. Unfortunately it hasn't been evolving as quickly as other databases, Postgres in particular.

There's a lot of quirks in MySQL that were excusable a decade a go, but now they're problems that've been solved in virtually every other database engine. Working with Sybase was never a pleasant experience, but versions a decade old have features that MySQL _still_ doesn't have.

Every so often you'll hit the wall with MySQL, little things like applying a LIMIT in a subselect, or row length limitations, that will make you wonder why you're using MySQL at all.

MySQL is absolutely the PHP of databases.

<comment><USER3> Be aware that when using text or blob types, these only contribute ~12bytes to the total row length limitation of 65kb.

<comment><USER6> Actually that's what I thought, but it's not always the case. Sometimes a good chunk of the TEXT type column is saved in the row itself (~768 characters), and when the length exceeds that, the remainder is stored in the blob area of the table store. ([http://www.mysqlperformanceblog.com/2011/04/07/innodb-row- si...](http://www.mysqlperformanceblog.com/2011/04/07/innodb-row-size- limitation/))

This depends on your row format, an obscure technical issue most never have to deal with, but it can be a problem.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I may be biased because I work on it, but I think TokuMX solves or will soon solve all of the "big data scaling" problems that exist in MongoDB.

There is definitely still room for polyglot strategies: Zookeeper is for tiny but crucially consistent data, Riak is for fancy distributed systems availability guarantees when you can afford a simple data model, and I believe Redis has value as a sophisticated programming model for things you can fit in RAM (but I actually have no Redis experience personally).

But in the past few months, I've grown to be really impressed with the document model, and the aggregation framework is getting more and more powerful with each release. I think that's the important part of MongoDB (sharding's a little messy and Riak seems to have their heads on straighter with that), and TokuMX takes that and sands off the rough edges you see with big data sets and concurrency, and I think that's going to end up dominating MongoDB and being a really compelling point in the NoSQL space.

</comment><comment><USER8> People like to give MongoDB shit, but is actually pretty fun to use. I wouldn't use it as a "big data platform." At scale I'd use Cassandra. For relational data I'd use postgresql. For memory caching, Redis. So when do I use MongoDB? For prototyping. Why? Because it's fun to use.

<comment><USER9> Sorry but "fun" or "developer productivity" aren't allowed.

You have to have a database that is expensive, obtuse or difficult to manage to be taken seriously.

<comment><USER10> "Developer Productivity" that leads to "ops people refuse to work here" isn't a stellar strategy.

<comment><USER11> Do 'ops people' tend to not like MongoDB?

<comment><USER10> Despise it.

<comment><USER11> I've been nothing but an 'ops guy' for 20 years now, and I've been enthusiastically using MongoDB for my 'devops' work since 2009.

Having said that, I've never used MongoDB 'on the large'. I've always used MongoDB against datasets that mostly fit into memory, and use some other solution for 'bigger data'.

I suspect most of the MongoDB dislike is rooted in three points: 1\. The 'Mongo People' really over-hyped it early on. 2\. The decision to not have single-server disk safety (but providing very elegant multi-server data safety) was a poorly understood decision. I personally agree with their direction. Your data isn't really, really safe until it's on more than one system. 3\. Based on what I've read, I don't think MongoDB scales up (even with their built-in sharding) as well as some other solutions. Perhaps this is limited to some kinds of use cases, and not others.

In my opinion, for people who do day to day fast development with dynamic languages, like myself, MongoDB, used correctly, is really fantastic, at least for many use cases.

</comment></comment></comment></comment></comment></comment><comment><USER12> At work we have an MS SQL Server instance managing ~1TB of data spread among various databases.

No one database is > 200GB so it's not "web scale" by any means but sometimes you need to pull out tricks like indexed/materialised views.

I use Transact-SQL directly via ADO in C++ and ADO .NET in C# and indirectly via EF and NHibernate.

It's easy to manage schema with SQL Server Data Tools and diagnosing a poorly performing query is straightforward with graphical execution plans.

Quite frankly, I don't care that there are alternative RDBMS' or alternatives to traditional RDBMS.

But ... I keep hearing about problem after problem with MySQL. What's the trick to using it successfully? Is the barrier to entry too low, and problems are from legions of rank amateurs? i.e. is it as simple as constructing your schema thoughtfully, with tables that are "well-it's-almost-3NF" and making some educated guesses about which indexes might be needed in advance?

<comment><USER13> MySQL is a good database for basing a CMS on. (for a loose definition of database)

PostgreSQL is a good SQL database.

And don't take me wrong, I'll take MySQL instead of PostgreSQL for simple and medium projects.

"Is the barrier to entry too low, and problems are from legions of rank amateurs?"

Maybe. Well, "low barrier to entry" is not a defect.

<comment><USER14> _MySQL is a good database for basing a CMS on. (for a loose definition of database)_

It seems to me that the success of MySQL can be tied directly to people using "database" for things where what they really need is "filesystem".

<comment><USER15> How easy is it to put a lock on a file while writing on it? What happens in an environment where you have multiple frontend servers that need to write to the same file? Are you going to have like one server that does the reading/writing from a file and have like a protocol with which other servers can send commands?

Great, you've just invented a half-assed DBMS.

<comment><USER16> > Great, you've just invented a half-assed DBMS.

My first thought is "Yeah, we already have one of those, it's called MySQL"

</comment></comment></comment></comment></comment><comment><USER17> MySQL is also generally poorly "designed". MySQL is to a database as PHP is to a programming language.

<comment><USER18> I've heard lots of complaints about mysql, do you know if query deadlocks are more common with mysql than with something like postgres? I've been running into that a lot lately, and I don't understand what the hell is going on. None of the queries I'm executing have particularly long run times (usually a few milliseconds, occasionally 100ms), but I still get random deadlock errors from time to time. I've never run into this on any other database platform (most of my experience is with Oracle and SQL Server), and I have no idea what the hell is going on.

<comment><USER4> We also have no idea whats going on with your system. I ran into this twice that I can recall. Once was a pathological SELECT FOR UPDATE where it was locking up way too much stuff, and another was a time where a boring looking update was locking a whole bunch of seemingly unrelated indexes.

So for the two examples I ran into all I can say is make your transactions as small as possible and take a close look at ALL the indexes especially if you think they're unrelated.

There's an option for logging the intimate details of deadlocks into the log file (or error file or whatever it was) anyway the main PITA is having to restart the server to enable it. On second thought the real PITA is operational in the "watched pot never boils" tradition as soon as you enable it the intermittent problem seems to go away, just dumb luck.

<comment><USER18> The query in question is an UPDATE mytable SET flag=1 WHERE ID IN(...) statement. I'm assuming that another system was in the middle of a performing a SELECT statement on the flag column.

In this particular case it's perfectly ok for the select statement to get slightly incomplete data (e.g. returning a record with a flag that might have just been changed), so it would be nice if I could configure the table so that select statements never lock. Is it possible to do something like that?

<comment><USER4> The first stack exchange when I googled was pretty interesting. For completeness, if you use innodb as your engine

SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

Seems to be the line you are asking for. There's the usual way to make it a system default in the config file. As for the difference between what you are asking to do, and what you should do, well... be careful. Especially if you're feeding the dirty read back into something that could lead to an update. That could be icky.

There's some way to do myisam stuff.

Generally I've found if you've got two things trying to lock one thing, overall processing goes a lot faster if one greedy dude grabs it for the whole thing. Cooperative multitasking doesn't work if it spends way more time lock switching than actually processing. Of course greedy processing is an excellent way to jam things up if you're not careful.

The funniest discovery I ever made about that related to importing zillions of rows with autocommit off it was dramatically faster rather than demanding an I/O store. This was the kind of data where if someone yanked the power cord it would be auto-re-imported without any loss; if your only copy of the data is in RAM and the INSERT you just ran, then autocommit off might be unwise.

Your IN (...) if ... is a select that might none the less be similar to my fun time a couple years back with dueling indexes. This is the origin of the whole never index more than you have to meme because it doesn't just waste storage and write time, an "innocent" looking pathological index intended to make some report run faster can kill read performance if its touches too many things other indexes touch while "separate" processing is happening. If you don't know A leads to B, B leads to C, C leads to D, then why does messing with A kill D performance when D should be totally separate, ah forgot about that peculiar linkage...

Note that none of this is a mysql problem. You can shoot yourself in the foot with a common 9mm or a weirdo caliber. More people get hurt by the 9mm not because it inherently magically hits feet more often but because theres just more people using it. I'm quite sure you can screw up Oracle or DB2 the same way if enough people try hard enough. I guarantee if you google you're going to find more "mysql did something weird" stories than any other DBMS simply because more people use mysql.

<comment><USER18> Thank you for helping me, this is great.

</comment></comment></comment></comment></comment></comment><comment><USER3> [http://www.xaprb.com/blog/2013/10/01/mysql-isnt-limited- to-n...](http://www.xaprb.com/blog/2013/10/01/mysql-isnt-limited-to-nested- loop-joins/)

<comment><USER19> I note for those who don't get to the bottom, there are comments there that suggest the author of the post you linked is incorrect. I'm not qualified to know who is right.

<comment><USER3> Both comments are speculative.

Are people, in general, aware that MySQL employs a pluggable storage engine? I wonder if much of the confusion about MySQLs abilities stems from arguments from people aware of it against those that have only ever used the stock engines.

<comment><USER20> Most people use the default storage engine, without really thinking about it, but almost every project I've worked with recommends innodb - not sure why.

<comment><USER4> There's no surprise it became the default storage engine instead of myisam around early ver 5.5 or so, it is superior to myisam in many ways.

Personally I like the row level locks and the way it can enforce foreign keys, which myisam can't do.

</comment></comment></comment></comment></comment><comment><USER21> Why is it that when it comes to databases no one is unbiased?

I cannot find any reliable articles on the web concerning database engines that I look at and trust the author. It's even more difficult considering I am far from a pro with databases but quite simply I won't use any MS products and so I've used MySQL for smaller projects. I've started using MongoDB a lot more lately and regardless of what people (mostly people who've never used it I would imagine) say about it I love it.

</comment><comment><USER22> Comparing apples to oranges really, but I suppose it's more accurate if you take into account all the people who are (or could be using) a relational database without much issue and then switching over.

The article is right though, I've heard scaling MongoDB into the hundreds of gigs to terabytes is an absolute nightmare for the operation team, the software is actually very cool when dealing with reasonably sized data that has an elastic structure.

There are other "NoSQL" databases out there that pay more attention to scale, like HBase. My HBase cluster is just over a TB compressed, consistently churning 6k requests a second, without an issue.

</comment><comment><USER23> > "In my eyes, MySQL has done great harm to SQL because many of the problems > people associate with SQL are in fact just MySQL problems"

That's a very strong and subjective statement. If any, resources heavy databases like Oracle or PostgreSQL, had brought more users to NoSQL than MySQL. That's doesn't there is something wrong with those databases, only that they weren't the right tool for the job is some cases.

Also "MySQL problems" are most of the time due to poor usage, not to the database system itself, actually when used properly MySQL/MyISAM is a great tool

<comment><USER24> That is pure FUD. Postgres is no more resource heavy than MySQL (the stock config will only use 32MB of ram!), and will perform better on many real world workloads.

<comment><USER16> I think it's not resource heaviness per se so much as high latency (and low user-friendliness) in a dev configuration. Postgres takes noticeable time to start up, both server and client, and the client feels less responsive; its commands are also more arcane (e.g. mysql's "show tables" is something like "\d"). Postgres is quite possibly better for a "production" configuration, but it's much slower to develop with, so developers get to thinking of it as "slow".

<comment><USER24> I've never noticed latency issues.

Edit: Timing

Running the client: 12ms Server Startup: <400ms (due to the way OS X services work hard to time directly) Server Shutdown: 925ms

Those are on a 3 year old Mac Mini - hardly a stud machine.

Why are you using the command line client anyway? You know pg_admin exists and is free and runs everywhere, right?

</comment></comment></comment></comment><comment><USER25> I think a lot of the NoSQL hate comes from people who don't understand the use-case, and don't build High Availability/Low Latency systems. A lot of the things I work on require pre-summarized data to provide fast response times at scale (aggregating normalized data would be too slow) and a NoSQL DB (not necessarily Mongo, I prefer Couchbase for most things) with a simple but flexible data model that gives more control to the developer Just Makes Sense.

<comment><USER26> is couchbase advancing? they have no release in a long time(excluding the mobile_stuff) ?

<comment><USER25> Last release was 2.1, in June afaik

</comment></comment></comment><comment><USER9> You would think someone who claims to be an expert on databases would have a clue about his industry.

MongoDB is a document database and is as different from almost every other NoSQL database as it is from every SQL database. It very much stands alone and requires your domain model to be structured in a particular way. To say that it "represents" NoSQL is ridiculous. And to act like a guide on how to scale MongoDB to store 100GB is a problem is also ridiculous. I can create domain models that almost every SQL database would struggle with that MongoDB could breeze through and vice versa.

And seriously anyone claims Cassandra or Riak are misspent adventures are simply delusional. They are solving real world problems in particular around horizontal scaling today that could never be done as cheaply or easily as before. A master-master cluster that costs nothing, scales linearly and can be managed by a developer. Would love to know what product existed years ago that could do that.

Likewise I take exception with the criticism of MySQL. It is an easy to use, manage and install and has the best tooling bar none. It does what it is intended to do perfectly. Some people who deal exclusively with ORM layers will never see the imperfections and just see the ease of use.

<comment><USER12> > I can create domain models that almost every SQL database would struggle > with that MongoDB could breeze through and vice versa.

I am always interested in learning about models that SQL/"traditional relation model" can't easily (or less easily) represent or query. Last time I asked someone pointed me to Datomic's EAVT (entity-attribute-value-time) model as a good example.

Do you know of any more than EAVT?

<comment><USER27> One classic example is tree/graph structured data.

I've worked extensively with modeling and querying medical concepts and relationships in RDBMS. I realize there are tools like recursive common table expressions an materialized paths that can aide querying such data, but now that I'm working at a different job using neo4j, I can see how much simpler the medical informatics domain could be modeled and traversed in a graph database.

</comment></comment></comment><comment><USER28> If the article assertion is true, then what is the Postgresql of NoSQL? In OSS RDBMSs, Postgresql quickly rose to the position of best designed, less quirky contender (albeit waay slower than MySQL in the past).

Is there a NoSQL equivalent?

<comment><USER29> >(albeit waay slower than MySQL in the past).

That isn't even remotely close to true. Single threaded "insert 10,000 rows" benchmarks had postgresql slightly slower, not "waay slower". Concurrent access benchmarks have always had postgresql faster than mysql.

<comment><USER28> I'm old. When I say in the past, I mean pgsql 4 ;-)

<comment><USER29> I'm old. I know that there was no postgresql 4. The first release of postgresql was 6, and what I said was accurate as of the release of version 6, in 1997 or 1998.

<comment><USER28> [slow bow, with flamboyant hat salute] :)

I never expected the very geek joke to be understood. Very cool!

On a more serious note, I didn't mean to support the ever old myth that postgresql is slower than mysql. It hasn't been the case for over ten years, and all the while being a better database in every aspect.

However, go back enough and it was slower (than mysql) for real-world loads. You could get near enough if you disabled fsync, but then were negating many of the advantages of pgsql. Even then, in the old days, table based locking would kill a real database. Table based locking was solved early enough ('98? '99? around that time). Fsync was solved with WAL, in the performance effort of the early 7.x series. It was around late '2000 that I came back to postgresql for good. Until then, it was either mysql for a quick hack or oracle for anything serious. Since then, it's postgresql for everything (I don't deal with the scenarios where Oracle still leads).

</comment></comment></comment></comment></comment><comment><USER30> So then does this mean that PostgreSQL is to SQL like CouchDB is to NoSQL?

</comment><comment><USER31> I can tell you from experience that joins are also slow in Oracle Database Enterprise Edition. Joins are slow.

A nicely normalized relational data model has many advantages, but speed is not one of them.

</comment><comment><USER32> Maybe that explains why Oracle is pushing MySQL, then. See? Open Source DB bad, must get the real thing from Oracle!

Or just use PostgreSQL :-)

</comment><comment><USER33> No it's not . Mongo sucks. MYSQL is a great tool.

<comment><USER34> Not even Apple/Google threads have this much fanboisms and useless comments.</comment></comment>


How fast is the Internet at Google? Mind blowing. - <AUTHOR> http://thenextweb.com/shareables/2011/03/29/how-fast-is-the-internet-at-google-mind-blowing/ <comment><USER2> When I read these I'm amused that people are surprised by Google's network. Its actual capability is nominally secret (my employment agreement and exit agreement both mention restrictions on revealing non-public details of Google's infrastructure) Except in this case there many publically available clues which should give you an idea here.

Consider that Google is investing in undersea cables [1] which nominally provide multi-terabit speeds between continents. And Google is cheap, really cheap, in terms of what they will pay for infrastructure which is why they started making their own switches [2] because even commodity ethernet vendors weren't cheap enough (although these days of state funded cyber armies one also has to protect against foreign made equipment [4])

What is more the amount of data you can send over a fiber is getting faster all the time [3] so replacing the inter-datacenter lasers can quintuple bandwidth.

There have been some interesting 'escapes' of course, once when one of the data centers became unavailable and all the YouTube traffic started coming from another one very far away serving a very large local population, and latencies went up but throughput didn't change, I know a couple of network administrators did a double take on what they were seeing :-)

[1] [http://www.google.com/intl/en/press/pressrel/20080225_newcab...](http://www.google.com/intl/en/press/pressrel/20080225_newcablesystem.html)

[2] [http://gigaom.com/2007/11/18/google-making-its-own-10gig- swi...](http://gigaom.com/2007/11/18/google-making-its-own-10gig-switches/)

[3] [http://www.newscientist.com/article/mg21028095.500-ultrafast...](http://www.newscientist.com/article/mg21028095.500-ultrafast- fibre-optics-set-new-speed-record.html)

[4] [http://www.popularmechanics.com/technology/gadgets/news/4253...](http://www.popularmechanics.com/technology/gadgets/news/4253628)

<comment><USER3> While on the topic of YouTube... why does YouTube constantly buffer these days?

<comment><USER4> Your ISP? Youtube takes 10-20% of typical internet traffic (more than bittorrent) and competes with telecom TV offerings.

<comment><USER3> My ISP doesn't throttle, which is entirely why I picked them.

I have access to a connection that downloads 20 megabytes per second from S3, and YouTube still buffers on that connection.

Note that Vimeo doesn't buffer, and Vimeo has higher video quality usually. Also Netflix doesn't buffer.

YouTube has been getting worse with time rather than better. Tons of ads and slow video.

The issue is definitely on the YouTube end, and I'm not the first person to notice it.

<comment><USER5> > My ISP doesn't throttle

That doesn't mean that they have infinite pipes to Google's edge caches, though (you're not exactly the only one pulling data through those pipes :-). Or that they're even pulling their data through the right pipes. I suggest talking to your ISP. If enough users complain to them, and the problem isn't in their network, they'll complain to Google.

<comment><USER6> It's not just one ISP. I've seen the behavior described by the parent from multiple ISPs in separate countries. It's been driving me nuts for so long that I started collecting information about the connections I've seen it from. Years ago, on 512kbit DSL, it was _MUCH_ better.

PAETEC DS3 (45Mbit/s) in San Diego, CA

Time Warner cable modem in San Diego, CA (20Mbit/s)

Colocation at AmericanIS.net (1Gbit/s via Level3.net) in San Diego, CA

Videotron DSL in Montreal, QC (10Mbit/s)

Teksavvy DSL in Montreal, QC (10Mbit/s)

AT&T 3G in San Diego, Detroit, Atlanta (varies)

Verizon 3G in San Diego (varies)

Rogers 3G in Montreal, QC (varies)

</comment></comment></comment></comment></comment></comment><comment><USER7> _Now, let's put these numbers into perspective. According to Ookla's Net Index, the average speed for a country in Europe is around 12 Mb/second._

That's partly because most people are happy with such speeds (or at least the speed/price ratio). At home I have a subscription with 120MBit/s downstream. Speed tests usually give 130MBit/s during daytime and 140MBit/s at night.

Given that, 500MBit/s downstream is the very minimum one would expect from one of the leading Internet companies, probably located closely to one of speedtest.net's nodes.

<comment><USER8> that sounds like an awesome investment, what are you subscribed too?

<comment><USER9> He must be from Northern Europe. Scandinavian countries have the best deals with internet connections.

As for the numbers, I think 12Mbs is quite an overestimation of reality: <http://www.speedtests.net/world/europe/>

In the UK the fastest one can go is with Virgin 50MB. Too bad then they force you to use their own cable modem which is a piece of shit.

<comment><USER10> > In the UK the fastest one can go is with Virgin 50MB

. . . well, unless you move to Bradwell Abbey or Highams Park: then you should soon get FTTP at about 100Mbps. An Openreach guy was telling me this afternoon it should just be a few months.

<comment><USER11> We (Bournemotuth) were due to be getting Fibrecity, 100mb to the home but it looks like they switched on a few roads, dug up a whole lot more then tanked. There original big fanfare plan was to route the fibre through the sewer system rather than digging up the streets again...doesn't seem that panned out too well!

</comment></comment></comment></comment></comment><comment><USER12> Obviously this screenshot was taken before they went over their 250 GB quota and began to be throttled mercilessly.

</comment><comment><USER13> Sigh. I invite you to semi-rural America: <http://www.speedtest.net/result/1284417179.png>

I live 20 minutes from one of the largest airports in the country.

</comment><comment><USER14> What's funniest is that if you look closely at the Speedtest.Net image, Google only gets a 3 1/2 star rating. No one is ever happy I guess.

<comment><USER15> That rating is for Google as an ISP. They provide free WiFi to the city of Mountain View which isn't always fast depending on where you are. That probably accounts for the low rating.

<comment><USER16> It's capped at 1mbps as well, even if you get a good signal.

</comment></comment></comment><comment><USER17> It's a bit depressing to think that some day we'll all have connections this fast, but unless we learn to violate the laws of physics, we'll never have single digit latencies around the globe. I like to imagine technology _always_ getting faster, but the speed of light really starts to limit things.

<comment><USER18> I had a site user in the Netherlands insist that he should be getting 10ms response times from our server in Seattle. I told him that in AMERICA we obey the laws of physics.

</comment></comment><comment><USER19> This is impressive, but it loses a little luster when you realize their proximity to the testing server.

SpeedTest's SF server is hosted by MonkeyBrains.net. A quick traceroute shows their servers are just one step away from Cogent's SF backbone. I suspect whoever ran this test has fewer than four steps between them and the testing server, and the slowest link between them is the gigabit ethernet port to their desk.

</comment><comment><USER20> This reminds me of the good old days, the 90s, when I was a teenager and finally upgraded to a ~50KB/s cable modem from my puny 33600 baud modem. That was a game changer, I even ran an FTP server for a while =) It's been getting faster ever since then, but I never again had that feeling of "OMG it's so fast let's download something for the hell of it".

<comment><USER21> My first job was in that era at my hometown ISP. I got a discount on shotgun modems. We could pirate Doom and Photoshop 3 on the T1. Nothing ever seemed that fast, even now when I have a 35 mbps fios line.

</comment></comment><comment><USER22> The Gathering, a large LAN party in Norway had a 100Gbit connection. As one of over 5000 participants, I achieved this: <http://speedtest.net/result/1260898925.png>

</comment><comment><USER23> Is the _average_ US home internet connection really 10Mbps?? I find that hard to believe.

<comment><USER24> Same. What about all those people still on 1.5-6mbit DSL, or... _shudder_ dialup?

<comment><USER25> I think most people have cable, which sometimes gives pretty good throughput.

I value customer service over raw speed, so I have Speakeasy DSL. It's not the fastest, but it is always up.

</comment></comment></comment><comment><USER26> And now for a bit of perspective, here in India I am working with a 512kbps connection, which is the only one I can get without any 'fair use' caps. The latency for the fastest DNS is about 350ms, and on weekends, my bandwidth can be as low as 200kbps.

<comment><USER27> Are you in a major city or a more outlying area? Is that a wired, or wireless, connection? Even wired access could have wireless backhaul... assuming yours is wired, is that the reason for the slowness, or have they really been mad at work all over India laying conduit with copper instead of fiber?

<comment><USER28> It is is same all over the country. That's the only plan without any 'fair usage cap'. You might get 4Mbps for $55, but after 15GB of usage, the bandwidth will be cut down for the remaining month.

Also, 3G just launched in India. No more than 50 cities have it, and the maximum they can provide is 32Mbps, again with some fair usage cap!

</comment></comment></comment><comment><USER29> I think the fastest connection I ever had was back in the day when I worked at CERN. They host(ed?) a European internet hub, so rates were pretty damn quick.

<comment><USER30> What were the rates like, if you can recall?

<comment><USER29> To be honest I can't remember. This was 10 - 12 years ago though so what was absurd then is probably pretty standard now. It was a heck of a lot quicker than the T1 line we had in the Physics dept back in London :)

</comment></comment></comment><comment><USER31> Sorry, mind blowing? Plenty office buildings and universities have faster links than that. I could rent an 1 GBit/s fibre to my office for ~3000 EUR/month.

Also couldn't he come up with a more meaningful test than a questionable speed-tester site...

</comment><comment><USER32> From the University of Toronto campus, all requests to www.google.com go through Canarie, the local hookup to Internet2, I think.

This means all my searches are very fast, and also means when our main provider (Cogent) is down, we still have access to google and their web caches.

</comment><comment><USER33> Of course it's fast. They cache the whole thing :)

<comment><USER34> wget internet ??

</comment></comment><comment><USER35> Am I the only one who thought they meant latency? I like my speeds well enough, but I could use some reduced latency. I guess there isn't much one can do about this, though.

<comment><USER36> Their ping is 3ms. Pretty latency-free :)

(FWIW, my ping is 450ms)

<comment><USER35> Their ping to what?

<comment><USER37> To a server in SF located < 50 miles away.

</comment></comment></comment></comment><comment><USER38> Is this really "mind-blowing"? I can upgrade my personal home fiber to 400/400Mbits for 6000NOK (1100USD) per month here in Oslo. Expensive yeah, but not too unreasonable.

</comment><comment><USER39> Wow, I did not realize my internet connection at work is actally that good... I ran speed test before some month ago and it was in the same range. (see <http://speedtest.net/result/1147935298.png>). Back then I though "not bad". But that a post about something like that would make it the homepage of HN...

<comment><USER40> Makes me feel like a tortoise by comparison <http://www.speedtest.net/result/1284419120.png>

</comment></comment><comment><USER41> At my office, we're struggling with a 0.25Mb up connection. What I wouldn't give for a single percent of google's connection speed...

</comment><comment><USER42> I find faster internet stops meaning anything once I have reached speeds that allow 100MB downloads in less than 30 seconds, and the connection provides < 100ms latency.

I am not every user, but I personally find that I only rarely download files larger than 100MB, so anything faster is generally underutilized.

</comment><comment><USER43> I usually can't get those kinds of speeds from most of my servers in datacenters generally.

Speedtest uses servers in the same city, and game downloads probably do as well (CDNs) but the average case is likely significantly slower.

</comment><comment><USER44> I beat google on Upload to the same server: (316mb down/ 165 up / 3ms latency) <http://www.speedtest.net/result/1284509371.png>

</comment><comment><USER45> Good but nothing special, i live in lithuania and i download files at 140MBit/s .I am using cheapest my internet provider plan and if i want to pay more i could get faster internet.

p.s. i pay 17usb per month

</comment><comment><USER46> In my experience, it's only relative to the speed of the server that you are connecting to. I have a 20megabyte (not megabit which is a common misconception) circuit but usually can only get 5-8MB of bandwidth on average from the server that I am connecting to.

We didn't notice much of a difference when we upgraded from 10MB to 20MB for this reason. We have more pipe to download concurrently but doesn't necessarily mean that the "internet" is going to be "faster"

</comment><comment><USER47> Came across these sorts of speeds when FTTP (Fibre To The Premises) was first released for domestic use in the UK last year, <http://alicious.com/2010/400mbps-internet-via-fttp/>.

For that blog post the writer compared the potential 400Mbps to the 40Mbps of a blu-ray disc.

This seems to be a precursor to widespread use of cloud based disc access for general computing with a relatively small SSD buffer?

</comment><comment><USER48> I'm more impressed that speedtest's server can push data upstream that fast, unless I'm mistaken about how the test works.

</comment><comment><USER49> 530/160? I happen to live in this country: [http://arstechnica.com/tech- policy/news/2011/05/want-super-f...](http://arstechnica.com/tech- policy/news/2011/05/want-super-fast-broadband-try-lithuania.ars)

300Mbps for about $30 a month. Well, I don't have the best plan, so that will be 100 for ~$20.

</comment><comment><USER50> I get similar speeds at work ( financial company in NYC ). Can't download any games though :-(

</comment><comment><USER51> While this might seem slow, I am using WiMax so this is at the high end of what I usually see. Also my upload is capped at 1MBit/s. <http://www.speedtest.net/result/1284650149.png>

</comment><comment><USER52> You'd be surprised how much of a productivity bottleneck "slow" internet can be.

</comment><comment><USER53> I don't get the point of this post. If the screenshot said MB/s then that was something. My Apt has a corporate internet freely available for the residents and I always get 500 - 700Mbps.

</comment><comment><USER54> IIRC the internet felt faster in my apartment in Palo Alto (100mbit burstable to town fiber) than on the Google campus.

Yahoo's office connectivity was pretty great, though.

Maybe they are messing with people?

<comment><USER55> Were you on the guest network in Google?

<comment><USER54> No. I worked there for over a year.

</comment></comment></comment><comment><USER56> Any of us could have that if we felt OK paying for an OC3 or OC12 connection at your home or office, right? I'm sure plenty of companies and universities have such service.

</comment><comment><USER57> It would be interesting to see if someone from SG or South Korea can chime in - i lived in SG 10 years ago and got 100Mbps downstream consistently.

</comment><comment><USER58> That's pretty crazy, I get 6Mbit on a good day.

</comment><comment><USER59> See the recent news about Lithuania, FTTH 10Mb for ~$6/month up to 300Mbps for ~$35/month.

<http://bit.ly/infdfc> (apologies for URL shortener, on mobile).

</comment><comment><USER60> I require stable SSH; I like to listen to the occasional tech podcast or watch some video, but basically I need SSH. So no, not blowing anything here.

</comment><comment><USER44> Internet at Stanford comparable.

</comment><comment><USER61> I suppose they're making some use of their gigantic caches of the whole web as a proxy, too?

<comment><USER62> probably not, why would they want cached (outdated) content unless a particular site is down? Also, it's not like that content lives in servers in their office, it's in data centres all over the world (presumably)

</comment></comment><comment><USER63> Internet at work: 100mbps download speeds when downloading ubuntu.

Its the nature of the game. My company and google both need high speed internet and they will pay for it.

</comment><comment><USER64> That doesn't even make sense. For you to get to those sorts of speeds, you need jumbo frame support. While you can get that taken care of on a LAN/WAN or for Internet2, you can't magically make the Internet support it just by sprinkling Google fairy dust everywhere.

<http://en.wikipedia.org/wiki/Jumbo_frame>

<comment><USER65> On my local lan I am able to get to about 40 MB/sec consistently over a gigabit link, and most of that is limited by the fact that I am reading from my local disk when doing so.

Also, the test probably isn't designed for internet connections that fast which could also lead to funky results.

<comment><USER66> I have a gigabit network at home and I can get up to 80MB/s, usually over 65MB/s sustained. I tried jumbo frames but due to the variety of different devices, I couldn't get everything working on the same MTU, so I had to drop back to 1500. Still get good speeds.</comment></comment></comment>