January 28th Incident Report - <AUTHOR> https://github.com/blog/2106-january-28th-incident-report <comment><USER2> > One of the biggest customer-facing effects of this delay was that > status.github.com wasn't set to status red until 00:32am UTC, eight minutes > after the site became inaccessible. We consider this to be an unacceptably > long delay, and will ensure faster communication to our users in the future.

Amazon could learn a thing or two from Github in terms of understanding customer expectations.

<comment><USER3> I recently stepped into a role with a devops component, and one of my first surprises was just how slow status.aws.amazon.com was to update about ongoing issues. I had to scramble to find twitter and external forums confirmation for the client.

<comment><USER4> What's even worse is that when Amazon finally updates their status page it's usually still a green icon with a little i tick for "information" even if it was a partial outage. It takes a lot for the icons to go red which is what you'd look for if you're experiencing issues.

I do the same thing, often searching Twitter for "aws" or "outage" and find people complaining about the problem which confirms my suspicions. It's a sad state of affairs when you have to do this and Amazon doesn't seem interested in fixing it.

<comment><USER5> If you have a support agreement with them then file a ticket requesting better customer communication and link back here as an example of how to do it right.

I think everyone complains in forums and online but doesn't actually file tickets about it. These things are worth tickets too.

<comment><USER6> I take it you have no experience filing tickets with them. A typical ticket goes something like this:

1\. File ticket.

2\. Wait. Then wait some more. Even if you pay big money for a support contract, they take a long time to respond (often > 1 hour).

3\. Get a response from a first level rep who has no access to anything, has little dev experience, and asks some inane questions which I'm _convinced_ is a purposeful stalling tactic.

4\. Play the dumb question/obvious response dance, waiting an hour or more for a response each time.

5\. If you are lucky (usually a couple hours in now) they acknowledge there's some problem (but never give you any detail) and escalate your ticket to a higher level internal team. If you are unlucky, you are calling up your account rep (do you even have one??) and getting them to harass tech support.

6\. Usually around now the problem "magically" disappears if you haven't already fixed it yourself.

7\. If you are lucky, a few hours, days, weeks later you get a response asking if you are still having the problem? You, of course, are NOT having the problem since you long ago solved it yourself. If you are really unlucky they try to schedule a meeting with one of their "solution architects" who is then going to waste an hour of your time telling you how to properly "design" your software for the cloud (i.e. trying to sell you on even more of their services).

8\. Ticket is closed having never gotten to the bottom of the problem, maybe get a survey.

I've _never_ seen this go down differently. Filing more tickets isn't going to change this. You want to really change things?

STOP PAYING THEM!

If a few mid-sized customers stop paying them and make a big-stink when they do it, then I guarantee you things will change! Until then, they have little incentive to improve and the big customers have a direct line to Amazon so they can circumvent all this crap. It's up to the small and mid-sized customers to push for change and the most effective way to do this would be to spend your money elsewhere.

<comment><USER7> To be honest, I've always found their support to be really good. Sometimes it can be a little slow to start, but I regularly experience technicians that go way above what I would expect to assist me & deliver a great outcome. If other companies in Australia were as responsive as them (e.g. telcos), I'd be a very happy man. EDIT: I'm on Business Support, so maybe that's your issue?

<comment><USER8> I'm on business support too and generally am talking to a rep in minutes. They aren't always able to find the problem before I do, but I always get follow up details later on the how / why that they did determine.

<comment><USER6> I wish our experience was like this. We used to have business level but we dropped it because we weren't getting value for it. Our experience was slightly better when we had it but we still ended up either fixing most problems on our own or waiting them out.

<comment><USER8> How much do you pay per month for AWS? That might be a difference.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> There's no mention of why they don't have redundant systems in more than one datacenter. As they say, it is unavoidable to have power or connectivity disruptions in a datacenter. This is why reliable configurations have redundancy in another datacenter elsewhere in the world.

<comment><USER10> Given the dependency in question is Redis, such a solution is probably exacerbated by the fact Redis hasn't really had a decent HA solution.

This is also hidden by the fact that Redis is _really_ reliable (in my experience at least). In my experience it usually takes an ops event (like adding more RAM to the redis machine) to realize where a crutch has been developed on Redis in critical paths.

<comment><USER11> A lot of tools and services people use either don't have HA at all or don't have a native support for true distributed HA. But that can't stop people from making some HA or alike solution. I am not sure what they use Redis for but along the line of caching and key-value store they must have figured out how to invalidate data, otherwise they'd be running only a single instance of Redis. i.e. they are running "HA" just in a single data center, so logically speaking that's not difficult to port over to another data center.

<comment><USER12> I'm not familiar enough with Redis's clustering features to speak to the exact issues with what you're proposing, but generally speaking, HA is almost a completely different problem than disaster recovery (DR). Sure, the protocol is the protocol, but you wouldn't want to cluster local and remote nodes together for several reasons, primarily latency, security, and resiliency. Performance will suffer if they're clustered together and a single issue could take down nodes in both data centers, which kind of defeats the purpose.

What you really want is a completely separate cluster running in a different data center (site). It should be isolated on its own network and ideally it should have different admin rights/credentials and a different software maintenance (patching) schedule. A completely empty site isn't much use so you'll need some kind of replication scheme. Naturally, these isolating steps make site replication difficult. You might patch one site and now the replication stream is incompatible with the other site. (You can't patch both sites at the same time because the patch might take down the cluster.) Or whatever you're using to replicate the sites, which has credentials to both sites, breaks and blows everything up. You need a way to demote and promote sites and a constraint on only one site being the "master" at a time. What happens if network connectivity is lost between sites? What happens if one site is down for an extended period of time? Maybe you need a third, tie- breaking site?

Once you work through these issues, you are still exposed to user error. Your replication scheme might be perfect... perfect enough that that an inadvertently dropped table (or whatever) is instantly replicated to the other site and is now unrecoverable without going to tape. Maybe you introduce a delay in the replication to catch these oopsies, but now your RPO is affected. Anyway, it's a bit of a shell game of compromises and margins of error.

Source: 10 years designing and building HA/DR solutions for Discover Card.

</comment></comment></comment></comment><comment><USER13> For all that work to be done in just two hours is amazing, especially with degraded internal tools, and both hardware and ops teams working simultaneously.

<comment><USER14> You're absolutely right.

We should collectively be using incidents like this as an opportunity to learn, much like the GitHub team does. Our entire industry is held back by the lack of knowledge sharing when it comes to problem response and the fact that so many companies are terrified of being transparent in the face of failure.

This is very well written retrospective that gives us a glimpse into the internal review that they conducted. Imagine how much we could collectively learn if everyone was fearless about sharing.

<comment><USER15> relevant:

[https://codeascraft.com/2012/05/22/blameless- postmortems/](https://codeascraft.com/2012/05/22/blameless-postmortems/)

</comment></comment></comment><comment><USER16> I don't know enough about server infrastructure to comment on whether or not Github was adequately prepared or reacted appropriately to fix the problem.

But wow it is refreshing to hear a company take full responsibility and own up to a mistake/failure and apologize for it.

Like people, all companies will make mistakes and have momentary problems. It's normal. So own up to it and learn how to avoid the mistake in the future.

<comment><USER2> As I said in another comment, the fact that they found an 8 minute delay from outage to status page update to be unacceptable speaks volumes to how much they value their relationship with their customers.

as an aside I feel that I'm quite fortunate to work in the EST timezone, as their outage apparently started at about 7pm my time. We have a general rule at my company to not deploy after 6pm unless an emergency fix absolutely needs to go up.

I saw the title of the story and said to myself, what outage? :P

</comment></comment><comment><USER17> Does Github run anything like Netflix Simbian Army against it's services? As a company by engineers for engineers with the scale that github has reached, I'm a bit surprised they are lacking a bit more redundancy. Though they may not need the uptime of netflix, an outage of more than a few minutes on github could affect businesses that rely on the service.

<comment><USER14> Google "Netflix downtime" for evidence that Netflix also has outages. Google has outages, sometimes very significant ones of Google Apps. Facebook has outages.

Complex systems fail. Period. All the time. Things like the Simian Army are fantastic tools that help you identify a host of problems and remediate them in advance, but they cannot test every combinatorial possibility in a complex distributed system.

At the end of the day, the best defense is to have skilled people who are practiced at responding to problems. GitHub has those in spades, which is why they could respond to a widespread failure of their physical layer in just over 2 hours.

The biggest win with the Simian Army isn't that it improves your redundancy. It's that it gives your people opportunities to _practice_ responses.

<comment><USER18> More than practicing responses, Chaos Monkey and Failure Injection Testing allow us to verify that we don't have unexpected hard dependencies. Sometimes you find out that your service can't start if another one becomes latent, in which case you can plan for it by adding redundancy/extra capacity, fallbacks or working in degraded mode.

</comment></comment></comment><comment><USER19> Every time I read about a massive systems failure, I think of Jurassic Park and am mildly grateful that the velociraptor padock wasn't depending on the systems operation.

<comment><USER20> Well as long as you're not Samuel L. Jackson in that scenario you should be fine. Ish.

<comment><USER19> Samuel L. Jackson taught me everything I know about ethics in software engineering.

Including the principle that if your software breaks, you're the on who has to go get savaged by velociraptors to fix it.

</comment></comment></comment><comment><USER21> This just shows how difficult it is to avoid hidden dependencies without a complete, cleanly isolated, testing environment of sufficient scale to replicate production operations and do strange system fault scenarios somewhere that won't kill production.

<comment><USER22> Or use the Netflix model: Chaos testing in production.

<comment><USER23> No system is perfect; as you continue to add 9s, the cost increases steeply.

Usually its just cheaper to be down for an hour or two, versus architect for the end of times.

<comment><USER24> > Usually its just cheaper to be down for an hour or two, versus architect for > the end of times

The opposite of this philosophy was the motivation behind creation of the internet in the first place.

<comment><USER25> This seems precisely wrong. Some reading:

[http://web.mit.edu/Saltzer/www/publications/endtoend/endtoen...](http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf)

[https://www.jwz.org/doc/worse-is-better.html](https://www.jwz.org/doc/worse- is-better.html)

[thanks for the hint 'thinkpad20! I don't know what I was thinking.]

<comment><USER24> It is not precisely wrong, and thanks for tricking me into opening an obscene picture at work, asshole.

The internet is designed to be highly fault tolerant, because it was based on an arpanet project to design a network that would NOT go down, even if there was damage to a significant percentage of nodes.

<comment><USER25> The "asshole" in this case is JWZ, [randomly?] switching on the Referer header. Apparently he has a hard-on for HN; he's not the only one, but I won't be linking to his site again. (Although, is that really "obscene"? It doesn't do anything for me?) Try this instead, since Stanford are unlikely to engage in such shenanigans:

[https://web.stanford.edu/class/cs240/old/sp2014/readings/wor...](https://web.stanford.edu/class/cs240/old/sp2014/readings/worse- is-better.html)

It's funny, my original comment had the links in plaintext so copying-and- pasting was required and Referer wasn't involved. I changed that on request. b^)

</comment></comment></comment></comment></comment></comment></comment><comment><USER26> > ... Updating our tooling to automatically open issues for the team when new > firmware updates are available will force us to review the changelogs > against our environment.

That's an awesome idea. I wish all companies published the firmware releases in simple rss feeds, so everyone could easily integrate them with their trackers.

(If someone's bored, that may be a nice service actually ;) )

<comment> vhost- This was one of the toughest things about admining hardware clusters. Firmware updates (and firmware issues) are so hard to track down. It's so annoying. I remember spending a week tracking down an issue with a RAID controller and then spending another day or two on the phone with the vendor trying to get a firmware update so we did not have 2 racks of hardware sitting on a ticking time-bomb.

</comment></comment><comment><USER27> > Remote access console screenshots from the failed hardware showed boot > failures because the physical drives were no longer recognized.

I'm getting flashbacks. All of the servers in the DC reboot and NONE of them come online. No network or anything. Even remotely rebooting them again we had nothing. Finally getting a screen (which is a pain in itself) we saw they were all stuck on a grub screen. Grub detected an error and decided not to boot automatically. Needless to say we patched grubbed and removed this "feature" promptly!

</comment><comment><USER28> You can very clearly see two kinds of people posting on this thread: those who have actually dealt with failures of complex distributed systems, and those who think it's easy.

</comment><comment><USER29> _" We identified the hardware issue resulting in servers being unable to view their own drives after power-cycling as a known firmware issue that we are updating across our fleet."_

Tell us which vendor shipped that firmware, so everyone else can stop buying from them.

<comment><USER30> I'm guessing they didn't disclose the vendor because they didn't want to be sued for defamation.

<comment><USER29> Truth is an absolute defense to libel in the US.

<comment><USER31> It doesn't stop you from getting sued, though, it merely stops you from losing. It's pretty reasonable to want to avoid a lawsuit you're absolutely certain you could win.

<comment><USER29> Vendors very seldom sue customers for publicly saying their product is defective. The negative publicity tends to backfire. Legal action can backfire even worse. If the vendor claims the product isn't defective, they have to prove that in court to win a libel action. That means discovery and examination of the company's internal documents and the complaints of other customers, all on the record.

</comment></comment></comment></comment></comment><comment><USER32> I feel it was good incident for the Open Source community, to see how dependent we are on GitHub today. I feel sad whenever I see another large project like Python moving to GitHub, a closed-sourced company. I know, GitLab is there as an alternative, but I would love to see all the big Open Source projects putting pressure over GitHub to make them open their source code, as right they are big player in open source, like it or not.

<comment><USER33> Git is a distributed version control system. Github is simply a place to host a repository and some issues. There is nothing stopping anyone from pushing to another remote hub for redundancy.

So you want Github to open source where they put your git repo and issues? Who cares about that? It's unimportant because regardless they're still the central endpoint to many open source projects, opened or closed source. If you want open source use Gitlab or any other service that sprinkles extra features around git.

I'll never understand this outrage of dependence on Github when you have a distributed version control system. It's not like it should be on github to setup third party repositories for you.

<comment><USER34> From a developers point of view, you're right. But there are package management systems and other stuff depending on being able to download from Github.

Ofc, Github isn't to blame for this, rather the ones that thought Github would be great to use as a CDN.

<comment><USER29> _" But there are package management systems and other stuff depending on being able to download from Github."_

Rust program building, for example, seems to require that Github be up.

<comment><USER35> Only if you need to fetch new dependencies.

<comment><USER29> Or do a clean rebuild.

</comment></comment></comment></comment></comment></comment><comment><USER36> It must be nice to know that the majority of your customers are familiar enough with the nature of your work that they'll actually understand a relatively complex issue like this. Almost by definition, we've all been there.

</comment><comment><USER37> If only Bitbucket could give such comprehensive reports. A few months back outages seemed almost daily. Things are more stable now. I hope for the long term.

<comment><USER26> Isn't BB's problem basically that there are too many users? GH's outage writeup is cool, because it's a one off and it can be analysed. When BB is just overloaded for a long time and needs more power, it's not going to be very interesting.

(unless I missed some specific non capacity related outages?)

<comment><USER11> Maybe. BitBucket was also an acquisition so for some time I believe there was a lack of resource provided to them and there was a huge technical debt/integration effort required. At this very time, I don't know if Atlassian actually care much about BitBucket. They are probably more concerned about delivering Stash than BitBucket, my wild guess.

I was an active BB user a couple years ago, and the project I worked on would hg clone from BB many times a day so I would be the first one to notice a 503 or whatever error coming from their service. Typically I would see one or two outage per month, some last a few minutes, some last several hours. Most of the time the outage impacted git/hg checkout, so I think that was their technical bottleneck.

<comment> lhc- FYI, Stash is now Bitbucket Server, and the plan as I've heard it is to work towards feature parity between the two.

<comment><USER20> We use Stash and it is surprisingly not bad at all. Github is much more polished but for code browsing and review, it does that which it is supposed to do.

</comment></comment></comment></comment></comment><comment><USER38> Weird that they didn't say what caused the power outage and what the mitigations are for that.

<comment><USER39> I'm also confused about how the racks would lose power. Surely they had UPSes.

<comment><USER7> Generally speaking, I'd recommend AGAINST running UPSes in racks that are managed by top-tier data centres. I've had way more trouble with UPSes misbehaving than I ever have with data centres losing power. EDIT: I'd also point out that 2 hours is a long time to be running on in-rack UPSes. I've usually seen them designed to withstand about an hour, but not much more.

<comment><USER40> The power outage was only brief, enough to halt the servers but, much less than the 2 hour outage window.

</comment></comment></comment></comment><comment><USER41> > Over the past week, we have devoted significant time and effort towards > understanding the nature of the cascading failure which led to GitHub being > unavailable for over two hours.

I don't mean to be blasphemous, but from a high level, is the performance issues with Ruby (and Rails) that necessitate close binding with Redis (i.e., lots of caching) part of the issue?

It sounds like the fundamental issue is not Ruby, nor Redis, but the close coupling between them. That's sort of interesting.

<comment><USER42> No the fundamental issue is that an application should not require any external service to boot.

It has nothing to do with Ruby, or Rails or even Redis. It's just a design flaw of the application, that you often learn the hard way.

</comment></comment><comment><USER43> Um, work from your local cache for a few hours? It's that the one of the main reasons for git?

<comment><USER44> Not all processes that involve GitHub are development processes. I've seen automated deployments fail inside a corporate network when the resident HTTP proxy had a bad day and could not connect to github.com.

</comment></comment><comment><USER45> If you use Redis, you should try out Dynomite at [http://github.com/Netflix/Dynomite](http://github.com/Netflix/Dynomite). It can provide HA for Redis servers

</comment><comment><USER46> I would have expected there to be a notification system owned by the DC that literally send an email to clients saying "Power blipped / failed".

That would have given them immediate co text and not wasting time on DDOS protection

</comment><comment><USER47> So, while it sounds like they have reasonable HA, they fell down on DR. unrelated, I could not comprehend what this means?..: technicians to bring these servers back online by draining the flea power to bring

Flea power?

<comment><USER48> I assume they mean completely disconnect the equipment from ALL external power sources. Typically even when a piece of equipment is offline in a data center, it continues to draw power, and will often keep running systems like DRAC and other management/status tools (since the whole concept of a data center is NEVER having to get up out of your chair, so even a "shutdown" system needs to be able to be remotely started).

Since the firmware had a bug, bad state could be stored, completely removing power may clear that state and appears to have done so in this case. They may have also needed to pull the backup battery, and reset the firmware settings, but I wouldn't presume that just from the term "flea power."

<comment><USER47> sure enough, it's a real term, and it's relatively old.. [http://answers.google.com/answers/threadview/id/185999.html](http://answers.google.com/answers/threadview/id/185999.html)

I have never known what to call this, but have definitely been engaged in draining a few fleas.

Also, I can't believe it's been that long since google answers has been closed..

</comment></comment></comment><comment><USER49> TL;DR: "We don't believe it is possible to fully prevent the events that resulted in a large part of our infrastructure losing power, ..."

This doesn't sound very good.

<comment><USER50> If your plan to avoid downtime is to prevent power outages, you're going to have downtime. All their sentence says is they can't prevent power outages. That's fine, because the other 1/nth of your servers are on a different power grid in a different state.

<comment><USER49> I totally share the same view that to best avoid failure is to embrace it and cope with it.

It is true that all their sentence is about recovery, however, it is disappointing that they didn't mention anything about a redundant datacenter.

</comment></comment></comment><comment><USER20> Anyone have a link to a description of the firmware bug that caused the disk- mounting failure after power was restored?

<comment><USER51> I'm going to guess that these are Dell R730xd boxes with PERC H730 Mini controllers (LSI MegaRAID SAS-3 3108).

A failed/failing drive present during cold boot could cause the controller to believe there were no drives present. To add insult to injury, on early BIOS versions this made the UEFI interface inaccessible. The only way to recover from this state was to re-seat the RAID controller.

There were also two bizarre cases where the operating system SSD RAID1 would be wiped and replaced with a NTFS partition after upgrading the controller firmware (and more) on an affected system (hanging/flapping drives). Attempts to enter UEFI caused a fatal crash, but reinstall (over PXE) worked fine. BIOS upgrade from within fresh install restored it.

From the changelog:

        Fixes:
        - Decreased latency impact for passthrough commands on SATA disks
        - Improved error handling for iDRAC / CEM storage functions
        - Usability improvements for CTRL-R and HII utilities
        - Resolved several cases where foreign drives could not be imported
        - Resolved several issues where the presence of failed drives could lead to controller hangs
        - Resolved issues with managing controllers in HBA mode from iDRAC / CEM
        - Resolved issues with displayed Virtual Disk and Non-RAID Drive counts in BIOS boot mode
        - Corrected issue with tape media on H330 where tape was not being treated as sequential device
        - resolved an issue where Inserted hard drives might not get detected properly.

</comment></comment><comment><USER52> > We had inadvertently added a hard dependency on our Redis cluster being > available within the boot path of our application code.

I seem to recall a recent post on here about how you shouldn't have such hard dependencies. It's good advice.

Incidentally, this type of dependency is unlikely to happen if you have a shared-nothing model (like PHP has, for instance), because in such a system each request is isolated and tries to connect on its own.

</comment><comment><USER15> > Because we have experience mitigating DDoS attacks, our response procedure > is now habit and we are pleased we could act quickly and confidently without > distracting other efforts to resolve the incident.

The thing that fixed the last problem doesn't always fix the current problem.

<comment><USER53> Occam's razor isn't a bad rule of thumb, however.

</comment></comment><comment><USER54> Anyone got a good tl;dr version?

<comment><USER55> Power outage in DC brought many machines down. Redis clusters failed to start owing to disk issues (not cleanly unmounted?). The reboot of remaining machines uncovered an unknown dependency on the machines needing the redis cluster to be up in order to boot.

There were other learning points such as immediately going into anti DDoS mode and human communication issues that didn't realise or escalate the problem until some time after the issues started occurring.

</comment></comment><comment><USER56> What is this "HA" jargon?

I would STFW, but searching for "HA" isn't helpful.

<comment><USER57> You could google "HA", click in the Wikipedia link that shows all the things "HA" may refer to, and deduct that the most logical thing in the list, given the context, would be this link: [https://en.wikipedia.org/wiki/High_availability](https://en.wikipedia.org/wiki/High_availability).

<comment><USER53> Would it have been so hard to just type "high availability" rather than making him feel bad for being one of today's 10,000? [https://xkcd.com/1053/](https://xkcd.com/1053/)

<comment><USER58> Why would you feel bad about not being familiar with an abbreviation?

<comment><USER59> The statement about having poor deductive logic skills was the more insulting part of the post (compared to ignorance of an initialism, which I think you are correct in thinking is insignificant).

<comment><USER57> Oddly enough, I wasn't stating that the person had poor deductive logic. I was stating the exact steps I took to find the answer myself.

</comment></comment></comment></comment></comment></comment><comment><USER60> I seriously doubt this version of the story. While it's possible for several hardware/firmware to fail in all your datacenters, for them to fail at the same time is highly unlikely. This may just be a PR spin to think they're not vulnerable to security attacks.

While this was happening at Github, I noticed several other companies facing that same issue at the same time. Atlassian was down for the most part. It could have been an issue with the service github uses, but they won't admit that. Notice they never said what the firmware issue was instead blaming it on 'hardware'.

I think they should be transparent with people about such vulnerability, but I suspect they would never say so because then they would lose revenue.

Here on my blog I talked about this issue: [http://julesjaypaulynice.com/simple-server-malicious- attacks...](http://julesjaypaulynice.com/simple-server-malicious-attacks/)

I think it was some ddos campaign going on over the web.

<comment><USER61> They're not hosted in multiple datacenters; there was a power interruption in their single datacenter that exposed this firmware bug. The point of this postmortem isn't the initial power interruption but rather its repercussions, why it took so long to recover from and how they can improve their response and communications in the future.

<comment><USER60> Ok...so this is another PR...without admitting the issue. I don't know github's infrastructure, but they have a single point of failure? Last I know, every place these days have backup power especially a datacenter...so those were not working either? My point is that it's much better to be upfront sometimes. In fact github didn't have to say anything about the whole thing since everyone forgot already...</comment></comment></comment>


Show HN: Scrape 400k Walmart prices with open source Pickaxe DSL - <AUTHOR> https://raw.githubusercontent.com/bitsummation/pickaxe/master/Examples/walmart.s <comment><AUTHOR> Download tool at [https://github.com/bitsummation/pickaxe/releases](https://github.com/bitsummation/pickaxe/releases)

</comment><comment><USER2> Cool tool!</comment>


Climate Engineers Sucking Carbon Dioxide from the Atmosphere - <AUTHOR> https://www.bloomberg.com/news/articles/2017-09-05/the-climate-engineers-sucking-co2-from-the-atmosphere-and-making-money-doing-it <comment><USER2> There's one significant thing they're achieving here: putting a price on CO2 in the atmosphere.

Some of the resistance to action on climate change comes from the notion that the cost is unknown, which in turn stems from the fact, that to date there was no scalable way to undo the emissions.

Now that a way has been demonstrated emitters can choose whether they want to reduce their emissions or undo them(assuming the right legislation forcing them to do so is in place). I believe most will go with the former option until it stops being economical.

<comment><USER3> The cost of undoing the emissions only sets the price on CO2 emissions if that cost is lower than the cost of enduring the damage associated with not undoing them.

The cost of enduring those damages may be (and probably is) immeasurably large. But unless we're starting from some kind of reasonable estimate of the cost of the damage, however approximate, it can't be right to say that these guys are setting the price on atmospheric CO2.

<comment><USER2> > The cost of enduring those damages may be (and probably is) immeasurably > large.

This is what I'm getting at. There's resistance because we're not giving any specific numbers, only saying that they're "probably very high". This is far from a convincing argument.

With this technology we can instead say: "Ok, so clearing a tonne of CO2 emissions from the atmosphere is going to cost $X. These are going to be your additional expenses in the foreseeable future."

<comment><USER4> [deleted]

<comment><USER5> The idea is to price CO2 emissions by the cost of CO2 scrubbing. Then a law would be set in place requiring measurement of CO2 emissions - CO2 scrubbed, which is then multiplied by industry median cost of scrubbing, which would be then taxed.

</comment></comment></comment></comment></comment><comment><USER6> I find that the idea of "climate engineers" and "geo engineering" has markers which will make it appeal to the same people currently peddling climate denial.

My running bet/prediction is that

1) the imbroglio in America on climate denial will continue,

2) most of the world will feel happy that they are doing less than it should, but more than the stat

3) Eventually someone will sell the idea of !!GEO ENGINEERING!! to the same people they feed all sorts of Denialism.

This will sell well because it has "WE'LL MAKE THE WORLD GREAT AGAIN! WITH HARD WORK AND GUMPTION", and will of course be subsidized by the Government so it will have "JOB CREATION!" written all over it. I could write a satirical Ad for it today, and be assured that its twin will play for real in 20 years.

While this comes across as deeply cynical (it isn't cynical enough), it's based on the debacle that is Environmental Protection, from before I started reading news papers in the 1980s.

The news today may be dominated by what America is doing, but lets not forget that it was and still is a MASSIVE uphill struggle to get people to care for decades.

And I am ignoring the effects of funding into Climate Denialism , and FUD campaigns.

In the end people are going to always choose themselves over the environment.

In a world where clean coal can be marketed, "Climate Engineering" sounds like an entire industry waiting to be born.

sorry, I wish I had something more optimistic to say.

<comment><USER7> I think a lot of global warming activists/campaigners hate the idea of a technological solution as it won't allow them to tackle things like "global inequality" which they see as important as fixing global warming itself.

<comment><USER6> Oh don't get me wrong. I think that global warming activists have lost, and that the end goal for the planet is set.

I'm not campaigning, I'm cooking near the fire.

I make assumptions about the world, and in the end I realized the best way for me to hold myself honest, is to make a prediction and then see how it pans out.

This is one of the predictions I am confident enough about that I can moan about it on HN.

I hope I am wrong, that either some technique is actually found where pumping MORE heat into a heat engine makes it cooler, or people ditch the idea.

But this is not about conservation or global warming campaigning. I think the ship has sailed on that one.

This is just a prediction about a new product.

<comment><USER8> > _that either some technique is actually found where pumping MORE heat into a > heat engine makes it cooler, or people ditch the idea_

It's not the waste heat that's killing us. Sure, given enough energy usage, just the generated heat would become a problem for the planet, but we're far from there. Like hundreds of years far, at minimum. Right now we're fighting with the heat insulation we inadvertently wrapped ourselves in.

<comment><USER6> Hey thats a nice distinction, thanks. you arr correct.

</comment></comment></comment></comment></comment><comment><USER9> I'm a fan of the Climeworks concept, but the article and the Climeworks website fail to address the obvious question: what's the net sum of CO2 collected to CO2 created in creating the energy needed to power their plants?

<comment><USER10> 50 tonnes a year per machine doesn't sound much, compared to just growing bamboo or something similar.

<comment><USER11> "Each year, a hectare of Moso bamboo absorbs 5.1 tons of carbon"

[http://inesad.edu.bo/developmentroast/2013/05/what-can- bambo...](http://inesad.edu.bo/developmentroast/2013/05/what-can-bamboo-do- about-co2/)

You would need 10 Hectares of land to equal one of these machine. Sounds reasonably impressive to me. Even more impressive if the climework "tonne" is 2200 pounds and the bamboo ton is just 1000 pounds.

<comment><USER2> There goes my genius idea to grow bamboo and turn it into charcoal to trap carbon.

</comment></comment></comment></comment><comment><USER12> There are basically two approaches to geo-engineering, and they both have unpredictable consequences:

* reflect sunlight back into space through "albedo modification", either by spraying sulfates high in the stratosphere to mimic a volcano, or by conducting cloud whitening. complication: it will affect precipitation patterns, and allows the CO2 buildup to continue, meaning a sudden cessation would produce wrenching warming. it also does not slow ocean acidification.

* suck CO2 out of the atmosphere. also known as carbon sequestration. That's what Climeworks is doing, but their method is not the only approach. What they've built is a kind of chemical tree. Another approach is to create algae blooms by seeding the ocean with iron filings, since iron is often the limiting parameter to algal growth. algae: the amazon of the sea. a joker named Russ George dumped a tanker of iron into the northern Pacific a few years ago to show this could be effective, but his experiment was poorly designed. carbon sequestration does not address the underlying cause. but we are losing the battle against the underlying cause, which is basically our addiction to economic growth driven by fossil fuels, and existing carbon in the atmosphere will continue to warm the planet for many decades to come, even we switch entirely to clean energy today. so it's likely that humanity will have to combine carbon sequestration with clean energy to avert even worse climate change.

[http://www.motherjones.com/environment/2015/03/geoengineerin...](http://www.motherjones.com/environment/2015/03/geoengineering- caldeira-climate-change/)

[https://www.sciencedaily.com/releases/2017/07/170724105044.h...](https://www.sciencedaily.com/releases/2017/07/170724105044.htm)

[http://www.climatecentral.org/blogs/iron-dumping-leaves- geoe...](http://www.climatecentral.org/blogs/iron-dumping-leaves-geoengineers- with-egg-on-their-faces-me-too-15147)

[https://www.theatlantic.com/magazine/archive/2009/07/re- engi...](https://www.theatlantic.com/magazine/archive/2009/07/re-engineering- the-earth/307552/)

<comment><USER13> Serious question. Why not just plant a shit load more trees?

<comment><USER14> Costs at scale, mostly. Algea grows faster and doesn't require land.

<comment><USER15> Honestly, how expensive is it to plant trees?

I'm sure a LOT of people would be interested in volunteering for that and while the military is not fighting people they could be fighting climate change, with trees!

</comment></comment></comment></comment><comment><USER16> > Working around the clock, each capture plant can vacuum about 50 tons of CO₂ > from the atmosphere a year, Wurzbacher says.

For a comparison the per capita CO₂ emissions per capita in the US are 20 tons. With an average household size of 2.58 you would need one of those plants + CO₂ storage facilities per household.

<comment><USER2> Or, given that the world annual CO2 emissions are around ~400*10^9 tonnes, there should exist one for literally every human being.

</comment></comment><comment><USER17> Each capture plant takes 50 tons of CO2 out of the atmosphere per year. We are putting roughly 40 billion tons of CO2 into the atmosphere per year. That's only 800 million capture plants needed, or roughly 44 million customers similar to the one they have lined up (using 900 tons per year).

Every little bit helps, and I'm glad they are making money doing this, but it's not a viable solution to the problem as a whole. They also didn't say what powers the turbines in their capture plants.

</comment><comment><USER14> These sorts of ideas are in a weird place. To many who are actively seeking better performance in our carbon reduction efforts, ideas like this could be seen as a threat to those efforts.

ATM, the debate is (1)"is the carbon-climate problem real?" and (2) "If so, how do we reduce carbon emissions?". "Climate Engineering" would complicate 2 a lot, probably splitting political support.

Politically, they open the door to continued or increased carbon emmissions today. They let us avoid addressing the "root cause." This is all regardless of timelines or progress. Just the existence of the idea in public debate might be enough to swing the opinion balance.

I imagine there are some who would ideally like to keep geoengineering as a quite Plan B so as not to disturb current efforts, but that's not really possible.

Overall, I think these ideas (if actually viable) will inevitably enter the discourse in the long term. We are _already_ engineering in some sense. We have models and targets for both carbon and temperature. IE, we want to take control (to a small extent) of the climate. That's engineering, and engineers always look for more tools eventually.

<comment><USER18> There is no debate and no climate scientist sees this as a threat. Reduction is needed but it won't cut it its too late for natural carbon sequestering at its normal pace. People who work on reduction also often work on sequestering and if not do not see these efforts as a threat to any of their work.

<comment><USER14> Sure there's a debate. We're debating and discussing right now.

<comment><USER18> For the most part we aren't climate scientists, there is absolutely no debate about carbon emissions and climate change.

</comment></comment></comment></comment><comment><USER19> Wouldnt it be cheaper to first capture CO_2 from the output of coal plants and other industries where it is in concentrated form? Also, economies of scale. Once we bring those down,we would still have to deal with decentralized sources like cars(electric cars and renewables can help), container ships, and the meat industry.

<comment><USER20> Agreed - check out Opus12 for an example of point-source capture: [https://www.opus-12.com](https://www.opus-12.com)

Would be relevant to get Climeworks' perspective on the question of point source vs atmospheric carbon dioxide.

</comment></comment><comment><USER21> For some reason I thought the runaway in temps will come from water vapor from melting permafrost, which is starting now. In other words, the CO2 ship has sailed, no?

<comment><USER22> My hope is that this study is correct: [https://m.phys.org/news/2017-08-hydrate-gun- hypothesis.html](https://m.phys.org/news/2017-08-hydrate-gun-hypothesis.html)

</comment></comment><comment><USER3> > Working around the clock, each capture plant can vacuum about 50 tons of CO₂ > from the atmosphere a year, Wurzbacher says.

And then from [[https://www.co2.earth/global- co2-emissions](https://www.co2.earth/global-co2-emissions)]:

> Global carbon (C) emissions from fossil fuel use were 9.795 gigatonnes (Gt) > in 2014 (or 35.9 GtCO2 of carbon dioxide).

This doesn't seem like a great solution to me.

</comment><comment><USER23> dumb question: say they take xxxx tons from somewhere in Switzerland. How long does it take for the air to mix with the rest so the co2 is averaged worldwide?

This looks like a great thing. The state can tax emissions and force emitters to pay such carbon gathering companies to become carbon neutral. You release xxxxx tons a years, show that you sequestered at least the same or pay a tax.

You could also have a Bill Gates type donate all his money to a such company. Or the government can fund them. If this works, it's just a matter of cash and we have it /can find it.

<comment><USER20> How long does it take for air to mix worldwide?

I visited NOAA's Mauna Loa Observatory in Hawaii last month. It's one of the key global sites where carbon dioxide is monitored. The scientist giving the tour answered this, and I believe it was on the order of a few weeks for mixing to occur.

Here's a visualization that starts to answer your question: [https://amp.livescience.com/48798-carbon-dioxide-global- comp...](https://amp.livescience.com/48798-carbon-dioxide-global-computer- model.html)

I would love a more detailed answer to this question as well.

(I visited the Mauna Loa Ovservatory because of our open source project to track carbon dioxide worldwide: [https://gitter.im/giving-a-fuck-about- climate-change/Lobby](https://gitter.im/giving-a-fuck-about-climate- change/Lobby) [http://carbondoomsday.com](http://carbondoomsday.com))

<comment><USER24> That is a nice link (showing global CO2 estimates in 2006). It illustrates how turbulence mixes the atmosphere at global scales.

The video mentions that better measurements will result from the OCO-2 mission, which launched in the meantime and has been measuring CO2 in ~3km^2 footprints across the globe since 2014. A result is:

[https://svs.gsfc.nasa.gov/4514](https://svs.gsfc.nasa.gov/4514)

which shows vertically-resolved CO2. Both videos make it clear that CO2 is generally well-mixed both horizontally and vertically in the atmosphere, with the exception of Northern vs Southern hemisphere. I believe CO2 emissions from China, for example, take on the order of a week to propagate across the Pacific and reach the Western US.

</comment></comment></comment><comment><USER25> People seem upset that this is working? Maybe because this provides a solution to the problem that doesn't mean we have to fundamentally change society, or bring down the 1st world, or say that "America was the bad guy to ignore all the EU regulations".

<comment><USER26> \- CO2 is plant food. Indoor growers supplement CO2 levels up to 1500 ppm (after which they say that there is not much return) to enhance growth. There is no reason to remove it from the atmosphere.

\- Supposedly CO2 is also a green house gas that helps earth trap some heat from the sun. The link between how much heat CO2 captures based on the concentration is very difficult to find, suggesting that the link is not very strong and propaganda driven "climate science" deliberately doesn't want anyone to know. My guess is that it is very weak at best which is why they supplement the theory with heating will cause water vapor concentration to rise, which is also a green house gas, which will cause snowball effect. This could be tested in a controlled greenhouse but most probably it didn't turn out to be favorable for the propaganda so the research has not been published.

\- Climate has been changing throughout the history of the earth and there are no known consistent patterns that we have been able to model which can help us predict anything, not that we have seen working at least. Overall though, it has been cooling and there is no reason that it will not continue to do so in the long run.

\- The causal relation between CO2 levels and global temperatures has not only not been established, but it is impossible to establish because of large number of factors affecting it and affecting each other. The best we can get is some kind of correlation but even that hasn't been consistent over time. They keep adding more explanations to anomalies to pretend that they know what is going on but it's obvious that they just don't want to accept it. Max Plank said "Science advances one funeral at a time", but it seems increased life expectancy will keep "climate science" going for a while.

So, yeah some people may be upset that people are touting this solution to a problem that doesn't exist.

<comment><USER27> [https://xkcd.com/1732/](https://xkcd.com/1732/)

<comment><USER26> That's all nice and good, except it assumes that CO2 emissions is the cause of warming and no such causality has been established.

<comment><USER27> A single chart can't possibly explain the entirety of climate change. I was just addressing your points that climate change in the past is like what we're seeing now and that the current trend is cooling.

<comment><USER26> It's very convenient to start from the ice age and show how it's warming up. Here is a much longer timeline: [https://en.wikipedia.org/wiki/File:All_palaeotemps.png](https://en.wikipedia.org/wiki/File:All_palaeotemps.png)

<comment><USER27> What do you think is the correct staring point? Obviously if you go back far enough the earth was just a hot ball of lava, and temperatures have been below that ever since.

Showing that temperatures have been relatively steady for a big chunk of human history/prehistory and then suddenly started shooting up right when we started burning mass quantities of fossil fuels seems most relevant to me.

<comment><USER26> >relatively steady for a big chunk of human history/prehistory

Please refer to the linked graph again. Homo sapiens evolved around 200k years ago. You can see endless spikes during and before that time. Climate has been relatively "steady" for ~10k years after it recovered from an ice age. If the latest is due to "human activities", what about all the countless spikes which have occurred forever?

<comment><USER27> When was the last spike that was this fast?

<comment><USER26> The current data is the only recorded data that we have which is why we can see it fine grained enough to know how "fast" it is. All the previous temperatures are speculations and it is impossible to know if the rise and fall has been this fast or way much faster.

Anyways, the fact of the matter is, all the climate studies have shown us is that climate is very unpredictable. All of the predictions based on various modeling of historical climate data have failed. And the reason is simple. The climate system is too chaotic. A butterfly flapping its wings can lead to a hurricane... or not, but nobody can predict it based on the observation of the butterfly or all the events in all parts of the world for that matter. Even if we had those data points of all butterflies flapping wings and all koalas farting, it would be impossible to compute what would happen in a very long time from now (a year or decades or centuries). Anyone who tells you otherwise is a snake oil salesman and it's pathetic to see reasonable people fall for this nonsense.

The reason it is still popular despite easy debunking is because why people fall for this has nothing to do with science. There was negligible data and climate models in the 70s and 80s but there was no less enthusiasm in people who wanted to shut down big corporations for "harming the environment". All of the "science" people reference now came very recently and not without organized effort to "prove" that it is the case while silencing and threatening anyone who disagrees by using ecoterrorist organizations and political slamming.

The main idea behind the religious following of climate change narrative comes from the fact that certain people are rich and they somehow should be doing something bad to be that rich, and being rich is bad so they should be departed of their money. That is all there is to it.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER28> I've always imagined blimps floating around and filtering our air above all the major cities and places where air pollution/CO2 density is worst. Does anyone have any reasoning as to why this isn't really being done?

<comment><USER29> The magnitude of the emissions. We are putting tens of billions of tons of CO2 in the atmosphere per year. Even assuming the filtering process is very efficient (net-carbon negative) where do we store that in the blimp? How many blimps would we need?

As a thought experiment - look at the average car - a Toyota Camry for example. That car emits 206 grams per kilometer. Over the course of the car's lifetime (about 200 thousand kilometers or so) that's 41 thousand kilograms of CO2. For one family car.

</comment></comment><comment><USER30> Do they have plans to remove Methane and NOx. Methane emissions have increased and we are seeing weather changes. One shouldn't forget CO2 is also needed in the atmosphere.

<comment><USER31> You don't need to remove those. You just need to make sure they get them wet, so entropy can follow its inexorable path.

<comment><USER30> Getting wet alright, Irma probably does it.

</comment></comment></comment><comment><USER32> I wonder how much use CO2 extracted from the air has. Greenhouse CO2 supplementation sounds interesting but I assume there are cheaper ways to produce CO2 already.

<comment><USER33> Presumably, most of the captured CO2 eventually ends up back in the atmosphere anyway - either through leakage, or eventual decomposition of the greenhouse plants being grown.

But it's still a net gain if the CO2 would otherwise have come from fossil- derived sources?

<comment><USER34> Some work has been done on mineralising extracted CO2, ending-up with a stable form that doesn't return to the atmosphere. For example: "... use electrolysis or heat to split common salt, sodium chloride, into hydrochloric acid and caustic soda, then react the soda with the carbon dioxide to make bicarbonate." [1]

The obvious question is whether this can be done in an energy neutral way. Either way its just a fix: we need to stop putting the CO2 into the atmosphere in the first place.

[1] [https://www.theengineer.co.uk/issues/25-april-2011/solid- as-...](https://www.theengineer.co.uk/issues/25-april-2011/solid-as-a-rock- mineralising-carbon-dioxide/)

<comment><USER35> > is whether this can be done in an energy neutral way

Wind, PV and waves can be employed for this. You don't need a constant supply and can control capture rate to match available energy.

For many uses, renewables like these are already perfect.

</comment></comment></comment></comment><comment><USER36> Is this cheaper than growing wood and turning it into charcoal? That doesn't take energy (in fact you get some out), it only takes time and land.

<comment><USER26> I have a better alternative: growing excess food and dumping it on the ground.

<comment><USER37> Char takes a lot longer to rot. So the carbon is pulled out of the short cycle.

</comment></comment></comment><comment><USER38> wow - <sarcasm>This technology is the future</sarcasm> \- as per the article each plant sequesters about 11 Cars worth of CO2 each year. Since there are currently about 1.015 billion motor vehicles (lets assume those are cars) we would only need 93 million of these things - how much energy do they use and can they suck more CO2 running them causes?

</comment><comment><USER39> > quickly bonded over their shared loves for mountain climbing and beer

Didn't realize that was so unusual in Switzerland.</comment>


Here is Why We Invested in BTC China - <AUTHOR> http://lsvp.com/2013/11/18/here-is-why-we-invested-in-btcchina/

<comment><USER2> funniest article of the day</comment>


Google Demanded T-Mobile, Sprint Not Sell Google Fi Customers' Location Data - <AUTHOR> https://motherboard.vice.com/en_us/article/d3bnyv/google-demanded-tmobile-sprint-to-not-sell-google-fi-customers-location-data <comment><USER2> I mean sure. Google certainly doesn't want wireless carriers to compete with them in advertising, and everyone having access to location data from smartphones just devalues Google's own access.

[https://qz.com/1131515/google-collects-android-users- locatio...](https://qz.com/1131515/google-collects-android-users-locations- even-when-location-services-are-disabled/)

<comment><USER3> The big difference is Google doesn't sell that data to third parties; they just use it for ad targeting. There's a big difference between the company that makes my phone's OS knowing my location, and anyone willing to pay a few bucks knowing the same.

(Also it looks like in the situation described in the article you linked, they didn't even use the data for ad targeting: "we never incorporated Cell ID into our network sync system, so that data was immediately discarded, and we updated it to no longer request Cell ID".)

<comment><USER4> As far as the advertising is concerned Google is no different(and so FB etc). Anyone with few bucks will know your location or better said will target you with ads. With Google they won't target only by location because as you said Google makes the OS so it knows pretty much everything about you.

<comment><USER5> Please show me where I can buy thefounder's location data from Google. Which service is that, and how much do I need to pay? I can't find it at [https://www.google.com/about/products/](https://www.google.com/about/products/)

> better said will target you with ads.

The difference between selling location data, and selling ad targeting, is the difference between actively stalking someone, and sending unsolicited spam to "Current resident" at "Every address in a city block".

What harm can I inflict to you, via targeted ads? Some. You can think of some creative ways to do so, but it's not a very effective way to inflict harm.

What harm can I inflict to you, via buying your location data? A hell of a lot.

If you don't trust Google to be a good steward of your data, that's one thing. But let's not conflate selling your data, with selling the ability to mass- advertise to a demographic that includes you.

<comment><USER6> > What harm can I inflict to you, via targeted ads? Some. You can think of > some creative ways to do so, but it's not a very effective way to inflict > harm.

Do you see how people have weaponized social media through data? Google ads might not have been used in a nefarious way just yet but that's likely because it hasn't been thought of.

<comment><USER5> With ratings, on a scale of one to ten, how would you compare the individual harm of someone showing you a demographic-targeted ad[1], to someone having the real-time location data sufficient to start _physically_ stalking you?

What about someone showing you an untargeted ad? Where would that fit in?

[1] Or, if you prefer, a remarketing ad, instead. I have very little patience for those.

<comment><USER7> The harm to me isn't the harm of someone showing _me_ a demographic-targeted ad, the harm is someone showing _everyone_ demographic-targeted ads designed to manipulate public opinion.

Remember how everyone got very hot and bothered about precisely this thing happening in a recent U.S. election?

<comment><USER5> Why are cookie-based demographic-targeting ads that manipulate public opinion bad, while demographic-targeting ads on television, in magazines, etc, are fine? Is it because more targeted ad spend is more efficient?

If your problem is that advertising is used to manipulate public opinion - consider - why do we even allow political campaigns and paid speech? Its only purpose is to manipulate public opinion, and to trick stupid voters into voting for bad people.

I'm afraid your position is not entirely ideologically consistent with how we determine what kind of political speech is allowed, and is not allowed.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> So many people are saying "this is just further evidence of Google doing evil".

Unpopular opinion: I feel like this is further evidence of Google being good stewards of user data.

There are many reasons Google may want to protect this data: They don't want user backlash from revelations like this, they don't want to have to be legislated, they don't want the PR spend of having to recover from such reporting. Or it could be that it is a company built of people who find such protection to be important.

<comment><USER9> Google, data point for data point, is one of the best stewards of data potentially in history.

They have more information about you that they choose not to sell than you can even imagine. They literally set the high-bar for PII and other types of tracking data.

The only ones who know nearly as much are Telcos and they have been openly selling and sharing all of your info to governments and shady shell corporations since day 1.

Edit - Forgot that Facebook is catching up to Google on data and they have been a leaky cup since day one as well.

<comment><USER10> Selling mass location data to the highest bidder is indeed extremely slimy and even a threat to "national security".

But the fact that Google (or any centralized entity) has aggregated so much detailed information on billions of people's personal lives is itself a threat to global/national security.

What happens when that data is hacked/leaked and _then_ sold to the highest bidder (who isn't interested in advertising)? That data could be easily weaponized for a shocking/devastating attack the likes of which we have not seen since the first atomic bomb.

<comment><USER8> Are you familiar with the design of the Google systems WRT securing this data? Based on the revelations Snowden was able to do, and a similar level of familiarity with Google's security measures, it sounds like Google even has the NSA beat as far as keeping data secure. In short: Even the people doing OS and hardware level maintenance of the machines in question don't have access to this data.

But I'm on the outside looking in, someone with more hands on can probably provide more details.

<comment><USER10> >Even the people doing OS and hardware level maintenance of the machines in question don't have access to this data.

Are you implying Google can't access their own data?

100% security is a dangerous myth. No internet connected system is impervious to hackers.

</comment></comment></comment></comment></comment><comment><USER11> How people's sentiments change here based on who tries to protect their privacy. If it's Apple, "wow, they are god's own people with purest intentions of protecting users' privacy". When Google does it : "Yeah, because they want to _sell_ it themselves so obviously they don't want any competition".

<comment><USER12> One is a massive advertising company that makes the vast majority of it's money on advertising and has a history of invasive privacy issues.

The other one AFAIK doesn't sell user data, has publicly denied to help the FBI (Have other companies done that? If not then they assume complied), and has put in a large amount of effort to protect privacy (It's one of the main arguments of why IOS/OSX is so locked down).

Obviously this doesn't mean apple doesn't do bad things behind the scenes, but based on the information available to us is why people trust them more with privacy then google.

<comment><USER13> > Have other companies done that? If not then they assume complied

Google literally lead the charge on this with their transparency report for government data requests and won huge victories in allowing for the reporting of things like FISA requests.

It's great that Apple has followed Google's lead and also publishes a transparency report, but their own report also clearly shows that they _DO_ comply with government requests: [https://www.apple.com/legal/transparency/us.html](https://www.apple.com/legal/transparency/us.html)

And both Google & Apple have comparable percentages of requests honored per their own reports.

Regardless an "assume complied" is an illogical position to take here. Assume a company did work that they didn't have to for one-off requests? That's a safe thing to assume they _didn 't_ do. Assume they did nothing, because that's way easier, cheaper, and simpler for them. Which means assume they didn't help anyone, including the FBI.

> large amount of effort to protect privacy (It's one of the main arguments of > why IOS/OSX is so locked down).

No. iOS / OSX are locked down to prevent competition. There's no privacy benefit in keeping the user from poking at their own device.

Apple did a lot of work specifically to enable apps to do things like track a user's location in the background. Things that didn't used to be possible, but which Apple put in work to do. The only significant thing they did here on iOS was make it a runtime permission, which other platforms have similarly done. OSX continues to have no real restrictions or enforcement for apps, except to try and prevent you from installing them outside of the app store.

They put more effort into bragging about their privacy than they did in actually improving privacy. Advertising turns out to be extremely effective, as Apple frequently proves.

</comment></comment></comment><comment><USER14> Ask HN: Is anyone here happy with Google Fi service? I ordered a Pixel 3 phone directly from Google Fi and signed up for service with Google Fi the end of December. When I ordered the phone it showed up as in stock and estimated I would get my phone by the 4th of January. On the 2nd I got an email saying that the phone shipping was delayed and I would receive an email when it ships. I have heard nothing from google so I called Google Fi support and was told I would get an email from the shipping team letting my know when my phone would ship but that was 2 days ago and have not heard a word from them. I tried following there instructions to cancel the order but when you click the edit order button where they say there is a cancel button none can be found. Anyone else have similar experience? Before I placed my order I was thinking the user experience from Google must be better then T-Mobile or Sprint but I am starting to regret the choice.

<comment><USER15> Sounds like your phone was so delayed because of a huge black friday promotion they ran that put their pixel stock a month or more behind back orders. My experience with them, outside of this one massive underestimate of how many phones they would sell because of the phenomenal deal that they ran, has been really good. Phones are normally in stock and ship almost immediately. And customer service is very responsive.

<comment><USER15> As a side note, the phone that I snagged during that promotion just barely arrived a week ago. And I ordered it at the end of November. I think you just ran into some unlucky timing, at least in regards to the delays in shipments that you encountered.

</comment></comment></comment><comment><USER16> I hate the idea that we're so beholden to one company's ethical compass over another. It's so clear that we're in need for some sort of regulation to great guideposts around what is ok and what is not... sigh... not like the government will ever get its act together on this.

</comment><comment><USER17> Say what you want about Google, its one of the few companies with a moral compass.

<comment><USER18> Dragonfly.

<comment><USER19> Google employees risked their jobs to protest against Dragonfly.

<comment><USER20> people with perhaps the highest demand for their labor than any other professionals risked their cushy jobs to influence their company to not be complicit in the censorship of over a billion people with a regime that has committed genocide? that's superhuman dedication right there, good job Google superheroes

<comment><USER21> There are other tech companies doing related work in China. Are you saying that the employees of those companies are all complicit with genocidal actions?

<comment><USER20> Assisting a regime (which has committed genocide) with censorship isn't the same as assisting genocide. That should be clear from reading my comment.

</comment></comment></comment></comment></comment></comment><comment><USER22> As shrewd and cynical as this move is, it is good. With Google you at least have some options for how much you want them to know about your location (even if they rely on dark patterns still to trick average people into reenabling it)

</comment><comment><USER23> "... as soon as we heard about this practice, we required our network partners to shut it down as soon as possible." Google did not say when it made this a requirement.

Hey, stop doing that, please?

</comment><comment><USER24> Pot, meet kettle. This is rich coming from Google, which doesn't respect users' choices in Android not to collect location data and yet the company wants others not to deal in that data from its other services. Perhaps it's just the worry of enabling other companies to make money off of customer information in any way possible, and wanting to be the only company that should be able to make money from this data. As far as the user is concerned, both are privacy violations and an expression of disrespect to them. The bar on privacy is so low that Google seems to be imagining that its behavior is "better than the others".

</comment><comment><USER25> I guess the attorney general is going to stay silent on this?

</comment><comment><USER26> This seems more of a do not compete attempt.

</comment><comment><USER18> This isn't a win for consumers, it's just the changing boundaries of a turf war.

Just like they made a big stink about getting everyone to HTTPS. It wasn't to reduce 3rd party incidental access to your browsing data. It was to retain their 3rd party incidental access while icing out ISPs.

</comment><comment><USER27> Let's translate PR speak here...

> "We have never sold Fi subscribers' location information,"

In other words "we never directly sold the info, but we never contracted them not to do anything with subscribers' location information". In cases like this, I'd say you are responsible for downstream data use. This isn't some API or hole, this is a large b2b contract and Google should have contractually obligated them to what they could do with their subscribers' data and then sued when they found out it wasn't the case. Why didn't they? Either leverage (Google has to rely on someone), ignorance (doubtful), or apathy (we don't care until the media does). I'm inclined to guess #1 and #3.

> a Google spokesperson told Motherboard in a statement late on Thursday.

In other words, we had to ask and get a small statement because being open, upfront, honest, etc is not their approach.

> "[...] as soon as we heard about this practice, we required our network > partners to shut it down as soon as possible [...]"

In other words "we have asked, refuse to say whether they agreed, refuse to say when it will happen, and in general are as opaque as those we are deflecting towards".

Obviously the cell carriers are bad actors. But deflecting instead of accepting responsibility is bad too. Just admit you have no leg to stand on because you require them, or show us where, in writing, they promised to do this and when. Can't do that? Yeah, because you're not in control and/or your shitty business practices are all behind closed doors. And I don't limit this to Google, this goes with anyone partnering with these companies (especially the more traditionally reticent ones). Be open or get hate.

<comment><USER28> > In other words, we had to ask and get a small statement because being open, > upfront, honest, etc is not their approach.

You don't seem to understand how the press works. When you write an article about a company (or a public figure), you reach out to them for comment.

<comment><USER27> Of course I understand. What I'm saying is that comment/information should be available eagerly, by the company, upfront to all of their customers when they found out about this problem without requiring a journalist.</comment></comment></comment>


Pandemics and Propaganda: How China Shapes Conversations on the Coronavirus - <AUTHOR> https://cyber.fsi.stanford.edu/news/chinese-state-media-shapes-coronavirus-convo <comment><USER2> Our service (feeder.co) got banned in China last week. We didn't make the connection regarding the timing immediately, but it makes sense that they are on the offensive with regards to restricting the free flow of information.

As another slap in the face, Apple also removed our app from the China app store, citing that our app was deemed illegal in China (an RSS reader???). Further cementing my opinion of the dangers Apple's monopolastic position. We asked to see the legal complaint from the government that triggered this, but have not complied.

</comment><comment><USER3> Controlling the narrative 101: People want to talk about this subject, so it's better you tell them how you want them to talk about it.</comment>


How the BSO takes 100-plus musicians on tour halfway across the planet (2017) - <AUTHOR> https://www.wbur.org/artery/2017/10/31/bso-instruments-travel-equipment <comment><USER2> Interesting, but it's nothing compared to a rock'n'roll tour. I've no detailed knowledge of what it's like now, but even 35 years ago we had five 40-foot trailers filled with gear, including 21 cannons and a 3 ton bronze bell, for a tour that did a different venue most nights. Only about 30 people in the tour party, though

<comment><USER3> It's amazing seeing a band with an enormous stage act -- most notably Rammstein -- packing up after a show.

The last time I saw them, it looked like half of the stage was already packed into cases within half an hour of the end of the show. News reports say they had _60_ trucks of stuff, and 200 people to set it up and take it down again.

I'd assume they don't have anything as fragile or intrinsically valuable as an orchestra, but with each show bringing in €3-6M (sometimes more) the value to the tour is greater.

</comment></comment><comment><USER4> I thought just about all bows are made from pernambuco (Brazil tree wood). The article seems to indicate there are travel restrictions with it. I recently bought one for my daughter, and they are not cheap, but they sound much better than carbon fiber.

<comment><USER3> The species is endangered [1], and trade of unfinished wood is restricted [2].

I strongly discourage buying anything made from endangered species -- most people in America or Europe wouldn't consider buying something made from an endangered animal, but don't consider plants in the same way.

There have been discussions about loosening restrictions on musical instruments using small amounts of endangered wood, where the majority of said wood is used in furniture etc (e.g. rosewood on guitar fretboards and furniture). I still discourage purchasing even that, but in this case the restriction[2] only mentions musical instruments, so that is presumably the majority use of this wood.

(These restrictions can change, so it's important to keep up-to-date. Instruments can be seized by customs otherwise.)

[1] [https://en.wikipedia.org/wiki/Paubrasilia](https://en.wikipedia.org/wiki/Paubrasilia) and links from it

[2] [https://www.speciesplus.net/#/taxon_concepts/94093/legal](https://www.speciesplus.net/#/taxon_concepts/94093/legal)

</comment></comment><comment><USER5> Alas, they just cancelled this year's East Asia tour: [https://www.wbur.org/artery/2020/01/30/bso-cancels-east- asia...](https://www.wbur.org/artery/2020/01/30/bso-cancels-east-asia-tour- coronoavirus)

</comment><comment><USER6> Interesting! Why do they bring all of these instruments? For example, are there not timpani available at their destination?

<comment><USER7> > Why do they bring all of these instruments?

I think people are very personally attached to their instruments.

<comment><USER8> Indeed, and in a top-tier symphony orchestra like BSO, many (if not most) of the musicians will be playing one-of-a-kind handcrafted instruments that are hundreds of years old.

</comment></comment></comment><comment><USER9> The article doesn't mention if the musicians are sent in multiple planes, like athletic teams often are.</comment>


Third World Quarterly Publishes "The Case for Colonialism" Leading to Censorship - <AUTHOR> https://legalinsurrection.com/2017/09/third-world-quarterly-publishes-the-case-for-colonialism-leading-to-censorship-demands/amp/ <comment><USER2> _" This is an issue of student safety and having people at the institution who hold views like this does not create a safe campus for everyone."

"This is especially appalling when the author elsewhere in the article takes the words of multiple decolonial scholars of colour out of context in order to justify his violence against their respective communities and cultures."_

I haven't read the whole viewpoint article yet, but I'm pretty sure it doesn't call for violence or threatens campus safety.

</comment><comment><USER2> Website doesn't load on my end, unless I remove the "/amp/" from the URL

edit: Seems to be a problem with my adblocker (uBlock)</comment>


Is Kaspersky Lab a Victim of a Cold War Witch Hunt? - <AUTHOR> http://www.cbronline.com/news/cybersecurity/protection/cold-war-kaspersky-lab-security-software-us-government/ <comment><USER2> True, I agree with the statement that Kaspersky "is caught in the middle of a geopolitical fight where each side is attempting to use the company as a pawn in their political game."

</comment><comment><AUTHOR> Doesn't look good for Kaspersky's future in the US</comment>


World's richest 85 people own 50% of global wealth - <AUTHOR> http://www.cnbc.com/id/101348398

<comment><USER2> Overly misleading title: they own as much as the poorest half of the world, not half the wealth in the world (which would be hardly identifiable anyway).</comment>


Your Uber Driver Can Secretly Report You on the App for Weed - <AUTHOR> https://herb.co/marijuana/news/uber-ban-weed/ <comment><USER2> Multiple drivers told me years ago that they had riders get into the car with a lit joint and demanded to be allowed to smoke it. They said they'd give the driver a bad rating otherwise. At the time (in my city), this often meant the driver would have such a low rating that they'd rarely get any fares.

I personally dislike it when cars smell like (any kind of) smoke because I smell like it, so I like this feature of the app.

<comment><USER3> It would of course make sense if getting in the car reeking of any kind of smoke was theoretically punishable by a ban. But getting in a car reeking of tobacco smoke is covered only by the usual ratings system.

I assume this is simply a problem of large-company bureaucratic policies: they don't want to have a separate policy for every local polity, and it's fairly reasonable policy from a liability perspective to ban people on, say, crack or heroine repeatedly getting into Uber. The unreasonable thing here is the fact that weed is illegal federally, but everyone already knows that.

<comment><USER4> _It would of course make sense if getting in the car reeking of any kind of smoke was theoretically punishable by a ban. But getting in a car reeking of tobacco smoke is covered only by the usual ratings system._

Where I drove, you were permitted to kick people out for smelling bad. Whether it was body odor, a drunk covered in his own puke, or someone reeking of cigarettes.

If you've ever been waiting for your Uber and he drives by while you take one last drag from your cigarette, now you know why.

<comment><USER3> Good to know. Presumably it doesn't trigger a ban from the system in the way that repeated infractions of the kind described in the article do, right?

<comment><USER4> Nope. But it does ding the driver.

</comment></comment></comment></comment></comment><comment><USER5> People are going to be smoking weed, doing drugs, having sex, generally making a mess when autonomous taxis roll out. They will be disgusting.

Once you remove the "let's respect the vehicle because it's owned by an actual human being sitting in front of me" you feel a lot less bad about being disrespectful when using a product owned by some corporate entity.

<comment><USER6> Sure, but I think a few interior cameras would deter that behavior. Not saying that's desirable; just likely where it's all headed.

<comment><USER7> There are cameras in taxis today. I don't believe for a second a self-driving taxi would ever be used without internal cameras.

As a privacy minded person I think that is ok. When I enter a taxi I don't expect total privacy (obviously don't expect the recordings to end up on youtube though).

I'm more concerned by the external-facing cameras. Possibly registering everything and everyone that didn't take part in my ride.

Without laws in place that data will most definitely be used for evil and the excuse might be targeted ads.

</comment></comment></comment><comment><USER8> Seems pretty reasonable to me. Look at it from the driver's perspective: if you're pulled over and the car reeks of weed or alcohol your day just got a lot worse. Doubly so if you're a minority and you get pulled over by someone on a power trip. What is the driver supposed to do, strip the customer looking for illegal substances? This gives an easy way to decline the ride or disincentive the passenger without having to make a long explanation.

<comment><USER9> Are you saying people who are intoxicated on alcohol are no longer welcome to use Uber? Or should not be welcome?

<comment><USER10> Alcohol isn't smoked and isn't likely to vaporize out of a drunk and settle on the walls of the car.

<comment><USER11> It can much more easily settle on the floor of the car though.

</comment></comment></comment></comment><comment><USER12> I feel like we're burying the lead:

"another claim of this nature could result in permanent account deactivation."

Uber wants to replace private and public transit, and then wants to be able to permanently "deactivate" your ability to transit.

<comment><USER13> This is what scares me about these stories. First kill any competition (go in debt if you need to) to become a giant. Then start enforcing (sometimes arbitrary or politicized) rules to the detriment of some customers who now have nowhere else to go.

Uber with surge pricing and this MomCorp weed ban (Uber drove the taxi's out of your neighborhood first, so now you have to pay for surges). "Sorry sir, I won't pick you up and drive you home, even though you are drunk, and unable to drive, because I only pick up people with a 3.7 rating or higher."

Netflix for entertainment (you rely on it so much for entertainment that you simply don't see a movie if it is not on there).

AirBnB refusing hosts to host you, if they suspected you came to Charlottesville to protest/march for your beliefs (that are counter to AirBnB's beliefs). Not saying that those protesters are not scummy or that AirBnB does not have the right, but look at the future where AirBnB owns 90%+ of accommodations. Can you imagine a hotel chain acting like this? "Sir, we saw you say on Twitter that you came here to protest. The management has decided you can't sleep here. The next competitor is 50 miles that-ah way. Good luck!".

</comment></comment><comment><USER14> In other news today, some Uber drivers pee into bottles while driving: [https://www.sfgate.com/cars/article/Uber-driver-pees-in- bott...](https://www.sfgate.com/cars/article/Uber-driver-pees-in-bottle- passenger-urine-urinate-13089274.php)

<comment><USER15> Every professional driver does that, at some point.

</comment></comment><comment><USER16> Meanwhile Lyft continues to be a pretty cool company to do business with.

</comment><comment><USER17> I don't really care. It seems like a reasonable response. But why single out marijuana? There is a rating system for customers, and this seems like it should fall under that.

<comment><USER4> Because the number of people lighting up a joint in Ubers vastly outnumber the people who are doing smack in an Uber.

My guess is that it's not being singled out. This is just the interest of the publication. Doing drugs of any kind in a car is illegal (at least where I drove), and equally problematic.

</comment></comment><comment><USER18> I'm not sure I'm following this. The smell can't be that powerful as to stay in the car. It's not like the customer lights up joints inside the uber

</comment><comment><USER19> Seems like there may be a market for a new tier of uber. Maybe one were the car standards and passenger expectations aren't as high.

<comment><USER15> ünter

</comment></comment><comment><USER4> I drove for Uber for a little while, and I kicked people out of the car for smoking weed, just like I kicked people out of the car for smoking cigarettes.

1\. My car is no-smoking. Ask before you light up. This is my personal property, not your living room.

2\. Smoking weed is legal where I was driving, but only in your private home. Not in a hotel. Not on the street. And certainly not in a motor vehicle. I'm not going to lose my license because you can't wait 18 minutes to get high. (Which is pretty much the definition of an addict.)

Where I drove, Uber would very quickly take $150 out of your credit card and give it to the driver if you vomit in the car, as a cleaning fee. I would not be surprised if the same applied to smoking drugs.

(I don't know if any of this was covered in TFA because I'm not going to visit what appears to be a stoner website on a work connection.)

<comment><USER11> Yeah that's fine and all, but not the point being made in the article. The person smelled of weed and got told on. No one tried to smoke in the car.

<comment><USER4> Uber drivers are allowed to kick people out for smelling bad. Reporting them for smelling bad seems like a less harsh result for the passenger.

</comment></comment></comment><comment><USER20> Now law enforcement would just mine that data.</comment>


Google Analytics JavaScript is Broken - <AUTHOR> http://www.seomoz.org/ugc/google-analytics-javascript-is-broken

<comment><USER2> All 3rd party javascript is broken. It's simply a fact. 15 years into the Web, still nobody has learned to check their external references before using them. Even google gets it wrong pretty much every single time.

If you really need to use somebody else's script on your site, the only thing you can do to protect yourself is to also add global exception handling to the page.

It would be nice if we could expect giant companies that publish Javascript APIs for a living to actually check that their Javascript works in most standard cases, but that's just not the way it works in the real world.

<comment><USER3> Got any pointers to "gotchas" articles or guides for cross-site JS widgets/includes? If not, want to write one? I've been looking for some blog articles on this recently for our team.

<comment><USER2> I actually almost wrote one today. Basically, if you're doing a JS API, you're likely going to end up with a pattern like:

      <script src="api.js"></script>
      <script>
        MyAPI.userID = 4000;
        MyAPI.Start();
      </script>

That's how most everybody does it, and it's guaranteed to throw ugly script errors any time a network error causes the .js file download to fail.

It's really easy to work around though. Somewhere in that .js file, simply declare a variable, or better still a method that sniffs for DOM dependencies like "does this div that I need to function actually exist". So you end up with something looking like this instead:

      <script src="api.js"></script>
      <script>
        if (window.MyAPI && MyAPI.CheckDependencies())
        {
          MyAPI.userID = 4000;
          MyAPI.Start();
        }
      </script>

That way, your thing will only run if it has fully loaded and has a reasonable chance of actually working.

Here's an old article I wrote with some other ideas:

[http://www.expatsoftware.com/articles/2008/05/one-rule-of- dh...](http://www.expatsoftware.com/articles/2008/05/one-rule-of-dhtml- programming.html)

<comment><USER4> Oh hi. It's you!

I read all of your blog a while back, maybe a year ago, when there was a long sabbatical. Most of the posts were about how to do the biz thing while all over the place, geographically, as I recall.

I see you're back!

But no more globe trotting posts? I'd love some updates to that part of your life, since I'm having to deal with such a situation myself.

<comment><USER2> Trotting around Venice, CA at the moment, after I got sorta ejected from the UK for coming and going too frequently. Lived in Basque country for a year, moved (unsuccessfully) to England. But yeah, I guess I haven't been writing too much about travel of late.

This might help a bit: <http://www.blogabond.com/jason>

</comment></comment></comment></comment></comment><comment><USER5> sounds more like your firewall filtering is broken.

<comment><USER6> When I read to the end I found that author proposes a mechanism that is far less convoluted than google's and devoid of this similar problems.

It's better to have less code in your page and more code in google's script - that way they can patch it without me getting involved. As it is now they added the try/catch block (really helps if users run ad-block or some such) but most web admins have not updated their stuff.

<comment><USER5> Though he accomplished it by altering the requirements. His solution does not work seamlessly with HTTPS sites -- it requires a server-side changes to the code. That's a pretty big issue for a script that's supposed to "just work."

<comment><USER6> But it doesn't "just work" - it breaks it the face of some firewalls and pops an exception if user has adblock. I actually had users complaining about the latter, this is not a theoretical problem (I fixed it with try/catch myself, but it should not have been a problem to start with and demonstrates brittleness of google analytics).

In other words it's a tradeoff between http/https transparency and odd errors on user's end. Pick your poison. :-)</comment></comment></comment></comment>


Apple Purchases iCloud.com for $4.5 Million - <AUTHOR> http://www.macrumors.com/2011/04/28/apple-purchases-icloud-com-for-4-5-million/

<comment><USER2> you forgot the question mark in your headline. the source has it.

<comment><AUTHOR> Good point, I thought I'd put it in. Seems my editing time has passed too, sorry!

</comment></comment><comment><USER3> Whether or not this is true, please note that the post has not one single piece of evidence. Just a rumor.

</comment><comment><USER4> I have a hard time believing they'd move away from the MobileMe brand (I mean, a music locker slots in perfectly), and an even harder time believing they'd call it iCloud - Cloud is very much still a buzzword.

Then again, I thought for sure it would be iMobile instead of iPhone, so what do I know?

<comment><USER5> Cloud is a buzzword, but there's very little brand equity in MobileMe. I bet less than 1% of iPhone owners have heard of it. And those who have heard of it probably haven't heard very good things.

<comment><USER6> I bet it's more than that, just because of Find My iPhone, which requires a free MobileMe account. However, I would bet far less than 1% actually has any real education about what it is.

</comment></comment></comment><comment><USER7> "Now you're thinking with clouds!"

</comment><comment><USER8> 4.5 million is a lot for icloud.com. It isn't even that great if a name.

<comment><USER9> But bought from a running business, venture funded. So, it had to be worth it to them to rebrand their entire business.

<comment><USER10> That doesn't make it a great name. It's worth $4.5 million to exactly two entities: CloudMe and Apple. If Apple didn't have $50+ billion in the bank it would probably be worth a lot less.</comment></comment></comment>


Was the Bitcoin Flock Just Sheared? - <AUTHOR> http://streetwiseprofessor.com/?p=7183

<comment><USER2> Can't read this on my phone. Hate it when sites think they're better than five years of smartphone development and their site is superior enough not to need zoom. Also wish there was a way to override the no-zoom tag; you'd think at least Mozilla's browser would feature that.

<comment><USER3> It's not much better on a desktop. The background interferes with the text color, making it very difficult to read without highlighting the text.

</comment></comment><comment><USER4> I gritted by teeth and tried to ignore it, but I just can't help myself: the past tense of shear is shorn.

</comment><comment><USER5> There is more to this to diagnose a pump and dump. According to these criterias, the internet bubble of 2000 would have been recognized as a pump and dump too.

<comment><USER6> There are quite a few people who hold exactly that opinion. For instance, check out _Alex Berenson's The Number: How the Drive for Quarterly Earnings Corrupted Wall Street and Corporate America_ , or read any "alternative" financial blog, and this line of reasoning might start to make at least some sense.

</comment></comment><comment><USER7> That is a seriously bad background

</comment><comment><USER8> The price per BTC was set back by 1 week.

<comment><USER9> If you assume the absurd growth just before the peak is "normal". It took over a year to get back to the 2011 peak, after going down much more than the first, sudden collapse.

</comment></comment><comment><USER10> Why is this on the front page of HN? This is just uninformed ramblings with a reference to an article (Krugman) contain similarly uniformed ramblings.

It just seems lazily uniformed. I doubt he even bothered to actually investigate and corroborate his pump and dump claims, into instead of spreading FUD. If he did, one would think he would have mentioned the alleged DDOS'ing of the exchanges etc. which are widely speculated (even among the 'true believers') to be part of a market manipulation scheme.

<comment><USER9> But it has a cute title!

</comment></comment><comment><USER11> Sure, let's discuss things no one seems to be talking about.

How is pump-and-dump even possible in an ecosystem such as Bitcoin?

Pumping is easy, sure, just load heaps of cash into all exchanges, buy every single wall that is put up, price explodes. But how is the pump-and-dumper supposed to get her money out when she decides to dump? All exchanges have a hard limit on how much cash they can divulge on each given day.

This is an impossible scenario with Bitcoin, as it is today.

<comment><USER12> How hard is it to setup multiple accounts on Mt. Gox using Tor?

300 accounts would give you access to $3 Million USD. And Mt. Gox has a simple API that you could coordinate all of your accounts together.

<comment><USER13> According to Mt. Gox, users of Tor are subject to having to submit additional anti-money laundering documentation.

</comment></comment></comment><comment><USER14> I am moving all my bitcoins into pogs. I understand pogs are back.</comment>