Issue 8788 - Every day around 9 AM Brussels time, huge drop in GAE performance - <AUTHOR> https://code.google.com/p/googleappengine/issues/detail?id=8788 <comment><USER3> > _I'm going to assume everyone experiencing this issue is using M/S. > Upgrading to HRD will solve your issue._

This is _the_ reason I abandoned AE and part of why adopting a platform that isn't standardized is incredibly dangerous. The problem is technical debt constantly accrues _even when you aren't making changes_.

Even though the API was unchanged, HRD differs subtly enough that breakage can occur on any non-trivial project. Edge cases (how indices behave within transactions comes to mind, but there are plenty more examples) will see new semantics compared to M/S, and so this "upgrade" involves not only thorough testing and auditing, but likely also code changes and potentially significant engineering hours.

<http://goo.gl/HVuaC>: _These techniques are not needed with the (now deprecated) Master/Slave Datastore, which always returns strongly consistent results for all queries._

This means a project written and signed off circa 2011 requires mandatory engineering costs just to continue running in a functioning and supported fashion. An AE app will never quite resemble that ancient perl5 behemoth running uninterrupted since 1997, as the underlying implementation and recommended APIs are constantly modified and replaced (Datastore, NDB, Python major version).

 _"A strong test suite will save your soul!"_ I hear you say, tests that a small project might have survived without if targeting any other platform, and testing on AppEngine is also yet another moving target (for example, testing nested subrequests was all but impossible using the SDK until relatively recently).

The promise was a carefree life for a project willing to code against their proprietary APIs; the reality is a constantly moving target, "not quite free" autoscaling and the threat that while you're asleep an unannounced change will take down your app (I could name a few, but as many will attest this has happened regularly since launch).

<comment><USER4> > The promise was a carefree life for a project willing to code against their > proprietary APIs; the reality is a constantly moving target, "not quite > free" autoscaling and the threat that while you're asleep an unannounced > change will take down your app (I could name a few, but as many will attest > this has happened regularly since launch).

Yeah, I got sucked in with the same promise and I had the exact same sour experience. Including panicky calls from the client when the app suddenly stopped working. The maintenance windows used to plop right in the middle of my client's busy time, once a month at least and often more.

The worst part is the apologists, like firstname.lastname@example.com in the original bug report:

> I got here from HackerNews, but after seeing the original poster spam the > forums in multiple places and have a bad attitude, I can't blame Google for > not fixing what looks to me like a non-issue. > > Fuck 'em.

That always reply to your request for help while you're attempting to fix a suddenly dead application and a totally screwed client.

</comment></comment><comment><USER5> Remember the old "thundering herd" problem with Apache children and things of that nature? You'd basically have a whole bunch of processes which had a listening fd from an earlier call to listen(). When a new connection would come in, the kernel would wake all of them, even though only one of them would actually have something to get. The others would go through the process for nothing. It caused a big performance hit back in the day.

Well, imagine now that you have a directory or lock service where you can store things and perform atomic updates. When you do a write to something in it, it fans out to all of its clients, and they all wake up (nearly) simultaneously and receive the update. They then have to do whatever processing you do with new data of that type.

If they all do this at the same time, then you have no processes left to service incoming requests. They're all identically busy with whatever mutexes held in order to apply those config changes safely, so no other work happens on those clients while they load in the new data.

It's not so much that it's taking a mutex and is getting stuck for a little bit, since that's going to happen no matter what. It's that _all_ of the children do it at the same time, so there's nobody to service your hit, and you're guaranteed to get stuck. If it was spread out, then only some percentage of incoming requests would get stuck behind this. The others would get lucky and would hit another instance which either had already run it or hadn't yet run it.

I'm not saying this is what's going on here, but it sure sounds familiar.

<comment><USER6> On what basis do you think these issues are related? The bug report provides very little insight in what's going on, only that there's a severe performance degradation at 9AM.

The thundering herd problem applied to waking up child processes is one possible explanation, but there are dozens of other explanations that are just as likely, based on the information we're provided with.

<comment> brown9-2 The commenter you are replying to is a former Google employee; the description of the lock service sounds like Chubby (<http://research.google.com/archive/chubby.html>), App Engine likely uses some sort of distributed directory service for keeping track of things like quotas.

</comment></comment></comment><comment><USER7> Well, the bug report doesn't really invite quick attention. Simply reporting your observations is not enough: you should position yourself as a competent customer, by explaining what you have done to ensure the problem isn't on your side. Mention the code hasn't changed, that you have no database cleanup cronjobs or similar running that could be interfering, etc.

My first instinct when I see a report like this is: he probably has some cronjob running he forgot about; perhaps one whose performance decreased with O(n^2).

By which I'm not saying that Google is right in not replying for days, but by which I am saying that as a customer, there are easy ways to get attention beyond shouting and threatening. Show it's an interesting problem and you're bound to get some techie's attention.

<comment><USER8> I consider him panicking more than shouting and threatening. I couldn't imagine having that kind of treatment as a vps customer else I'll be moving out asap.

<comment><USER9> If you were a VPS customer, you'd be less locked in and can just walk to another vendor so they actually are shit hot with support usually.

It was obvious when I first tried it that GAE has crappy support.

<comment><USER10> What are we actually talking about when discussing "good support" and "bad support"? Is it just someone nice to talk to whilst someone else fixes a problem for you? There was an interesting article along these lines by the former President of Enterprise at Google written recently: [http://gigaom.com/2013/01/26/the-delusions-that-companies- ha...](http://gigaom.com/2013/01/26/the-delusions-that-companies-have-about- the-cloud/)

In this case, the GAE feature that underlies this issue is the Master/Slave (MS) datastore. It's been deprecated for ages in favour of the High- Replication Datastore (HRD).

<comment><USER11> He falls in to a trap of knowing machine behavior, but not dealing with people behavior.

 _Insanity #2: I need somebody to talk to when a service interruption occurs_

You hear about an earthquake in California, you call your aunt to make sure she is ok.

You are getting bad weather in the area you live, your mom calls and checks on you.

The server you use disappears off the internet and your providers status page hasn't been updated for a week, you '...'?

When something goes wrong, it's not an event that effects everybody (even if it is), it's an event that effects you. As long as humans are still involved in the purchasing and managing of servers you'll always need someone to call and yell at/be soothed by.

<comment><USER10> That's true. I think that his broader point still stands, though. Once you get beyond variants of "are you working on it or do I need to convince you to?" the role of support is basically catering to irrational desires.

</comment></comment></comment></comment></comment></comment><comment><USER12> Google support is an absolute disgrace.

I had a Nexus 7 go AWOL at Christmas and I've never had such a shambolic customer service experience.

They have absolutely no respect or customer service ethos when it comes to people who are actually paying them real cash money.

Not in a million years would I sign off on hosting a production project on App Engine.

<comment><USER13> This customer is complaining about a service component which has been deprecated since almost 11 months ago. There is a tool which migrates application data from the old datastore to the new one. When you don't move off of deprecated infrastructure, I'd say you've set yourself up for problems.

<comment><USER14> A comment from Google explaining the issue wouldn't be much to ask. "Move off of deprecated infrastructure" is much better than radio silence.

</comment></comment></comment><comment><USER15> Why would anyone put anything production critical on Google these days, knowing that they provide 0 support across most of their business?

<comment><USER16> because we're paying customers? my bill is peanuts, but i know there are many big customers, e.g. Khan Academy. and also they have Premier Support which is $500/mo.

<comment><USER17> One thing I've learned with Google is that they don't give a rat's ass if you're a paying customer or not.

Khan Academy may have it easier, I'm sure Google won't let _them_ down

Really, go somewhere else, spend less money and have better support.

(At the expense of, if you're lucky, Google will give you almost zero headaches)

<comment><USER18> There's nothing about Khan Academy's application (or any other customer's) that would somehow immunize it from platform-wide serving issues.

<comment><USER17> Of course not, like when AWS fails Netflix stops working like every customer.

But you can bet that if Khan Academy has a problem it will be looked into with extra attention.

<comment><USER18> You're correct--because we have a Premier account, which anyone else can obtain too: <https://developers.google.com/appengine/docs/premier/>

<comment><USER17> Well, Caveat emptor

Also, Khan academy receives funding from Google ( <http://en.wikipedia.org/wiki/Khan_academy> ), so I don't think it boils down to only having a Premier account.

<comment><USER19> "You just have to pay for support and you get support? I don't believe it, there must be more to it than that!"

Whether you believe it or not, a Premier account is what you need if you want support. You could argue that $500/mo is too expensive, but it is what it is.

<comment><USER17> Oh, I believe you can pay and get support

What I don't believe is that the support provided by Google is good or sufficient. Based on experiences with _paying_ Google Apps, I'd say it's not.

<comment><USER20> My question is whether or not you personally have paid google for said support and then not received what was promised?

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> A GAE user sees a problem of his service being slow, writes a frantic bug report with caps and exclamation marks and threatens to leave GAE. As a GAE user myself, two questions come to mind:

1\. Is GAE outside of their .9995 SLA* uptime? If they aren't, then it probably isn't important enough spend time looking into it. Customers cannot expect better than the agreed upon uptime percent, and hosting companies are obligated to reimburse customers if they go below SLA. Both of these are covered in the SLA doc.

2\. Is it reproducible? So far, the bug report mentions 2 people out of GAE users. Is 2 people enough to say its a problem with GAE? One person is panicked, and the other provides few details for the bug report.

*<https://developers.google.com/appengine/sla>

<comment><USER22> 1\. 0.9995 SLA means about 6 minutes of downtime a month. Since it's a daily event, I'm guessing that yes, the SLA is violated. 2\. It's a problem that is occurring daily, with a test case that has pretty much no code at all. That in itself does not prove anything, but it really makes me wonder how it could be a problem on the user's side.

<comment> mark-r My math for downtime per month works out a little differently: (1-0.9995)x30x24x60 = 21.6 minutes. Still I mostly agree with you.

</comment></comment></comment><comment><USER23> Having never used GAE, it would nice if someone could expand M/S and HRD for me.

It looks like OP of the bug-report is using a depreciated feature/program which according to the Project Member is causing latency issues at a specific time daily. But that could not be the real issue since another commentator who is using the new HRD is also having the same problem. It is even frustrating for people who are reading this. All it implies is the lack of communication from Google when something goes awry. Come on Google, stop reinforcing my stereotypes about your customer support!

Selling to a customer is different than selling to a business, you may have a great product at a great price but if you offer terrible CS, in the B2B world everyone is going to avoid you. It is a place where support is valued more than the product itself.

Therefore, unless you start offering a decent CS, you can lower your price all you want, I will be sticking with AWS.

<comment><USER24> M/S is Master/Slave: [https://developers.google.com/appengine/docs/python/datastor...](https://developers.google.com/appengine/docs/python/datastore/usingmasterslave)

HRD is High Replication Datastore: [https://developers.google.com/appengine/docs/adminconsole/mi...](https://developers.google.com/appengine/docs/adminconsole/migration)

M/S is deprecated, and HRD is the new hotness (and it conveniently costs more).

<comment><USER25> I think it cost the same, when it was first out it cost more, but now it cost the same.

<comment><USER10> The rates are the same ($1 per million writes and $0.70 per million reads beyond the daily free threshold), but the daily free threshold is 0.05 million of each for master/slave, and 0.01 million of each for high-replication.

</comment></comment></comment></comment><comment><USER26> I manage the 3rd line support of some of the busiest websites in the world (we provide back-end e-commerce software).

I can't say I think much of google's response here. Nearly two weeks before the first comment, and then shut down after 2 days and a question directed at who knows who, and no explanation?

The analysis elsewhere on here suggests they're violating SLA, so this should get more attention. I'm guessing support is under-resourced @ google, and the culture of support is a bit shabby (no acknowledgement of inconvenience or indication or evidence of work undertaken in the background) - <USER2> surprising for a large-scale software business based on free services.

</comment><comment><USER27> I'm sorry, but this is the price you pay for running your business that is dependent TOTALLY on a 3rd party service. Forget Google, everyone out there is most likely the same, that's why it's important for you to run your 'apps' on something you have control over - Like Linode, AWS, Rackspace, Openshift, etc. and also have back-up nodes from other providers for redundancy, for emergency situations, incase of storms, etc.

I would recommend trying your apps on OpenStack (Openshift in particular), which doesn't have the vendor lock-in, which you face right now.

<comment><USER28> Interesting comment. When AWS went down in the US, devops rightly said "we told you to run your apps on something you have control over."

<comment><USER29> What's that? A server I own on a rack I own in a datacenter I own on a power factory I own and a telco I own?

"Having control over" something is a scale, it's not binary.

<comment><USER28> Indeed, I was trying to say it is all relative.

<comment><USER29> Sure, but it's a tradeoff. No need for devops vs no control over outages.

</comment></comment></comment></comment></comment><comment><USER30> To their credit, people are apparently using something that's been deprecated and should be changed regardless. At least, that was their conclusion when it was changed to wontfix. The replies are very rare and curt though, I can't really say it's quality service when you're paying for a product.

Customer support from Google has always been like this as far as I've experienced and heard. There is no way to actually reach and converse with anyone, regardless whether you are paying them for the service or what kind of request it is.

Once a Google employee randomly replied to a complaint of mine about Google+ (I didn't even +mention them). After a few comments and him confirming that it was added to the bugs list, I asked if it was okay to +mention him in the future with similar issues. It was okay. I did. He never showed his face again. (His profile still says "Works at Google+".)

Another Google employee I know online also never replies to anything concerning Google. I know he works on the Google+ project, but I can only hope he passes on any bugs I +mentioned him in.

For Youtube, you can post in their forums but merely hope for a reply. Copyright complaint disputes are no priority, either.

I haven't used many paid products, but I have read about their customer support being one of the very worst and also have never been able to find a single e-mail address or phone number to get support at for any service.

Edit: By the way, I would have moved away from the Google Apps Engine a long time ago if my app went down every morning during rush hour for 10 days straight.

</comment><comment><USER31> It is interesting that basically no one (including the news poster) noticed that there is an comment (#12) which states that this problem happens on HRD too. This statement may be false and/or a completely different issue, but at least it should be considered here for HN comments which state "M/S is deprecated, Google is right, just use HRD."

</comment><comment> brown9-2 A bug tracker seems like a horrible way to report production (or non- production) support issues. This is the same bug tracker OSS projects on Google Code use.

Is it really helpful for the public to comment on my support request? Seems like the signal to noise ratio would be quite low, and then you get inane comments like:

 _I got here from HackerNews, but after seeing the original poster spam the forums in multiple places and have a bad attitude, I can't blame Google for not fixing what looks to me like a non-issue.

Fuck 'em._

You have to believe that the choice of tools has some bearing on the quality of the response from Google. Seems like there is very little incentive for any "Project members" to trawl through open bug reports when no one is ever responsible.

</comment><comment> Al-Khwarizmi Not surprising... the second most-voted bug in Google Code, reported exactly a year ago ( <http://code.google.com/p/support/issues/detail?id=24324> ) deplores the removal of a feature that was already there (the Updates page) and was the single most useful feature in Google Code for many of us. After one year and more than 800 people registering their interest on the issue, they haven't even explained why they removed it or whether there are any plans of brinding it back.

</comment><comment><USER21> Comment from the WontFix mark:

"M/S is deprecated and there is a clear and straightforward path to migrating to HRD."

M/S was deprecated April 4, 2012, so it has been some time since the notice has been out there. High replication data store has been available for over 2 years now. Whether or not less than a year is too short a deprecation period is another issue.

<comment><USER32> Read the following message - other customers using Python on HRD are reporting the same issue as the parent, who is Java on M/S.

</comment></comment><comment><USER33> Ok, so here's the deal. If your app runs exclusively on GAE you've essentially tied yourself to one cloud vendor. Now disregarding the respective benefits and drawbacks of google as a hosting company for your app (I would never do that), being dependent on one cloud provider is a very bad idea. No matter if you run on EC2, Azure or GAE, if you can't seamlessly switch to another provider, you're screwed. These all go down regularly and have issues. They're big companies, you're a small company, you have no such thing as "recourse". The court of public opinion will not save your company.

<comment><USER34> Agree with this to an extent however the company I work for deploys on AWS and is far too cautious about vendor lock-in, to the point where we use AWS basically as a VPS, not a cloud service, and get none of the advantages (and all of the disadvantages, e.g. worse performance, higher price).

</comment></comment><comment><USER35> Many on the thread say the reporters are over-reacting. They are not. What would amazon do? They would not consider this an issue, would respond in less than 24 hours, and would take complete responsibility. GAE is a pay service. I think this level of service is pathetic.

As noted the only attempt at diagnosis is completely wrong (even the reporter is not on MS) and very late.

</comment><comment><USER36> The headline blows this out of proportion.

Few people (who act obnoxious as hell) report a problem that can be solved by moving away from a deprecated system, yet they fail to even read the note because they're busy smashing exclamation marks into the issue tracker.

<comment><USER13> I couldn't agree more. In fact, it was deprecated just about eleven months ago: [http://googleappengine.blogspot.com/2012/04/masterslave- data...](http://googleappengine.blogspot.com/2012/04/masterslave-datastore- thanks-for-all.html)

When your datastore gets deprecated, you act sooner rather than later.

<comment><USER10> The two datastores even have the same API. As long as your app doesn't depend on the exact performance characteristics of the old one, the migration is very straightforward. I did it for one of my apps in a morning and was done well before lunch.

</comment></comment></comment><comment><USER37> Why would anyone expect customer support from Google? They have made it clear time and time again that they don't provide it. [http://shkspr.mobi/blog/2013/02/googles-customer-contempt- co...](http://shkspr.mobi/blog/2013/02/googles-customer-contempt-conundrum/)

<comment><USER13> Support packages are available. [http://googleenterprise.blogspot.com/2013/02/google-cloud- pl...](http://googleenterprise.blogspot.com/2013/02/google-cloud-platform- introduces-new.html)

<comment><USER38> The problem with saying things like "Support packages are available", is that time and again we see paying Google customers with support packages being treated awfully.

For example, this guy was a paying Google customer but couldn't get help <http://www.sultansolutions.com/google-voice-lost-number/>

These are paying customers who are paying a non-trivial amount of money for support (though not the "Premium" support in this case, which is an extra $500 per month for GAE).

<comment><USER13> We're a paying customer of GAE. I think it's quite clear that paying for the basic service doesn't include support beyond the public issue tracker and the forums. Support packages start at $150 per month, and at that point you get a 4 hour response time. I think that's entirely reasonable. We have yet to sign up for a support level, but then again we're not really seeing any troubles with the service.

<comment><USER38> I guess the question is, if you're paying and you find what you believe is a system wide issue, do you expect a resolution?

In this case, Google's response is seemingly "It might (or might not) be a system-wide issue, but we don't care - we won't fix it".

There's no indication in this case that someone paying $500 a month for the premium support would get a better answer.

</comment></comment></comment></comment></comment><comment><USER39> Customer support of Google really sucks! Currently the GAE cloud has a reliability problem (also for new customers). Instances are restarted like crazy. This leads to downtimes. But that's not enough. Customers have even to pay more(!) instance hours because of this. There is the running gag on the mailing-list: "Whenever GAE is unreliable for weeks Google needed to make revenue targets ;-)"

References

Current Issue: [http://code.google.com/p/googleappengine/issues/detail?id=88...](http://code.google.com/p/googleappengine/issues/detail?id=8844)

Same issue from last year that took weeks to be resolved (check last comments!): [http://code.google.com/p/googleappengine/issues/detail?id=80...](http://code.google.com/p/googleappengine/issues/detail?id=8004)

Some Pros and Cons of Google App Engine in this blog-post: <http://www.mosbase.com/>

</comment><comment><USER40> BTW, this issue is not simply a due to MS, it also happens on HRD. So any google support apologists here, please read the BUG thread submitted by this poor customer before dismissing it simply as a 'migration issue'.

I have had some issues with Google Docs (paid for premier commercial account). Some documents we had stored simply vanished from our account. After getting the run around for 3-4 days, finally a google engineer tolds us they can't help us recover the documents THEY 'lost' unless we have the URL to the document ... Thankfully someone on our team had kept the URL when I first shared that document with them (1+ year after the document had been created).

Nightmare ...

</comment><comment><USER41> Quick tip for anyone making a system with high load and daily or hourly quotas: When an account is created, assign a random start time (e.g. 05:43 for daily quotas or minute 12 for hourly) to measure that account's quotas against. Then you can avoid this issue of the system getting a huge spike in load when everyone's quota refreshes at the same time.

</comment><comment><USER42> It happens that 9 AM Brussels time is midnight pacific time. I'm sure Google is running some maintentance cron at midnight thinking "This is a low demand time," and it is, across the US, but not in Brussels. These are old instances, and Google probably doesn't want to re-time or rewrite the cron job to be more efficient.

<comment> guard-of-terra Do people go to bed that early in the US? Looked at my project's charts, the low demand time is definitely 4AM and midnight is 40% of peak.

</comment></comment><comment><USER43> Ironically, the guy who closed this issue owns this project:

<https://code.google.com/p/sentimentally/>

"sentimentally is a tool that determines sentiment of your emails. Once determined, it helps you gauge your relationships with co-workers, customers, friends, or other individuals based on the tone of your conversations with these people."

</comment><comment><USER44> Never pay Google. It has terrible support for all products

<comment><USER45> Not my experience at all. While it's true I never had much trouble with App Engine, when I had problems, they were solved rather quickly.

</comment></comment><comment><USER17> " Upgrading to HRD will solve your issue. M/S is deprecated and there is a clear and straightforward path to migrating to HRD."

Can anyone explain why this is not possible for them?

Wonderful Google support apart, there are a lot of alternatives out there.

<comment><USER46> Multiple comments mentioned that this occurs in HRD as well.

<comment><USER21> Only one other comment mentioned the problem with HRD as the datastore.

<comment><USER47> And the guy having the problem with HRD doesn't count because ... ?

<comment><USER13> Because they did not post any kind of evidence (request logs, Pingdom report, etc.), not to mention the App ID in question (so that Google would know where to look). All too often, bug reports end up being some kind of misunderstanding.

</comment></comment></comment></comment></comment><comment><USER48> I am also a GAE-user, I have had no problems like the OP. But I start to miss a fundamental feature, sockets. I have worked around it by using other services and polling.

Maybe wrong forum, but is there any infrastructure templates for setting up a scalable web/db/loadbalancer/memcached for a simple tradional webservice, in my case a game?

I want to be able to sleep at night, and easily scale up by adding some more machines in case of higher load.

I could use denormalized myslq/postgre or mongodb for speed. Preferred language is Python (or maybe c# or java).

Any ideas?

<comment><USER49> Channels.

<comment><USER48> Channels are for communicating with javascript clients. I want to communicate with other servers and applications.

<comment><USER50> stay tuned

</comment></comment></comment></comment><comment><USER50> Hi folks. We are fully aware of this issue. We've added it to external issue tracker ([https://code.google.com/p/googleappengine/issues/detail?id=8...](https://code.google.com/p/googleappengine/issues/detail?id=8901)), please follow up there.

Response from us was initially muted because it looked like it only affected M/S apps, but it turns out (a) it can impact HRD as well, and (b) we're pretty unhappy about the level of impact for many M/S apps so we're looking at ways to resolve. It's a high priority and we're looking at a number of ways to address it. It's also a pretty interesting issue, because indirectly it's caused by (a) the large scale that App Engine is running, and (b) the large extent with which GAE is running free applications.

Regardless, apologies to those who felt support was unresponsive. We are working very hard to improve support. For the sophisticated audience that comes to these pages, please link to me on Google+ to get my attention if we are failing you (<https://plus.sandbox.google.com/110401818717224273095>).

</comment><comment><USER51> Gae is cool but its not worth the money. I shouldn't be out of beta.

</comment><comment><USER52> I've heard a lot of people saying this is why Google can't get a lot of businesses to sign on. There's no one for the CEO to call and complain to directly when their stuff is down.

</comment><comment><USER53> Well, there's a good reason not to use this service.

</comment><comment><USER54> Jesus, so glad I switched to node when the hosting rates increased back in the sept 2011 with the "GAE out of preview" move

</comment><comment><USER55> Ahh the perils of being a customer of Google.</comment>


3D Printed Artistic Representation of Noah's Ark - <AUTHOR> http://www.3ders.org//articles/20141229-taiwan-artist-reimagines-noah-ark-as-a-distorted-8-meter-shipwreck-3d-printed-pieces.html

<comment><USER2> I was expecting less "art discussion" and more technical discussion.

Fortunately, the details occur later in the article, but there's a lot of "art talk" at the beginning of the article.

They used 3D Max for software, and simply cut the Ark into regular cubes, which were then glued together. Personally speaking, I'd be more interested if they had a more complicated technical methodology.

If they simulated assembly modules, or had a more intricate assembly system (Joints are... surprisingly complicated and intricate), I'd have found the article interesting. Otherwise, this was a very brute-force methodology to build a massive art piece.

For example, imagine if the Ark were built out of a metal skeleton simply welded together. The Skeleton could have been scanned in with a 3d scanner, and then a custom model built around the support.

The model could have then been a hollow-shell, simulated with FEA for structural integrity and then designed to be press-fit together (without glue).

Such a methodology could have been accomplished without using literally tons of plastic that this guy has done.

The way the article goes, it sounds like the frame was designed after-the- fact. Which is kinda surprising that they made this work at all without collapsing in on itself.

<comment><USER3> Your dismissal of the "art talk" is surprising, given that the message is highly pertinent to the techno-expansionist worldview that seems popular among HNers:

> _While mankind turned to crudely built vessels in Biblical times, modern man > would undoubtedly turn to modern machinery if faced with a similar > catastrophe, which has been reflected in the project 's cruise ship-like > appearance. '[We are] about to enter the era of Mechanocene [the mechanized > era], where machinery automation will be applied to production lines and > where human beings are being replaced with machinery.' But will that be > enough? If a modern day Ark would sink, what would happen?_

> _Those are exactly the type of thoughts that Peng seeks to provoke with his > project. 'This work is a metaphor to expose the collision between Mother > Nature and the accelerated development of an industrialized civilization. If > Noah's Ark, a symbol of mankind salvation, becomes just as a shipwreck, > human and nonhuman were placed in an equal position. The human subject is > losing his predominance as the supreme center of the world.' That is exactly > why this cruise ship has been twisted and distorted; it is essentially > transformed into a beached whale as well._</comment></comment>


Ask HN: Toys to help children aged 3-7 learn the basics of coding? - <AUTHOR> Do you have any recommended toys (preferably physical, touchable objects, not software) for teaching young children the fundamentals of coding? I understand that at this age they may be too young to grasp all of it, but if it lays the foundations for understanding how you can use basic building blocks and rules to solve problems, that would be a great start.<p>The only toy I'm aware of is the Cubetto by Primo Toys, and I'm not sure how effective it is. Any suggestions would be appreciated! <comment><USER2> Just my 2c and not coding related; I cannot recommend LEGO enough. Creative and logical, it was a huge influence on my desire to build things that are greater than the sum of their parts.

<comment><USER3> Yup, and way better than staring at a screen.

</comment></comment><comment><USER4> Robot Turtles seems to be thought well of, I did the kickstarter for it (so far my only one) and I think my son is just about old enough to play it.

For practicals, Shoots and Ladders may be the most like day to day coding :)

<comment><USER5> I have also bought Robot Turtles just a few months before my son turned 4 (we are now a few months past 4). He liked it but it seems he was still too young for it. That is why I have sort of "forgotten" about it (on purpose) until he is a bit older.

</comment></comment><comment><USER6> Fisher Price made a coding caterpillar - [http://www.theverge.com/2016/1/5/10716994/fisher-price- thing...](http://www.theverge.com/2016/1/5/10716994/fisher-price-thing-and- learn-code-a-pillar-toy-ces-2016)

</comment><comment><USER7> I know I am not answering your question but having played with my daughters (almost 4 and 7) I found [http://www.scratchjr.org/](http://www.scratchjr.org/) one of the best options. Yes, it is a mobile application but it is much easier than Scratch where the kids have a lot of different blocks, need to use the keyboard and mouse instead of a touch interface and must read. ScratchJr also motivates the story telling side of programming making stories and characters.

Personally, I am rather skeptical of coding toys for very young kids, or to say it in a different way: you can spend one year pushing a 3 year old kid to learn something that at 6 can learn in a few hours and with much more insight.

</comment><comment><USER8> I work with the lower age groups of a primary school (age 6-10) and we have explored some basic concepts using the following:

Toys: * BlueBot / BeeBot. Simple robot programming with arrows. If you draw a grid on something transparent you can overlay it on maps/images and put "obstacles" in some of the squares that the robot has to be programmed to go around. * Makey Makey. Not programming so much but kids love the creative aspect and learn about hardware

not quite toys, but some useful iPad apps: * Free/creative: Scratch JR, Hopscotch * Guided/"missions": Kodable, A.L.E.X., Lego Fix The Factory

Also, as others have said, LEGO.

</comment><comment><USER9> Legos FTW. The spacial skills they help cultivate would be equally applicable to any kind of engineering later in life.

</comment><comment><USER10> These are computer games, but I've had a lot of fun teaching my mom about recursion through Cargobot[1] and Robozzle[2].

I think it would be relatively simple for you to make this into a physical game-- hand tracing the execution with them.

[1]: [http://twolivesleft.com/CargoBot/](http://twolivesleft.com/CargoBot/)

[2]: [http://www.robozzle.com](http://www.robozzle.com) (Use the javascript version, or the android app)

</comment><comment><USER11> When i was kid, i used to watch a lot of cartoon network channels and used to by CAR toys and open them to see inside how its made.

maybe let them play with stuff like how its make to discover themself then later they become good logical analyst.

</comment><comment><USER12> Check out makewonder [https://www.makewonder.com/dash](https://www.makewonder.com/dash)

</comment><comment><USER13> Kano: [http://us.kano.me/](http://us.kano.me/)

</comment><comment><USER14> I know the Bee-Bot and Pro-Bot, are more like the old Logo programming.

</comment><comment><USER15> Let me supplement physical toys (which are most important for sure) with some additional ideas for building future programmers.

Children ages 3-7 are really just beginning to learn how to think abstractly, which is a pretty core competency for programming. So when you say you want to teach fundamentals of coding, you really aren't even at "coding" yet. You're really at the fundamentals of math, language, reasoning, and problem solving and curious mindsets.

So what are the fundamentals of coding that 3-7 year olds can actually do?

1) Math - I would scour the web for all the different ways you can do basic math with any type of physical object (pencils, fruit, whatever). You may think it's too simple, but if you take a cognitive approach to this, practice must be done at some point with mapping a number of any object to it's linguistic representation in one's mind. Practice counting up/down, sorting stuff into groups and sets, quantity, patterns (2x3, 3x2, are they the same? what's different?) etc, and doing simple arithmetic with actual objects. This is new to them and is fundamental to everything else.

2) Language - have adult conversations with your kids, and no not about adult topics, but with adult words and with adult grammar; no "baby-talk" in other words. And it's not about purposely choosing large and obscure words to obfuscate, but picking accurate words that probably leave little holes in understanding for the child to try and reason about. Hopefully they will ask if they don't know, or may even surmise correctly about word/phrase meaning. This might help with...

3) Reason/problem solving - Encourage experimentation in everything. If they see you constantly trying new things and failing gracefully and focusing more effort on finding solutions then hiding failures, you've taught them one of the most important lessons of all. Try, try, try many times, then go ask someone. You should be unquestionably _the_ expert in some things, and a complete novice that asks questions and learns quickly in other things, and they should watch you in both situations. Seeing role models do both is an immensely powerful thing to observe to young children. It shows that adults can be both powerful professionals and learners. Where do you think they learn to laugh at, criticize, and fear failure? It's cultural. Make failure just another step to mastery.

4) Curious mindsets - in every category above, there are always opportunities to ask "why do you think this is?" or "how do you think this works?". You might get gibberish most of the time but it doesn't matter, they are going through the thought process and will get better with age. They need to be very comfortable with those questions at early ages and keep being willing to answer them. When they stop caring, THAT'S the problem.

So to try and answer your question more concretely, you can buy expensive toys made for programming but I don't see what it will accomplish. LEGOs are the gold standard, but really, every-day objects can do much of the above. Build a fort! Why are we building a fort? How do we do it? Tell me the steps for building a fort so we can do it again someday. How do we improve this fort? Tell me the steps for improving...</comment>


Apple Develops Peer-to-Peer App Sharing System - <AUTHOR> http://www.readwriteweb.com/mobile/2010/12/apple-develops-peer-to-peer-app-sharing-system.php

<comment><USER2> I am all over this.</comment>


How to represent a user's internet trek... - <AUTHOR>

hi...
interested in looking into methods to represent a user's internet trek, ie, the sites that a user would peruse, so i would have a "tree/graph node" representation.
basically, i'd like to be able to create a system that would allow me to track my sites that i visit, and to be able to create a tree/graph node representation of where i've gone, and the referral relationships.
lot's of different articles via google, but nothing solid in terms of db schema, or code applications.
any thoughts/sugestions would be helpful.
thanks
-bruce email@example.com <comment><USER2> Is use this bookmark link (javascript:var%20b=document.body;var%20GR________bookmarklet_domain='<http://www.google.com';if>(b&&!document.xmlVersion){void(z=document.createElement('script'));void(z.src='<http://www.google.com/reader/ui/link- bookmarklet.js'>);void(b.appendChild(z));}else{}) to agregate the links I've read, and publish a RSS feed.

If there was a piece of software that could take a public reading feed, against a complete local/private path, you might be able to come up with a decent hieristic that would infer the tree/graph you desire; or at least a fair approximation.

<comment><AUTHOR> based on continuing research..

i think i'm looking towards research in the area of web data mining/xgmml applications... some of the articles i've looked at today, appear to have application to what i'm considering.

if anyone has additional thoughts regarding this area, let me know!!

thanks

-bruce firstname.lastname@example.com</comment></comment>


Edward Snowden: Classified US data shows Hong Kong hacking targets - <AUTHOR> http://www.scmp.com/news/hong-kong/article/1260306/edward-snowden-classified-us-data-shows-hong-kong-hacking-targets <comment><USER2> What is with HN?!

Just because they are Chinese means they don't deserve basic human liberties? Just because you live in a different country doesn't mean you lose these rights, they may not be recognized but you are still entitled to them.

Furthermore, there's an extremely large cognitive dissonance here. China owns trillions of US national debt... this is debt that the leaders of our country have given willingly, and gladly pocketed the resulting cash. How is that any less treason than alerting our lenders that their systems have been broken into?

"We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights..."

Chinese, American, British, Pakistani, Russian, French, it _doesn 't matter_. The targeting of innocents by drones is wrong no matter where you are from, why does the right to privacy and property (or happiness, depending on your school of philosophy) not follow that same logic?

Perhaps we should be more concerned that we are continuing to pick petty fights with our fellow man over ideologies, rather than the fact that one man is perceived to have violated our specific ideology. But I guess my idealist is showing.

My 2 cents.

<comment><USER3> What is with HN?!

Just because you disagree with someone doesn't mean you can exaggerate your point. China doesn't "own trillions of US national debt" like you suggest. Plus that debt is "given willingly, and gladly" to them because they purchase it. We are after all a capitalist system. If China is willing to pay more for US debt than Japan (who holds somewhere between 80% and 90% of the amount that China holds last time I check), than they should have the right to purchase it. Just because they are Chinese doesn't mean they don't deserve the same basic free market opportunities.

Perhaps we should be more concerned with the issue at hand instead of tangentially mentioning an issue we don't know a whole lot about in the hopes that no one will point out our mistakes.

My 2 cents.

<comment><USER2> I don't see where we disagree[1], but perhaps your argument is lost in the attempt to parody my post?

[1] Minus the "trillions" bit... see posts below. Perhaps "over a trillion" would have been a better phrase.

<comment><USER3> _" China owns trillions of US national debt... this is debt that the leaders of our country have given willingly, and gladly pocketed the resulting cash. How is that any less treason than alerting our lenders that their systems have been broken into?"_

A passage like this perpetuates the misconception that China owns a dangerous portion of US debt. It is stretching the facts of the case so it better supports your argument. They own 1.25 trillion, which is not "trillions". It is also less than either the Federal Reserve or Social Security trust hold and right around the last numbers I saw for private US holders. The phrasing you use to explain the transaction makes it seem like the US government shouldn't sell this type of debt to China. In your post you are claiming that the Chinese should have the same level of human rights as us but that we shouldn't provide them the same level of economic rights? Finally it is also disingenuous to say that our leaders "gladly pocketed the resulting cash". That implies some sort of embezzlement, which is just furthered by your comparison to treason.

Selling debt is a critical part of the economic equation for almost all world governments (yes, even China). It annoys me when people state that selling debt to China is somehow destroying this country or causing long term damage.

</comment></comment></comment></comment><comment><USER4> _The detailed records - which cannot be independently verified - show specific dates and the IP addresses of computers in Hong Kong and on the mainland hacked by the National Security Agency over a four-year period. [...] "I don't know what specific information they were looking for on these machines, only that using technical exploits to gain unauthorised access to civilian machines is a violation of law. It's ethically dubious," Snowden said in the interview on Wednesday._

So he has no idea what foreign entity the NSA is targeting or why, yet he decides to warn them about it? This is supposed to be ethical, responsible whistleblowing?

<comment><USER5> I believe that this is reasonably targeted. He does know who owns those computers, and did not include any government computers on the list.

I know that we Americans like to see al qaeda around every corner. But, for instance, when you target a university's system in a non-Muslim country, the odds are much better that you're conducting industrial espionage than that you're targeting al qaeda.

At the least, from a public policy point of view, it shows the hypocrisy of recent articles in the USA complaining about the extent of Chinese hacking into US civilian targets.

<comment><USER6> _He does know who owns those computers, and did not include any government computers on the list._

It's completely possible that these computers (or netblocks) were used as jumping off points for attacks against the U.S. or U.S. companies. We'll probably never know, but there certainly isn't enough information in the article to determine whether or not this was legitimate counterintelligence activity or whether these computers were reasonably targeted.

</comment></comment></comment><comment><AUTHOR> I don't get the sense Snowden is pro-China, or that his goal is to defect to China. Definitely however, these new classified disclosures raise temperatures across many fronts, including his own boiling pot.

The status of his four computers (whether in his possession, or hidden elsewhere in HK) is significant because China could overrule HK and get its hands on all of Snowden's data if he's arrested.

I personally trust Snowden's actions, and believe his personal motives are ethical, albeit controversial.

国民我相信,美国和中国政府不相信。

See related NY Times article. [https://news.ycombinator.com/item?id=5880362](https://news.ycombinator.com/item?id=5880362)

<comment><USER7> From the NYT article: "The data, if independently verified, could help Chinese officials figure out which computers have been hacked, patch security holes, itemize compromised data, analyze the quality of computer security defenses and develop techniques for hardening other Chinese computers against future surveillance by the N.S.A."

So he is offering aid to the Chinese government in the form of top secret US data. I understand the need to have whistle blowers and respect that some things need light shined on them. However, I draw the line at specifically assisting foreign governments who are often hostile to democracy and the West. If the reverse happened and a Chinese security contractor fled to the U.S. with state secrets I would be less worried but I do think that person would be a traitor to China.

<comment><USER8> I've been listening to the book Spycraft, and if there's one theme that keeps recurring, it's that things never end well for the agent informing against his country.

The handlers often turn the guy over once he's outlived his usefulness to his homeland, or eventually he gets gotten.

Snowden's play here seems to be that he assumes the U.S. is going to lock him up for life or worse, so he's throwing his lot in with China and showing up with, "Hey, here you go! Loads of free stuff!"

But he apparently doesn't know much about the Chinese, or the history of turncoats and how these things usually play out.

Nobody trusts a traitor, not even if he showed up with plentiful offerings. It's like having a gal cheat on her husband with you, then tell you she wants to divorce him and marry you. No chance of her doing the same thing to you now, right?

</comment></comment></comment><comment><USER9> He's going to lose American support with this kind of thing.

This is more or less what the NSA is supposed to be doing, and many Americans are going to openly support these activities. It will make many much more sympathetic to accusations of espionage against Snowden.

Undermines the privacy concerns raised earlier, unfortunately.

<comment><USER10> It's illegal to break into computers in all civilized countries. The director of the NSA should be extradited to China to face justice for his crimes (after being punished in the US for his crimes here, of course).

<comment><USER11> It is not illegal in the US to break Chinese law. In fact, it is the job of the NSA to gather intel, not to obey laws.

<comment><USER12> Oh you can fuck right off with that. It wasn't illegal to have a linking site in the UK yet that poor bastard was extradited for breaking US law, not having been there. If the US wants to do that kind of shit, then hell yes if they break laws of other countries they should get extradited there to face charges. I'm so sick of American exceptionalism.

<comment><USER11> I suppose we should extradite people to Iran for breaking Shariah too. This isn't American exceptionalism. Do you think China will extradite their military leaders for hacking US computers? Fat chance. If agents are caught in country breaking the law, they are fair game. That's the risk. Suggesting the US should extradite our leaders to China is armchair idiocy.

<comment><USER12> >Suggesting the US should extradite our leaders to China is armchair idiocy.

So is suggesting that a guy who did something totally legal in his country (UK) should be extradited to the US to face charges there.

<comment><USER11> In most cases, that's probably true. I'm not apologizing for all US policies.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> He is a smart kid. He has to position himself as a bargain chip, that`s one reason he picked HK.

You can number the countries in this world who would be powerful enough to withstand US pressure.

Never forget he is only 29 without an escape plan you dont make this kind of journey. If he has to live his remaining life in China/HK he is probably fine with that.

And please step out of your box it does not matter where someone is from. Now everyone is screaming "traitor traitor" and you all forget what he uncovered and brought to main media attention.

But soon he will be forgotten marked as a traitor and everything will move back to square one. And secret courts will grant agencies powers nobody knows about :).

<comment><USER14> Definitely. When you have a life sentence coming your way (like in Manning's case) anything goes. If this is a way for him to protect himself, then it is a decision we should respect, because this is the price to be paid for his other revelations.

</comment></comment><comment><USER15> The pattern of events is pretty strange.

Snowden, a US citizen, chose to leak the documents to Guardian and not New York Times, meant to be published while he is in Hong Kong, which is under Chinese rule in the matters of foreign policy.

I'm genuinely trying to understand as to why he didn't trust a single person from his country? With the claims of the access he made, it should have been far easier to find one, without going to a non-US newspaper* and boarding a plane to a nation that is not on exactly the friendliest terms to US.

(* Nothing against Guardian, it's just that the visible pieces of the puzzle do not seem to fit together.)

It is very hard to believe that he had sweeping access levels. For one, NSA has its own chip manufacturing facilities, teams of mathematicians and security experts. Even if we don't know how NSA works, we can deduce from its contributions to open source: selinux, Apache Accumulo. Surely, they are expected to have tighter systems.

Besides, he was an employee of a contractor. Even less reason to provide elevated access to him. So, if he has material in his hands, than his security clearance would allow, then it is way more surprising.

Snowden's statements ought to be taken with a grain of salt.

Edited: grammar.

Update: thanks to zecho and brown9-2 for their comments; Snowden leaked it to an American.

<comment><USER16> > I'm genuinely trying to understand as to why he didn't trust a single person > from his country?

This is untrue. Snowden contacted The Washington Post first, according to their account, but when they did not move as quickly as he would like, he approached The Guardian.

However, the accounts from the Washington Post and The Guardian conflict with one another, so it's possible he worked with both in tandem while trying not to tip his hand to either.

</comment></comment><comment><USER11> As much as I'm glad NSA snooping came to light, I'm very uncomfortable with a rogue 20-something singlehandedly deciding issues of national defense. It is not enough simply that something be illegal. IMO, something must be highly unethical and capable of causing severe lasting harm in order to justify turning over classified information. Snowden is not a justice, an elected representative, or someone in general who we want deciding what we should and shouldn't know.

<comment><USER17> > Snowden is not a justice, an elected representative, or someone in general > who we want deciding what we should and shouldn't know

You're going to trust the people in power to make the judgement calls over their own power....? "Hey guys we can blackmail anyone. Let's make sure that the proper checks and balances are placed on that, I hate unlimited knowledge and power."

<comment><USER11> I certainly don't think every foot soldier should be deciding what needs to be secret or not.

The definition of power is making these decisions. Those in power legitimately deserve some latitude to excercise that power. "Those in power" is now Snowden because he took that power illegitimately. There is a time and a place for that, and some of these leaks may be that. Some of this is definitely borderline.

<comment><USER18> He's done a heroic thing, and I think he deserves some latitude in defending himself from the unjust prosecution of those he's exposed. They brought it on themselves.

</comment></comment></comment></comment><comment><USER19> I really don't understand this outrage about the NSA (allegedly I know, but let's be serious) hacking Chinese targets. That's their job. The Chinese, through hacking, have obtained highly classified information about our most advanced weapons systems. They've stolen critical information about the F-35, AEGIS radar, not to mention hacking US companies to steal their intellectual property.

The internet open, free, etc but the world is still defined by borders and national interest. Nations have always needed to collect intel to inform their decision making and they've employed spies to do it. It just so happens the medium that the NSA operates in is the digital one, with their spies sitting in a big black building in Virginia instead of being some James Bond style agent.

<comment><USER20> Do you have any evidence to back up your first paragraph? Anything that has stood up in a court of law? Or are you just regurgitating talking points you've heard from the mainstream press?

</comment></comment><comment><USER21> While it's interesting to have confirmation that we are also engaging in cyberwarfare internationally, I think this most recent disclosure is evidence that he is in way over his head. This seems to be a desperate attempt to use this information to win safe haven over there. The problem is that the fact that we hack them and they hack us is not news, and no one is going to risk trade relations with the US in order to provide safe haven to him because he told them something they already knew. Also, he no longer has access to top secret information. Therefore he has zero value to them as an intelligence asset, and his mere presence is an enormous liability.

While I am grateful for what he has done, and he's welcome to hide at my place anytime, the moment a warrant is issued, he is toast. He has overplayed his hand, and unfortunately he is about to feel the wrath of a very vindictive US Government. He will be locked up, likely for the rest of his life.

<comment><USER22> I will not agree with you that he has overplayed his hand. Obama was just meeting with Chinese leadership not but a week ago with pleas for the government to stop their cyber attacks on US targets. This is exposing that hypocrisy. Additionally he could be an asset to the Chinese government because he likely has MUCH more information than is being made public. However, I think because of his strict ideals to do no harm I think he will be unlikely to cooperate with them on divulging anything that would be of substance to the Chinese government. All that said I think he may still have information he wants to make light of and so, simply because he still does not have access to this information, does not mean he does not having something explosive waiting to be revealed. I think he is a liability to the HK and Chinese governments but they are not easy to kowtow to the US demands simply on the grounds that he has violated laws which HK does not recognize themselves. Grab some popcorn, this will develop much more interestingly than I think you think it will.

</comment></comment><comment><USER23> I can't quite understand what Snowdon is trying to do here. Maybe I'm missing something, but it seems the most likely outcome from this revelation is to alienate his existing supporters in the USA and to possibly seed a cold war with China.

<comment><USER24> There is already a hot cyber war going on with China, it isn't even really 'classified', I am sure the full extent is classified, but its well known that china's cyber force have been relentlessly attacking US government and commercial assets on a large scale for years. That the US is doing the same thing, while not publicly stated, i would think is a pretty reasonable assumption.

He is literally committing espionage, I am glad the privacy related items came out, and frankly I hope the bulk of what he stockpiled eventually becomes public knowledge.

But he is without doubt releasing US government secrets to a 'friendemy' foreign nation with the express intent of personal benefit. thats spying, text book, thats not whistle blowing.

<comment><USER20> Nonsense.

<comment><USER25> At first, I was dismissive of your point of view, but your cogent reasoning convinced me otherwise.

</comment></comment></comment></comment><comment><USER26> So is this his strategy for why he chose hong kong over iceland for the initial leaks, he believes he can get china on his side to offer him protection?

</comment><comment><USER27> Not to go all conspiracy, but maybe it's US Government providing the info in order to discredit Snowden and lay the groundwork for getting rid of him?

<comment><USER24> My Pet conspiracy theory is Snowden's is a US operative, and the whole goal of this operation is to get the US populous to accept this whole surveillance state, and to get congress to fully legitimize these surveillance operations...

<comment><USER28> > get the US populous to accept this whole surveillance state, and to get > congress to fully legitimize these surveillance operations...

that does, frankly, seem the most likely outcome of all this, specific orchestrations aside.

no one really has the stomach to roll back the security state, so after a good catharsis, these programs will get legitimized, we'll accept pervasive surveillance as the new normal, and adjust our lives accordingly.

<comment><USER29> Well you better be very careful about what you say because you don't want to denied entry to the US now do you?

<comment><USER28> Keep your head down and stay out of trouble, yes.

</comment></comment></comment></comment></comment><comment><USER30> Just speculation, but any thoughts on the possibility of this being coordinated FUD put out there to discredit Snowden (although this would require some degree of USG collaboration with South China Morning Post)?

If this were the case it's already worked reasonably well in muddying the opinion / perception here on HN, and would obviously give significant ammunition to the mainstream media.

</comment><comment><USER31> I think its a good thing to bring light on this. I hope this will get both sides to draw up some boundaries for cyber war - at least we might get a relatively peaceful standoff so we can trade with each other.

A standoff is better than all out escalation with increasingly dramatic incursions and dialog where each side loses face, raising political tensions.

We should realize the beneficiaries of cyberwar escalation are the contractor companies coupled tightly to the NSA. I don't think that many hackers are getting rich, comfortable maybe, but not the big bucks.

</comment><comment><USER32> So basically he's trying to get people in HK to like him so they'll fight his extradition.

"Hey HK, the US gov is spying on you too! We're in this together! Please don't send me back."

</comment><comment><USER33> I have to say, it would renew my faith in the competency of government if this was all some elaborate scheme to get an agent close to other countries to plant targeted, specific misinformation, and Snowden is actually some super- spy.

Alas, there are easier ways to accomplish that, an operation this big would never remain secret, and acting in the open like this has to have a hundred other downsides.

</comment><comment><USER24> Dude is going away for a long long time... He honestly better get himself at least 1000 miles deep into the center of mainland china

<comment><USER34> If China grabs him, they will put him in North Korea.

<comment><USER35> Pretty sure that China is not keen on owing North Korea any more favours.

<comment><USER34> Interesting that now North Korea wants to open talks with the U.S., according to a news tweet tonight (June 15th):

[https://twitter.com/NewsBreaker/status/346074606270038017](https://twitter.com/NewsBreaker/status/346074606270038017)

Strictly coincidence?

</comment></comment></comment></comment><comment><USER36> Sounds like Snowden is warning the NSA that there are far more secrets that could be revealed, potentially very damaging to the US. I wouldn't be the least bit surprised to learn of a dead man's switch.

It's one way to make sure the US doesn't insist too much for the right to extradite.

<comment><USER37> > I wouldn't be the least bit surprised to learn of a dead man's switch.

...and secondary time released emails and other publishing events not trusting the Washington Post or Guardian.

</comment></comment><comment><USER20> This is to garner local support.

A HK legislator is backing Snowden because civilian infrastructure in HK has been hacked. This is a criminal offence and must be investigated.

[http://www.cnn.com/video/data/2.0/video/bestoftv/2013/06/13/...](http://www.cnn.com/video/data/2.0/video/bestoftv/2013/06/13/china- support-for-snowden.cnn.html)

There will also be a rally in HK to support Snowden on Saturday.

[http://supportsnowden.org/](http://supportsnowden.org/)</comment>


Family sues Amazon after hoverboard fire destroys $1M house - <AUTHOR> http://www.usatoday.com/story/money/nation-now/2016/10/27/family-sues-amazon-after-hoverboard-fire-destroys-1m-house/92866460/ <comment><USER4> I sincerely hope they win.

Amazon is known to be a disgusting swampy wasteland of intentionally deceptive knockoff garbage. Their warehouse policy of arbitrarily mixing stock is actively customer hostile in practice, and they know it, and they do nothing about it.

But that's not the interesting part. Consumer public need assurance that what they're being sold, NOT what they're buying, is safe. The difference is perspective. Regardless of provenance, it should never be the buyer's responsibility to gamble correctly on product safety.

<comment><USER5> You can buy a power strip with a two prong "figure 8" connector providing three _grounded_ outlets. Try to sell that in a brick and mortar store, it would be closed, razed and the ground salted so they won't even try it next time.

<comment><USER6> Genuinely curious, can you expand on why that power strip is bad, and who exactly would shut down the brick and mortar store?

<comment><USER7> A power strip like that isn't connected to the electrical ground in your house -- that would need a three-pin connection, and it only has a two-pin connection. Despite this, it has three-pin grounded sockets for you to connect appliances to it.

Three-pin appliances require a ground to be safe. By using a power strip like this, you're creating an unsafe configuration with a high risk of electric shock if there's a fault with an appliance.

Also, the little figure-8 connectors can't pass a high amount of current safely, and lots of 3-pin appliances are relatively high draw. That's a fire risk.

I'm not sure who's responsible for the enforcement in the US.

<comment><USER8> > Three-pin appliances require a ground to be safe

Can you elaborate on this?

My MacBook charger has both a two-prong and a three-prong cable. Why would it be any more dangerous to plug my MacBook into this power strip using the three-prong cable than to connect it directly to the wall with the two-prong cable?

<comment><USER5> Your MacBook then is prepared for ungrounded operations. Now take a laser printer, it uses very high voltage on the inside and if something gets dislodged and touches an outer facing metal part and you touch that in turn, you are dead. That's why the outer parts are connected to a safety ground. Do note Xerox warns you not to use a "cheater plug" [http://download.support.xerox.com/pub/docs/4400/userdocs/any...](http://download.support.xerox.com/pub/docs/4400/userdocs/any- os/en/UserSafety.pdf) . Laser printers are also a great example of when that C7/C8 connection is a fire hazard because laser printers need a lot of power when heating up their fusers. This is also why you must not connect a laser printer to a UPS.

<comment><USER9> It's also possible that op has a socket like this, so the grounding is on the side: [http://i.imgur.com/lLM3gQO.png](http://i.imgur.com/lLM3gQO.png)

However, there is also this: [http://i.imgur.com/ftPT7sC.png](http://i.imgur.com/ftPT7sC.png), which grounds from either sides or in the middle for sockets like these [http://i.imgur.com/FnXMndC.png](http://i.imgur.com/FnXMndC.png)

<comment><USER8> No, I have American sockets.

The charger looks like this: [https://support.apple.com/library/content/dam/edam/applecare...](https://support.apple.com/library/content/dam/edam/applecare/images/en_US/macbookpro/45w_l_ac_adapter.png)

And also comes with a longer, three-prong cable that replaces the little two- prong piece: [http://www.ebay.ca/itm/like/172352962036?lpid=116&chn=ps](http://www.ebay.ca/itm/like/172352962036?lpid=116&chn=ps)

<comment><USER7> The MacBook chargers are designed to be safe without grounding.

The grounding pin does have a use, though: some cheap powered USB devices will put out a voltage relative to ground on their '0v' lines. If you touch a MacBook that's plugged into one of these devices, you can get a small shock. If you're using a 3-pin adapter, this doesn't seem to happen.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> Amazon has completely lost control of their own marketplace for electronics and accessories, as I suspect nearly every HN reader has noticed.

Perhaps being held accountable for it will cause a change in policy. It's about time.

<comment><USER11> I've also been annoyed by Amazon lately. I am not sure it has gotten worse over the time, but I _am_ sure that there is a lot of potential for improvement. The product sites are horribly cluttered and often even load _slowly_ (on a 5 year old laptop). The home page is 6 MB in size! The product descriptions are also typically bad and it is often surprisingly difficult to find what you want (e.g. a laptop with specific specs). I always have the feeling there is some dark matter/inventory of things that no one considers, and hence no one writes reviews for, simply because the inventory is so poorly structured and the items have poor descriptions.

<comment><USER12> You missed the part where sellers misrepresent sizes on their products, so you expect to order a human sized rack shelf (usually due to misrepresentative pictures) but it's the size of your foot.

<comment><USER5> I have found 1000+ pound LCD monitors, messenger bags that would fit the Big Friendly Giant and who knows what else.

</comment></comment></comment></comment><comment><USER13> They bought the board on Nov 3, 2015.

Amazon banned Hoverboards from their site on Dec 12, 2015.

The family kept using the board despite this.

The house fire was on Jan 9, 2016.

Amazon announced full refunds on Jan 21, 2016.

The CPSC put out a warning on Jul 6, 2016.

The legal claim makes absolutely no mention whatsoever of Dec 12, 2015, and why the family kept using the board.

Amazon did not notify customers on Dec 12, 2015, only Jan 21, 2016. But this was all over the media - it was the lead story on the nightly news.

I get that this is pretty horrible for them, but I'm not convinced Amazon acted negligently here.

Amazon acted pretty quickly. How could they know ahead of time the boards would be dangerous? In between Dec 12 and Jan 21 it seemed they were trying to find good boards and get information - it doesn't seem like an excessive amount of time to do that.

<comment><USER14> "Amazon did not notify customers on Dec 12, 2015, only Jan 21, 2016. But this was all over the media"

But would it have been _that_ difficult to generate a warning email (if not also surface mail to the ship-to address) to every buyer of an affected product after the 12/12 ban, given that they had all needed data immediately handy?

<comment><USER13> I don't disagree, but I'm not convinced it's worth $30 mil when at that time they were not sure it was necessary to warn people.

At that time it seemed like they felt they could find the good hoverboards among the bad (hence their program of asking for certification). Only later did they realize all of them were bad.

<comment><USER14> Those that faked UL approval tags should have been removed immediately.

<comment><USER13> They were. All of them were removed, and Amazon demanded documentation in order to reinstate them.

</comment></comment></comment></comment></comment><comment><USER15> You know whats really amazing? I bought an aukey charger from amazon that ships with an ETL mark -- similar to UL, a claim that the product is safe and follows generally accepted practices -- that ETL didn't grant. ETL says on their website that it isn't a valid mark! I even told amazon and, while they did refund my money, they continue to sell it.

amazon link: [https://www.amazon.com/gp/product/B013US9FFY/ref=oh_aui_sear...](https://www.amazon.com/gp/product/B013US9FFY/ref=oh_aui_search_detailpage?ie=UTF8&psc=1)

etl warning mark is fake: [http://www.intertek.com/WorkArea/DownloadAsset.aspx?id=34359...](http://www.intertek.com/WorkArea/DownloadAsset.aspx?id=34359756154)

Unbelievable.

Don't buy anything that converts ac/dc, anything you eat, or any cosmetics from amazon.

<comment><USER16> There should be some kind of legality issue with selling something that you have reason to believe is falsely marked with safety approvals, no?

<comment><USER15> I would certainly think so/wish so/assume so.

</comment></comment></comment><comment><USER12> > "We've spent months investigating it and to this day I don't know who > manufactured this product, and it doesn't appear that Amazon does," Anderson > said. He said Amazon charged and shipped the hoverboard.

The amount of fake product complaints I see in reviews on Amazon now is pretty incredible. They really need to figure something out with regard to controlling their inventory.

Uh, also, why is this article playing a 53 second jazzy soundtrack? Is this a new method of fighting off adblockers? You're gonna throw me nice jams instead of your video? I'm down.

www.gannett-cdn.com/360player/comscore/streamsense.220.127.116.11316.min.js:8 getClip() is deprecated. Use getPlaylist().getClip() instead.

<comment><USER17> I watched a vid some time ago about "who exactly invented the hoverboard" \- it was really interesting... I can't find it right now, but if I find it - <USER2> post it.

<comment><USER18> Planet Money on NPR did an episode on that:

[http://www.npr.org/sections/money/2015/11/27/457404184/episo...](http://www.npr.org/sections/money/2015/11/27/457404184/episode-666-the- hoverboard-life)

<comment><USER17> Thats the one -- thanks - for some reason I thought it was a video.

</comment></comment></comment></comment><comment><USER19> I am finding it increasingly difficult to know how much to trust items to buy on Amazon.

It used to be fairly obvious which was a "third party seller" and which was Amazon (as in one company that sourced manufacturer products, bought them and shipped them). These days I basically click through to purchase and try to guess from the amount of P&P if they are Amazon, fulfilled by Amazon or some guy in his garage.

And why can't fulfilled by Amazon actually be on the product description !

This is another example of Internet enabling business models that law and society have not quite kept up with. Amazon probably don't have a strategy of confuse the shit out of the customer, but they can twist their business model so many times that's what has happened.

<comment><USER20> _And why can 't fulfilled by Amazon actually be on the product description !_

That's weird. In the UK it absolutely is: you can tell on the product page whether you're buying something sold by Amazon themselves or "fulfilled by Amazon". Items sold by Amazon have the text "Dispatched from and sold by Amazon" in the product description whereas an item from another seller will contain the text "Sold by <seller name> and Fulfilled by Amazon" or "Dispatched from and sold by <seller name>".

EU consumer regulation is notoriously stricter than in the US, so maybe it's a legal requirement?

<comment><USER16> It's the same here in the US. People just don't always know to look for "Ships from and sold by Amazon" under the item's initial description.

<comment><USER10> I know to do that and I have still had bad experiences. I am pretty sure Amazon does not have a simple/transparent way to do this. They certainly don't have a search tool for this, which would be necessary for it to matter anyways.

<comment><USER20> The following works in the UK, but weirdly only for some seraches: Search for "item of choice" in top search bar. When the results come up, scroll down until you can see the "Seller" section on the LHS. Tick the box at the top marked amazon.co.uk

Why some searche results have the Seller list on the LHS and some don't I've no idea.

</comment></comment></comment></comment></comment><comment><USER21> Interesting news. The result of this verdict may set a precedent to further curtail online sales of electronic goods from questionable manufacturers (read relatively cheap).

The only question is whether this will lead to a more booming underground economy for folks who are more risk friendly.

<comment><USER22> It doesn't even need to go underground. Just purchase off Ebay with direct shipping from China.

<comment><USER23> [https://www.aliexpress.com/](https://www.aliexpress.com/) and [https://www.dx.com/](https://www.dx.com/) fit the bill better.

</comment></comment></comment><comment><USER24> I've always thought Amazon were very exposed with their handling of the 'fulfilled by Amazon' stock.

AFAICT - with these items, a seller can send them in bulk to an Amazon warehouse and they'll all be put into the same box. So, a (for example) set of Sennheiser headphones sold by Amazon themselves can be in the same box as a fake set sold by a third-party seller, as long as the EAN is the same.

I've had this previously with Sennheiser headphones - sold by Amazon, on their site, and I got the fakes. I just didn't think about the idea of it including items where the fakes were unsafe.

<comment><USER16> Wait, what? Do you have a source for this? I am bewildered by the thought that Amazon dumps third-party stock into their own inventory.

<comment><USER12> You can easily send products to Amazon to have them fulfill the sales with their shipping/warehouse network.

[https://www.amazon.com/gp/help/customer/display.html/?nodeId...](https://www.amazon.com/gp/help/customer/display.html/?nodeId=201171200)

<comment><USER16> Right, but that doesn't mean they combine your items with their own items, I hope.

<comment><USER12> It tells you it's fulfilled by Amazon. They're referring to the shipment. A lot of Prime members don't buy non-Prime/Amazon shipped items because of the shipping delays. I myself am one of them.

</comment></comment></comment></comment></comment><comment><USER25> This doesn't surprise me at all, I've completely stopped shopping at Amazon since it is impossible to determine the quality of a lot of the goods on their site. It seems to take longer to research what is worth buying versus going to the store and finding something that looks like it is good quality.

</comment><comment><USER26> Sue the pants off of them... maybe it will get Amazon to crack down on what they're willing to sell (or host the sale of) before it's a completely lost cause.

I used to really appreciate Amazon, but in the last couple of years it's become a bad joke.

</comment><comment><USER27> While is tragic they had to rescue their kids from a burning house .. this is funny .. " It alleges the family was sold a counterfeit product from China instead of a brand with a Samsung lithium ion battery they believed they were buying from Amazon"

<comment><USER28> Well, the fire is proof that it was a genuine latest generation Samsung battery. /s

</comment></comment><comment><USER29> Suddenly music starts playing on that site. The internet is so broken.

</comment><comment><USER30> Did they not have insurance? The law attempts to compensate someone for a loss, it doesn't allow someone to profit from losses.

</comment><comment><USER31> > instead of a brand with a Samsung lithium ion battery they believed they > were buying from Amazon

Plot twist: It was the genuine Samsung product...

</comment><comment><USER32> I'd be glad to see Amazon (or anyone selling faulty products) pay for the damages.

I don't understand how 'destroyed a 1mil house' turns into 30mil. This "pay for the immaterial damage you've done" (assuming that the 1mil contains most of the furniture etc. and that the rest of their belongings don't come close to 29mil dollars) attitude is strange, hard to grasp here.

<comment><USER33> If you don't fully grasp the American legal system it does seem silly. Basically, the idea is that this falls in the civil court system. I.e. you're wronged, so you sue for damages. However, one of the main reasons why you can sue for 30M and not for the price of the loss is the concept of punitive damages. If Amazon loses 30 and not 1M,they're less likely to allow the wrong to happen so easily again.

In places like France, you can't sue for 30M, but you can maybe sue for what you lost. At the same time, the criminal laws are more likely to affect corporations via regulations and even legal action. Thus, the philosophy in the US is, less government regulation, more emphasis on non governmental interference in private entities working out their differences with the government only as impartial judge.

Hence it's very important for individual consumers that we be able to sue for these ridiculous sums. It's the only disincentive for the Amazons and other powerhouse corporations from getting away with such things as a fire caused by negligence, and their writing it off as a business expense. The expense needs to be high.

<comment><USER32> Thanks a lot for the explanation. I think what I mostly don't get is

a) isn't "replace 1mil+ for a product that costs 100" (pulled that number from thin air, no clue what these things cost) already punitive? The replacement cost eats the profit for a HUGE number of sales already, no?

b) the arguments for the _specific_ number seem completely arbitrary and scaling in ways I cannot understand. It's not "our home was destroyed, pay to replace it and pay for another house on top, because we have to replace all the stuff inside and the experience was crazy". It's "our home was destroyed, pay us enough to buy 2 dozen of these homes with a lot of room to make every single one of these 24 homes look nice as well".

That scales in ways that are utterly alien to me. If you can somehow defend your request for 30mil, why not 50? 100? As soon as you enter 'emotional damage' (hard to quantify) or punitive damages (the more the better, right?) I fail to see how you come up with these numbers.

Then again, I'm not saying it's silly. It's just completely confusing for me and I DO feel better about the local system. It is an entirely different culture and I'm trying to point this out in a thread that so far is full of "Yeah, makes sense" posts. Internationally it might not be that one sided.

<comment><USER28> a) The manufacturer must at least reimburse the damages he's done with his product.

Otherwise anyone can manufacture explosive broken fire-trigerring devices. There's no point of making a device 'safe' if a $1 device can't pay back more than $1. The only incentive is to make stuff cheap, ignore safety, risks and regulation.

If the manufacturer must close, so be it. Better that than dead people.

b) The scale is American. They sue for very high amount (higher than just the reimbursement), as the previous poster was explaining.

<comment><USER32> Your first point misses what I was saying (or I was bad at saying it). Of course you pay for the damages caused, not the product's price. In the case here I expect someone (Amazon? Whoever) pay for the 1 million plus to replace the house.

I brought up the product price just because I feel that paying more than a million in damages for a product that costs (say..) 100 dollar is already punitive in terms of business. Whoever has to pay even 1 million probably will be quite unhappy about carrying that product, without any further inflation: The price for this accident eats the profit for a BIG number of sales - ignoring the PR issues.

So: Sure, if you sell a $1 device that causes $1,000,000 in damages, pay a million (or more). But paying a million (or more) caused by a $1 device seems already like a decent way to make the business .. reconsider some practices?

<comment><USER34> If you are making $100 milling in profit off of a faulty device, then $1 million is just the cost of doing business. A penalty has to be painful enough to the company that they don't want to do this again. Given how large Amazon is, and how their marketplace is a nightmare of consumer fraud, I think $30 million is not enough of a penalty to make them clean up their act.

For more on this, I suggest reading "Unsafe at Any Speed" by Ralph Nader.

<comment><USER32> Are you arguing about the case in this article or about general issues you have with the Amazon marketplace? Text is a crappy medium, this is not my native language, but you seem to have a problem with Amazon regardless of the issue in the article?

If 30 million isn't enough, what would be enough? "All the profit of this product"? "All the revenue based on this product"? And .. would you award that to some random party? Because that's what this is about as well, no? Someone was hurt (financially, emotionally) and now there's supposed to be a pay out. Would you suggest giving the "victims" 100 houses for the one house they lost? One party/family should be able to own a small village of luxury houses to hurt Amazon?

I'm going to retire from this thread - it's past midnight and I was confused a long time ago. Further arguments make this process only _more_ alien and I certainly don't want to convince the US population that they Should Not Do This.

But there's no way in hell to integrate this process with my upbringing - <USER3> feels entirely wrong and far too much like a cash grab. Chalk it up to cultural differences?

<comment><USER33> Cultural differences? Maybe, but not in the way you think. It's cultural in the political philosophy more than anything else.

Let's summarize it thusly: the U.S. regulatory system, while not totally a free market, is a lot less stringent and red-tape-heavy than places like the EU. HOWEVER, you need a disincentive for 'bad actors' to cause harm, and in the U.S. we rely more on tort law ("wrongs") than in the EU, where things are more tightly regulated (in theory). It's a matter of degree. Rather than risk fines or even, in some cases, jail time, in the U.S. companies are afraid of getting sued, and thus you see all those 'don't put your finger in the socket' 'this knife is sharp, be careful' type notices on consumer products. The cultural aspect is around the idea that government shouldn't interfere overly in the market. The country was founded on the ideas of philosophers like Adam Smith.

 _If 30 million isn 't enough, what would be enough? "All the profit of this product"? "All the revenue based on this product"? And .. would you award that to some random party? Because that's what this is about as well, no? Someone was hurt (financially, emotionally) and now there's supposed to be a pay out. Would you suggest giving the "victims" 100 houses for the one house they lost? One party/family should be able to own a small village of luxury houses to hurt Amazon?_

Good question, and sometimes class action lawsuits (involving hundreds, or even thousands of people) happen. Note, the point of 'punitive' is punishiment. Disincentive. Don't do it again. It's up to lawyers / juries to 'decide' what the renumeration is. As to whom goes a bulk, well, you can rest assured law can be a lucrative profession for a reason.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER35> is it possible to be a blind marketplace for products without liability? like ebay? but without the auction part?

if it is, I wonder how amazon can transition into having ebay levels of liability, and what that would entail.</comment>


Mark Zuckerberg's sister sorta threatens to take down bar's facebook page - <AUTHOR> http://davidkendall.tumblr.com/post/602010120/randi-zuckerberg-would-be-mark-zuckerbergs

<comment><USER2> <http://www.youtube.com/watch?v=WWKG6ZmgAX4>

<comment><USER3> The difference is Obama has charisma

</comment></comment><comment><USER4> This is really old, nice linkbait though.

<comment><USER4> From May 8th, 2009: [http://gawker.com/5245852/randi-zuckerbergs-excellent- new-yo...](http://gawker.com/5245852/randi-zuckerbergs-excellent-new-york- adventure) (at the bottom)

</comment></comment><comment><USER3> Marlon Brando is a much better Don Coreleone than the Zuckerbergs.</comment>