The Infrastructure Behind Twitter: Scale - <AUTHOR> https://blog.twitter.com/2017/the-infrastructure-behind-twitter-scale <comment><USER2> 100,000's of servers for 100,000,000 of messages/day ?

I understand that half the servers aren't even doing messages, but, isn't WhatsApp doing _2 orders_ of magnitude more messages with _3 orders_ of magnitude (?) fewer servers?

Is that right? I'm curious how one would justify 10,000X worse?

So for each message, 10,000X more equipment is needed?

<comment><USER3> Also:

\- whatsapp doesn't have to allow browsing the entire history of their billions of messages;

\- whatsapp doesn't have tags. A message can go not only to 1000000 users, but also to so many apps requesting updates for one tag.

\- Twitter allows advanced search, where you can browse, in real time (or down to the entire history), a complex combination of people, tags and free text. With settings such as choosing the lang or the date.

\- Whatsapp has a list of messages. But Twitter has a graph : message can be RT again and again, answered to and liked.

\- all those features have some impact or the other on the way the tweets are displayed to the user.

\- Twitter's API is much more used than Whatsapp's.

<comment><USER4> WhatsApp also does not need ad related analytics.

<comment><USER5> That's a big leap. They don't have to actually show ads for that analytics data to be extremely valuable.

</comment></comment></comment></comment><comment><USER6> The day before, Discord did high-quality a write-up on why they chose Cassandra [1], and now this post hits explaining how one of the world's most popular and trafficked service has engineered their infrastructure; it's like a dream.

I'll echo the praise I wrote earlier, that insights like this aren't only some of the best content to hit HN, but become some of the most valuable resources for designers who have yet to face a scaling issue, but know they will soon.

Since you have developed custom layers on top of open-source software to fit your particular usecase and load profile, and host all this in-house, have you considered monetizing your infrastructure for outsiders who may have similar needs?

Today, one has limited, unpleasant choices: either pay out the nose for something like AWS or Google Cloud to get elastic scaling and the captive storage systems that can be made to handle these kinds of workloads and still have to write a fair bit of custom glue to get all pieces to play nice, or you can build out the servers yourself, but have to employ dedicated talent with the requisite expertise. Either way, the barriers are fairly steep; you could tap into an under-served market should you choose to sell IaaS (edit: or, more accurately, PaaS). Has this conversation come up in the past?

[1] [https://news.ycombinator.com/item?id=13439725](https://news.ycombinator.com/item?id=13439725)

</comment><comment><USER7> This is a fairly technical analysis, and the terminology used in many cases is above what I know about networking. But the one quote that will stick is this.

"There is no such a thing as a temporary change or workaround: In most cases, workarounds are tech debt."

<comment><USER8> Tech debt is a choice. Sometimes you'll want to embrace some level of technical debt in order to bring something into production quickly, with the understanding that you will fix later. It's part of a triad with Speed and Quality.

That may not be the choice Twitter has made in this instance, but it's a viable choice nonetheless. Defaulting to tech debt = evil wrong imo.

<comment><USER9> The problem with technical debt versus financial debt is that the latter has a monthly mandatory cost ( interest payments ) that can't be ignored and which is visible all the way up to the C-level, whereas middle-management can keep obscuring the presence of technical debt and pushing its repayment out to the right.

Essentially it's a 'free' internal debt, regardless of how often architects and developers complain of its cost.

Thus in a contest between doing something right, but expensively, versus good- enough-for-now but technically-constrained the latter will usually win.

<comment><USER10> It's not "free", it's just much harder to measure.

The cost is reduced development velocity, and perhaps reduced systems stability.

<comment><USER11> Which developers get blamed for, even though it was a managerial decision to take on the debt.

Financial debt has clear cost; technical debt doesn't. So it seems 'free' to the non-technical.

</comment></comment></comment></comment></comment><comment><USER12> It strikes me that so much of the components they use (e.g. under "Storage") are in-house built (several dbs, blob store, caches, etc). Is that because at that time equivalent solutions didn't exist? Is that because Twitter suffers from NIH?

<comment><USER13> I built custom storage for Twitter back in the 2010-12 period. There wasn't much off the shelf in those days that worked out-of-the-box at scale besides Cassandra, and Twitter had a well-documented attempt at using Cassandra for primary workloads that failed due to the amount of variance in IO and latency for high-volume workloads.

Most of the time, we were building custom distribution layers on top of open source storage (e.g. memcached/mysql/redis/etc). I think blobstore was the first thing twitter put in production that was mostly custom, followed a year or two later by manhattan. I'm not sure if there's even now good open-source competitors for those projects, largely because any reasonable smaller company uses s3 or dynamo.

There's plenty of open-source things twitter created, or nurtured out of the existing ecosystem, from mesos to memcached to some of the hadoop/scalding/parquet stuff.

<comment><USER14> The flip side of that imo is that if you don't have Twitter-scale needs for the specific things they've optimized their infrastructure for, you probably don't need their solutions :)

</comment></comment></comment><comment><USER15> I know nothing about storage, so I'm a bit confused about why Twitter needed:

1\. Hadoop

2\. Graph

3\. Redis/Memcache

4\. Blobstore

5\. SQL variants

(and a few others).

I do see that the post has a short snippet briefly describing what they're storing, but I'd be curious to know why (speed, cost, latency, space tradeoffs/constraints).

Also, if any more experienced folks want to chime in: Elixir/Erlang is "built for concurrency" as they say. I'd love to hear people's opinions on their one sentence simplifications of what kind of situation Hadoop/SQL/Redis/etc should be used for (similar to how Erlang is best used for situations where concurrency and fault tolerance is desired). In particular, is there a "Code Complete" type book for storage?

<comment><USER16> That seems a pretty standard stack, I don't know about twitter but I can say why I'd use them in my experience:

5\. SQL variants: This is where you keep your source of truth data needed to serve request. It's flexible, reliable but moderately fast. The backbone of your infrastructure and you can consider everything else as enhancements to it to overcome specific limitations.

4\. Blobstore: Your SQL database is not going to be good at storing large binary blobs such as files or images.

3\. Redis/Memcache: Your SQL database may be too slow for some requests and/or it's data model may require too much work in the API layer to align with the final presentation model. So you want a blazing fast store that you can use for that final presentation view when requests don't differ from each other.

2\. Graph: SQL databases aren't the best for graph queries or data (I mean you can hack it but it's going to be slower and uglier code wise) so if you need to look at data as a graph something dedicated makes sense.

1\. Hadoop: You're getting lot's of log data from users, what they do, when they do it and so on. You want to analyze this data, build models (who to recommend for people to follow for example) and use it to probably serve ads (gotta pay the bills somehow). You could put this in SQL but it's a lot of data and you'll be doing a lot of read requests which will end up being slow and messy in SQL. Hadoop, on the other hand, is well suited for this type of data and workload. Large slow read heavy queries on data that is appended to but not modified in place.

<comment><USER17> FTFA: >Graph: Our legacy Gizzard/MySQL based sharded cluster for storing our graphs. Flock, our social graph, can handle peaks over 9 million QPS, averaging our MySQL servers to 30k - 45k QPS.

Isn't storing a GraphDB in MySql a Bad Idea? Graph queries are extremely poorly suited for relational databases.

<comment><USER18> I don't work on GraphDB, but basically the situation is this: 1) yes, it is not a great use case for MySQL, but that's how it started, partly because Twitter needed join on its graphs; 2) legacy systems die hard, especially at scale- Twitter is working on a better solution, but for now Flock is still what's running in production.

<comment><USER17> Thanks for explaining. I had a similar situation -- storing a graph db in postgres. The performance sucked, but we didn't have much of a choice.

</comment></comment></comment></comment></comment><comment><USER19> Lots of CIO-type buzzwords and acronyms in this, including many coined by Twitter themselves. More worthy of a skim than a rigorous reading. But interesting for a bird's eye view of how complex Twitter is behind the scenes.

</comment><comment><USER20> They mention that they used to base their geo routing on the location of the client's DNS server but have moved to BGP Anycast. I've heard that there can potentially be routing issues for long-running connections using anycast to end users, is anybody else doing something like this and do these issues happen in practice?

<comment><USER21> TCP Anycast - Don't believe the FUD

Matt Levine (CacheNetworks), Barrett Lyon (BitGravity), Todd Underwood (Renesys)

Operational experience with TCP and Anycast

[https://www.nanog.org/meetings/nanog37/presentations/matt.le...](https://www.nanog.org/meetings/nanog37/presentations/matt.levine.pdf)

</comment></comment><comment><USER22> > Fast forward a few years and we were running a network with POPs on five > continents and data centers with hundreds of thousands of servers.

That seems high given Twitter's size and the hardware distribution pie chart they showed. Does anybody have an idea how this compares?

<comment><USER23> I'm working for a very successful Internet company with 9 figure user count. We have 7 or 8 POPs on 4 continents and run a low 5 digit number of servers.

This being said: we do not have to deal with anything like the network effect and lifestyle type usage that twitter has. Once you pass a certain size and complexity of infrastructure it happens really easily that economy of scale doesn't help and the complexity of everything having to be distributed works against your efficiency big time. So I'm not shocked by their numbers.

</comment></comment><comment><USER24> | We have over 100 committers per month, over 500 modules, and over 1,000 roles.

| we were able to reduce our average Puppet runtimes on our Mesos clusters from well over 30 minutes to under 5 minutes.

This isn't just tech debt .. it's poorly designed, poorly thought, poorly architect-ed and poorly managed in the first place.

Is it because Twitter cannot find good talent because of its falling stock?

</comment><comment><USER25> I've talked/pitched (full disclosure: got a "no") some of the bigger names behind the product at Twitter. I was a little disappointed because they seem to be proud of their at least $15M+/month server costs which is partly driving their company into the ground (the user facing product hasn't improved despite "re-engineering" of their backend, at non-substantial price differences, and lack of innovation for consumers have made them lose everyone to Snapchat or anti-censorship sites).

Of tweets alone (about 200bytes per tweet), over the last decade, they probably have about 3 petabytes. Unknown to them (because of the aforementioned pride) they have 1.5 petabytes a month of free storage/caching they aren't even touching. If they switched to a P2P model like IPFS or (full disclosure: I work here) [http://gunDB.io/](http://gunDB.io/) , but Twitter seems determined to stay as a centralized monolith. Which is too bad, because that has now become their own death - the regime change is happening and decentralized services will win instead.

Edit: Compare against 100M+ messages (1000bytes each) for $10 a day. 2 min screencast here: [https://www.youtube.com/watch?v=x_WqBuEA7s8](https://www.youtube.com/watch?v=x_WqBuEA7s8) . Even if you multiplied the feature set by 10K times, you would still be saving $12M+ a month. At this rate the Discord guys (pushing 120M+ messages a day) are doing way better - there post is on top of HN right now too, see my comment there as well. And they only have 4 backend engineers.

<comment><USER26> Why the downvote? The post contains valuable architectural links worth a discussion

<comment><USER27> 1\. Because it's a .js database.

2\. Because the website does a poor job explaining what problem it solves.

<comment><USER28> Agreed, their website definitely turned me off.

Quick tip for them: Having a section for showing that a bunch of people are using and "Trusting" the product is good, but your's seems to mostly (4/7 from what I can see) be the investors behind your product.

Edit: Looked at the names more, and it seems 7/7 are individual investors or VCs

[https://angel.co/gun](https://angel.co/gun)

Which is all well and good, but maybe the wording should be "Investors and Backers" rather than "Trusted By".

<comment><USER25> Sorry the website turned you off :( what recommendation do you make for balancing both enterprise and developers? Currently we have two website, the other for developers: [http://gun.js.org/](http://gun.js.org/)

Better links:

(again) [https://www.youtube.com/watch?v=x_WqBuEA7s8](https://www.youtube.com/watch?v=x_WqBuEA7s8) 100M+ msg a day for $10.

Doing distributed machine learning: [http://myrighttocode.org/blog/artificial%20intelligence/part...](http://myrighttocode.org/blog/artificial%20intelligence/particle%20swarm/genetic%20algorithm/collective%20knowledge/machine%20learning/gun- db-artificial-knowledge-sharing)

30M+ ops/sec performance testing: [https://github.com/amark/gun/wiki/100000-ops-sec-in- IE6-on-2...](https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2GB- Atom-CPU)

1 minute demo of seamless multi-server failover during a live chat app: [https://youtu.be/-FN_J3etdvY](https://youtu.be/-FN_J3etdvY)

1 minute demo of recovering from a complete server meltdown: [https://youtu.be/-i-11T5ZI9o](https://youtu.be/-i-11T5ZI9o)

You are right - trust should be proven by demonstration, not the blind faith of others.

We're also deploying out to customers with 1.5M users in production and customers with a product being shipped to 1.5K homes. And we're nearing our production-ready v1.0 stable release.

Feedback like yours is helpful, so please hit me up with any other critiques or ideas on how we can explain ourselves better. Thanks for jumping in on the convo :)

<comment><USER28> Thanks for being receptive. My original recommendation is all I have right now.

>maybe the wording should be "Investors and Backers" rather than "Trusted By"

<comment><USER25> Done! Thanks :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER29> They switched to a BGP anycast model for twitter.com, which implies TCP. I'm curious how they deal with situations where route preferences change in intermediary ISPs mid-TCP stream. Does the new server reject the TCP connection or are they synchronizing TCP sessions across clusters?

</comment><comment><USER30> No Ansible? In an older (2014) Ansible video they claim Twitter is using Ansible but I only see Puppet mentioned.

<comment><USER31> Only ever used puppet during my time there, fwiw

</comment></comment><comment><USER32> If Twitter can manage to operate at profit, then I might be interested in Twitter's infrastructure.</comment>


Jepsen: Testing the Partition Tolerance of PostgreSQL, Redis, MongoDB and Riak - <AUTHOR> http://www.infoq.com/articles/jepsen

<comment><USER2> The exchange between Antirez and aphyr following the post about Redis sentinel is a fascinating comparison between two engineering approaches. Antirez makes a qualitative argument ([http://antirez.com/news/56](http://antirez.com/news/56), especially [http://antirez.com/news/56#comment-910996445](http://antirez.com/news/56#comment-910996445)) about the behavior of the system in some 'real world' where complex network partitions are rare. On the other hand, aphyr made a much more theoretically sound argument (including using TLA+ to demonstrate the validity of his argumement) in his post ([http://aphyr.com/posts/287-asynchronous-replication- with-fai...](http://aphyr.com/posts/287-asynchronous-replication-with- failover)).

Despite having a huge amount of respect for Antirez and Redis, I strongly believe that the approach aphyr took is the one we are going to need as we build larger and more complex systems on unreliable infrastructure. Our engineering intuition, as excellent as it may be for single-node systems, almost always fails us with distributed systems. To get around this, we need to replace intuition. The tools that aphyr uses, such as TLA+ and carefully crafted counterexamples and diagrams, are an extremely good start in that direction. Getting a computer (in this case TLA+'s model checker TLC) to exhaustively test a design specification is very powerful. Comparing those results to the ones that we expected is even more powerful.

The comment made by Metaxis ([http://antirez.com/news/56#comment-905001533](http://antirez.com/news/56#comment-905001533)) on Antirez's second reply is very good. Especially:

> I think your attempt to differentiate formal correctness and real world > operations is deeply flawed and amounts to asserting anecdote - that what > you have observed to be common makes for good design assumptions and better > trade off decisions.

> Allow me to counter: Real world operations will inevitably tend to approach > formal correctness in terms of observed failure modes. In other words, over > time, you are more and more likely to see edge cases and freak occurrences > that are predicted in theory but happen rarely in practice.

This closely matches my own experience. Just because I don't believe a network can behave in a particular way, doesn't mean it won't. The real world is full of complex network partitions and Byzantine failures, and our systems need to be safe when they happen

<comment><USER3> That all may be true - but is it asking the wrong question?

It can be tempting to stand on an ivory tower and proclaim theory, but what is the real world cost/benefit? Are you building a NASA Shuttle Crawler- transporter to get groceries?

<comment><USER2> > It can be tempting to stand on an ivory tower and proclaim theory, but what > is the real world cost/benefit?

That's an important question. The real world costs of fulling understanding the behavior of a system like Redis, amortized over all the users of that system is likely to be very low. That's a huge benefit of using well-tested pieces (whether they are Redis, Cassandra or Oracle) as the foundation of systems.

The important question to ask is what you are losing. Losing availability in these complex cases is acceptable for the vast majority of applications. You can save a lot of complexity and cost there. Losing safety is much more of a problem, because the effects can be very long lived. Once you have put inconsistent data into your database, you are in a world of hurt for potentially a very long time.

I think that it's actually cheaper in the long run, even for small systems with small goals, to build on top of safe components. The costs of writing code to deal with the shifting reality of inconsistent data is higher, as are the costs of not writing that code.

I don't think that proving the safety of protocols, testing those safety properties once implemented, and understanding failure modes is "ivory tower" at all. It's just good engineering practice.

</comment></comment></comment><comment><USER4> Fantastic in depth article. In my opinion, the tests he perform show not so much drawbacks of the tested systems, but more the fact that defining them in terms of the CAP theorem can be misleading. For example, a CP system should, in case of a partition, wait until the partition is resolved, potentially forever. This is not useful in practice, where almost always some timeout is used. This is why, if I'm interpreting the results correctly, not even a Postgres running on a single node can claim to be CP.

<comment><USER5> _a CP system should, in case of a partition, wait until the partition is resolved, potentially forever_

Not always. In the presence of up to N/2-1 failures (or alternatively, so long as a connected majority exists), many CP systems can continue to operate fully on some nodes. Other nodes may not be able to service requests, though.

 _This is not useful in practice, where almost always some timeout is used._

Exactly--if you broaden your time horizons for a request so that messages _always_ arrive, you'll be OK. As far as I can tell, this is NuoDB's approach. On the other hand, it means the database blocks some or all requests, potentially forever. This is essentially unavailability, with the added fun that all those requests might suddenly complete well after the client thinks they failed. When your clients need to hand a response back in a matter of seconds, you _need_ to provide for timeouts--which brings you back to the problem of dropped messages again. :)

 _Postgres running on a single node can claim to be CP._

The Postgres node proper is (assuming you're using the right transaction isolation level) linearizable. The client and the server, considered together, may not be CP--depends on how you interpret network failures. I don't think there's any real way around this--to the best of my limited knowledge, it's a consequence of the FLP impossibility result.

<comment><USER6> So, essentially nothing can ever be CP, not even a single node, since browsers and humans will never wait forever for something, correct?

<comment><USER5> C is a _safety_ property ("nothing bad happens"), but what you're talking about is (kind of) a _liveness_ property ("something good happens"). Real systems need both, so we have to discuss what to do about making correct progress in the face of failures.

Theoretical CP systems can't just block indefinitely, because the network can delay messages for 1 second longer than you choose to wait. Real-world CP systems can usually assume the network recovers--but it might be days before it does, so they'll need some strategy to handle all the requests in the mean time.

Typically, systems choose to time out or otherwise abort certain requests-- exactly which depends on the logical topology of the system. For instance, if the primary node for a given shard of the DB is unreachable from a client, a CP-over-shard system would fail those requests--but not others. A totally CP system like ZK might fail requests in any minority components, but continue to work just fine if a majority component is preserved.

Bottom line: you can build CP systems. The only constraint a CP system has to obey is linearizability: successful operations must appear in the same order on all nodes.

</comment></comment></comment></comment><comment><USER7> I think (╯°□°)╯︵ ┻━┻ and ヽ(´ー`)ノ need to be declared as constants in all logging libraries and used accordingly.

<comment><USER8> Let's hope for some tight integration with Android's Log.wtf method:

[http://developer.android.com/reference/android/util/Log.html...](http://developer.android.com/reference/android/util/Log.html#wtf\(java.lang.String%2C%20java.lang.String\))

</comment></comment><comment><USER9> Interesting, but needs a summary to help conceptualize the details which it slogs through.

<comment><USER5> TL;DR: designing distributed systems is hard, so think carefully about your problem space, required invariants, and risk tolerance; then test your systems accordingly.

<comment><USER10> TL;DR: first rule of engineering reliable distributed systems: "do not distribute"</comment></comment></comment>


David Miranda, schedule 7 and the danger that all reporters now face - <AUTHOR> http://www.theguardian.com/commentisfree/2013/aug/19/david-miranda-schedule7-danger-reporters <comment><USER2> Jaw: on floor.

 _The mood toughened just over a month ago, when I received a phone call from the centre of government telling me: "You've had your fun. Now we want the stuff back." There followed further meetings with shadowy Whitehall figures. The demand was the same: hand the Snowden material back or destroy it. I explained that we could not research and report on this subject if we complied with this request. The man from Whitehall looked mystified. "You've had your debate. There's no need to write any more."_

<comment><USER3> Unless I'm misinterpreting this, those are paraphrases and not quotes. There's no feasible reason to believe a government official is overtly villainous or foolish enough to say something like "You've had your fun."

<comment> UVB-76 It does sound _exactly_ like the type of goading comment a government official would make to a journalist off the record.

<comment><USER4> All of this controversy aside, do you have any real reference point for what a conversation between a government official and a journalist sounds like, let alone a goading one at that?

<comment><USER5> Yes; take what happens at the local politics level, and amplify it up several orders of magnitude.

</comment></comment></comment></comment></comment><comment><USER6> Is it just me it does anyone else feel like this is one of the most important articles posted in... decades?

The guardian is obviously put at grave risk for revealing that UK spooks have gone so far as to oversee destruction of inconvienient materials in the London office, threatening legal action if this material is not dropped.

I applaud the guardian for their bravery and excellence in reporting.

<comment><USER7> I grok a different subtext:

It's a journalist's job to know where the skeletons are buried, and where the escorts keep their address books. Journalists also have ready access to dictionaries, with which to clarify the meanings of words. Such words include "mutually", "assured" and "destruction".

<comment><USER8> Sadly the way US and UK are mishandling the whole thing can lead to a lot stuff leaked that could possibly cause real harm.

</comment></comment></comment><comment><USER9> There is something truly entertaining about the insistence they smash the hard drives.

The number of computer-illiterate C-level assholes I have met all insist on applying analog world logic to their digital decision making. When you try to properly correct them for wasting their time or embarrassing themselves to end up on sites like HN, they lose it. I find it funny how badly such people in positions of government corporate power need to prove to themselves they have any recourse in contexts or fields where they are too stupid to find a proper solution to their own dilemma.

<comment><USER10> This is not entertaining (or, reading into subtext, absurd) at all. It is widely accepted that complete physical destruction of hard drives is required to avoid future forensic recovery, which is what Rusbridger is referring to when he mentions "passing Chinese agents."

<comment><USER11> I think he should with a completely serious face comply, deliver the disk to them and then smash it with a sledgehammer right in front of their eyes.

Now of course he should have copies spread in multiple locations. But, hey, they just asked for the information to be returned and destroyed, just comply, very easy.

Sometimes it is worth taking pleasure laughing at the stupidity of those in power. That has to be exposed alongside malice and corruption.

<comment><USER12> Reminds me of the many times I emailed spreadsheet files to marketing, saying, "I need that back when you're done"

Generally they got it, but it took some of them a while.

<comment><USER13> Reminds me of the bad old days of locking VCS.

</comment></comment></comment></comment></comment><comment><USER14> Rusbridger responding to an AMA question on reddit a week or two ago: "Various difficulties have been put in our way (sorry to be vague, but this is an ongoing story). We're working round them. And, yes, more to come....."

[http://www.reddit.com/r/IAmA/comments/1jf6uc/i_am_alan_rusbr...](http://www.reddit.com/r/IAmA/comments/1jf6uc/i_am_alan_rusbridger_editor_in_chief_of_the/cbe31e7?context=3)

</comment><comment><USER15> I am not entirely certain about the history of how this happened, but in the US "the press" has taken on an unintended meaning - to refer to the fourth estate. So somehow we have created a privileged class of citizens who enjoy special constitutional protections. Originally, and rightfully, "the press" refers to the the machine used to print broadsheets (not the people who do the printing) - and so freedom of the press would probably be best interpreted as "freedom of _anyone_ to use _technology_ to disseminate information", which is one step beyond "freedom of speech".

<comment><USER16> A worthwhile read on the British history of this subject is "Power without Responsibility: Press, Broadcasting and the Internet in Britain"

My biggest take from this is that your notion that "freedom of anyone to use technology to disseminate information" as upheld by the printing establishment is significantly undermined by the business of printing itself - whereby the cost of disseminating information has increased consistently to make a truly "free" press completely prohibitive to the average person. The book goes into great detail about how this happened and the nature of how the internet effects this pattern. I would highly recommend this to anyone reading this thread.

<comment><USER15> yeah but the average person doesn't need to disseminate information to maintain a free society. It's totally fine to have a community of news mavens that supplies most of the information. What is not okay is a process of accreditation, either state-run (or less worrisome) an independent process - that affords special roadblocks to get protected status.

<comment><USER16> Common Sense[1] might disagree with you on that one.

If we are talking about truly "disseminating information" I think we need to consider any form of broadcast or public proclamation and so as we approach the 50th anniversary of a certain speech[2] I would say the dissemination of information by an average person is absolutely fundamental to the checks and balances required to construct and maintain a free society. But therein lies the rub - just swamp the channels with noise and you raise the bar for actually transmitting a meaningful signal. This is what I was alluding to in my other comment about the way the internet has effected this age old pattern.

It's not clear what your second point is saying - it's okay to have only experts informing the public, but they cannot be afforded "special... protected status"? Like parliamentary privilege? "The fourth estate"? Freedom of speech? The right to protect their sources? I'm genuinely curious about what you mean, sorry if I've misunderstood completely.

[1: [http://en.wikipedia.org/wiki/Common_Sense_%28pamphlet%29](http://en.wikipedia.org/wiki/Common_Sense_%28pamphlet%29)] [2: MLK - August 28th 1963]

<comment><USER15> I'm taking a literal view of the word average. Anyone motivated to write a pamphlet is by definition no longer the average person, unless almost everyone else is doing so - i don't think we need 50% of the population writing pamphlets to maintain a free society.

As to your understanding of my second point: Exactly. I am saying it's okay for only experts to be informing the public, as long as there is no barrier to anyone becoming an expert. The only way to ensure there is no barrier to anyone becoming an expert is to have no special protected status. The only way to have no special protected status is for the protected status to be extended to everyone.

In summary: Everyone should have the protection of the freedom of the press, therefore, anyone can become an expert with all the attendant protections (without having to be special), but it's okay if not everyone is an expert, as long as anyone COULD be an expert.

</comment></comment></comment></comment></comment><comment><USER17> "Bluntly, we did not have to do our reporting from London. Already most of the NSA stories were being reported and edited out of New York. And had it occurred to him that Greenwald lived in Brazil?"

So there is the answer for how the UK/US division was picked for the NSA stories.

This is an absolutely incredible article. Incredible because I both am shocked (yes, that's the right word this time) about the contents and incredible because I'd rather not have to believe it.

Apparently the government of the UK now equates journalism with terrorism and will use laws instigated to squash terrorism to squash the free press or at a minimum attempt to intimidate it and to disrupt communications between members of the press. On top of all that they have spooks that seem incapable of understanding the first thing about digital communications visit a newspaper to oversee the physical destruction of a computer holding data of an investigation in progress.

Incredible... what's next?

I just read about the harassment Laura Poitras has to go through each time she travels. And all that for just doing your job? It is becoming really hard to distinguish the good guys from the bad guys.

<comment><USER18> [https://www.youtube.com/watch?v=aLJUocaDYw0](https://www.youtube.com/watch?v=aLJUocaDYw0) It's getting easier to distinguish.

<comment><USER17> "they didn't get to design our uniforms"

</comment></comment></comment><comment><USER19> The Guardian shifted from a .co.uk to a .com _very_ recently (~10-14 days ago), now showing content with a /uk subdomain. I wonder how much of that decision was based around the pressure they've been put under by the UK government. I imagine it's been a considered change for a while, but if you're starting to get concerned about journalistic freedom in the UK moving your reporting outside of it gives you a little wiggle room. Not sure how well it'll play long term, so we'll see.

It's sickening seeing how far the UK government is going to persecute journalists for exposing the surveillance state that's been created over the last decade. Pulling journalists partners in for questioning as a terrorist, forcing destruction of uncomfortable evidence? All in the name of protecting us from another round of terrorism, when really there's been significantly fewer terrorist incidents in the last 10 years than in the 10 before.

Edit: a quick traceroute shows that The Guardian seems to have it's servers for the .com outside the UK, and guardian.co.uk is/was located in London.

</comment><comment><USER20> If you want a picture of the future, imagine a boot stamping on an encrypted, backed-up, redundantly distributed hard drive — forever.

</comment><comment><USER21> With the NSA debate, I keep wondering if there is something people in positions in the government and security agencies know that makes them keep doing what they're doing.

I mean some of these people are really smart and understand how this spying can be abused. There must be something they know that makes them build or support these programs.

<comment><USER22> I had a similar thought, in 2003, when the US invaded Iraq. At the time, I was a bit of a UN nerd, and I had been following the weapons inspector reports and the Security Council debates. I was angry at how belligerent the Bush administration was getting, given that the publicly available evidence did not corroborate their claims that we might wake up to a "mushroom cloud". Iraq was maybe dragging its feet, but the weapons inspectors all agreed that inspections were working and should continue.

When the the US invaded Iraq, I started to doubt my own firmly held beliefs. Did they possess some classified evidence that I did not? Going to war without the authorization of the Security Council was such a grave breach of precedent and international law that I didn't believe the Bush administration would take such a drastic action if they didn't fully expect to find WMDs.

It turns out they didn't secretly have reliable information the rest of us didn't. They were blinded by conviction and ideology.

I'm sure these people in government and at the intelligence agencies truly believe these programs are vital to protect the US from terrorism. They know spying can be abused, but in their minds any abuse is unintentional and minor relative to the purported benefits of spying on terrorists.

<comment><USER23> > _It turns out they didn 't secretly have reliable information_

And more to the point, they knew exactly what their information was worth. A document known as the "Downing Street Memo" is pretty blunt on that: "intelligence and facts were being fixed around the policy." (From: [http://en.wikipedia.org/wiki/Downing_Street_memo](http://en.wikipedia.org/wiki/Downing_Street_memo) , [http://downingstreetmemo.com/memos.html](http://downingstreetmemo.com/memos.html))

We're not talking about any random document either. It was, in effect, the executive summary of the pre-Iraq war situation, given to the prime minister of UK by his private secretary.

</comment></comment></comment><comment><USER24> There is a bizarre jump in this story. How did they go from phone calls and meetings to incinerating MacBooks?

(Obviously, as a journalist, you need to start recording these conversations and publishing names. There can be no privacy for government employees.)

<comment><USER25> The modern equivalent of burning books?

<comment><USER26> Fahrenheit 4566: The temperature at which your MacBook Pro's aluminum casing boils.

</comment></comment></comment><comment><USER27> In the comments Rudsbridger, in all sincerity, is asking why a newspaper should have fought a government request to destroy source material.

I mean this is the strongest position the Guardian could have challenged the government from: No risk to the story, public visibility, government has nothing to directly gain from having the material destroyed. What is being fought over is purely principle and a demonstration of obedience: Gov asks and Guardian rolls over.

I presume we can expect the Guardian to routinely delete source material upon gov request if it hasn't been doing so already?

<comment><USER25> Or is it just complying because of the stupidity of the request? Because nothing was actually lost and nothing was deleted as such. A copy got wrecked, so what?

<comment><USER27> Both parties knew there were copies elsewhere. The act was a demonstration of compliance which has reinforced a relationship of subservience to the government.

</comment></comment></comment><comment><USER28> What infuriates me is that not enough people care for them to demand change.

It goes on the news one day then the next it's gone. Information-overload. That's their card and they'll keep playing it.

<comment><USER29> _What infuriates me is that not enough people care for them to demand change._

This is a common theme on Hacker News, and an understandable one: You see a situation that seems unjust, and you want other people to be upset about it. But they're not, because people don't work that way.

People will become upset when they have social proof that they ought to be upset. This takes a long time, which means that politics, metaphorically speaking, has massive momentum and takes a long to change direction.

At this stage, watch the decision makers, the influencers, and the people who pride themselves on being "middle of the road" above all else. When they start to flip, it means a larger political shift has started. And sure enough, obsessively middle-of-the-road writers like Andrew Sullivan are starting to doubt the government's arguments.

If the government keeps raiding newspapers, detaining spouses, and otherwise getting caught, public opinion will continue to shift. But it will do so slowly.

<comment><USER23> > _This takes a long time, which means that politics, metaphorically speaking, > has massive momentum and takes a long to change direction._

I've said for a very long time that Newton's First Law (conservation of momentum, or inertia) applies for all masses. Including masses of people.

Hell, I've even joked that it may actually apply _better_ to masses of people.

The silly thing is, I had arrived at that conclusion before I had ever read Asimov's Foundation.

</comment></comment></comment><comment><USER30> The British Public are the (ideological) descendents of people who had the "Magna Carta Libertatum" as a cornerstone of their civil society. Now from everything that comes out it looks like Britain is closest to becoming the Orwellian nightmare - constant surveillance of citizenry, arbitrary detentions, suppression of the press and an uncaring bureaucracy.

British folk ..."Remember remember the fifth of November". Speak out against this at every forum possible and kick out these enemies of freedom ASAP.

<comment><USER31> The developmental history of what might be called loosely the British Establishment is indeed interesting.

If you're interested in a previous example of the relationship between MI5, the Home Office and the 'Press', you could look at the publication of the biography 'Spycatcher', by ex MI5 officer Peter Wright.

[http://en.m.wikipedia.org/wiki/Spycatcher](http://en.m.wikipedia.org/wiki/Spycatcher)

</comment></comment><comment><USER12> I'm sure the main point of grabbing Miranda (ironic name) was to get at whatever files he was carrying, but...

Any thoughts that someone might use this as a way to "expose" Greenwald as being gay? Sorry, but there are a tremendous number of people for whom that will change their mind about the whole thing

</comment><comment><USER32> What's up with HN and these shitty titles lately? This says literally nothing. It isn't rhetorical or helpful at all. Tell me about the article.

<comment><USER33> The original title was: "You've had your debate. There's no need to write anymore." (theguardian.com)

The new title is: David Miranda, schedule 7 and the danger that all reporters now face (theguardian.com)

I prefer the first one.

<comment><USER34> Once again the posters title was overruled despite the fact that it was on top of page 1, the posters title was a quote taken directly from the article, and the post was generating good discussion.

<comment><USER32> None of those things matter at all. The second title is objectively more descriptive and that's what titles should be.

<comment><USER35> > The second title is objectively more descriptive

Is it? Probably most people who haven't yet read the story don't know who David Mirdanda is, since the story they haven't read yet is the thing they would most likely know him for. "Schedule 7" is meaningless without context. "The danger that all reporters now face" is meaningless without context -- they could just as well be talking about libel laws or piracy or some news about the Apple antitrust case.

The new title is objectively _less_ descriptive because it gives the reader no clue as to what the article is actually about before reading it. The original title tells you _exactly_ what the article is about. "You've had your debate. There's no need to write anymore." That's what the article is about.

<comment><USER32> No, second person "you" is less telling than third person "David Miranda." "Your debate" is not descriptive in the least. I chose the word "objectively" deliberately.

<comment><USER36> The first title doesn't correctly translate what the article is about. The quote might well be taken from a ranty essay, and without context hints at empty sensationalism.

This doesn't make the second title 'objectively' more descriptive, though. A person who happens to miss Miranda's name in news and isn't familiar with Schedule 7 term, such as myself, wouldn't find it descriptive at all.

Still, second title looks subjectively better to me. I'd say 'all reporters' is bad phrasing for HN (the article doesn't imply that _every_ reporter, regardless of their country or what they write about, now faces danger), and HDD destruction may deserve mentioning (it seems central to the story), but overall this headline does better job at translating the spirit of the article and sets correct expectations for the reader.

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> Our governments have lost a great deal of good faith as a result of the manner in which they've handled this. They're applying tactics from the 1950's in the modern age, and it's just blowing up in their faces and they're not coming out looking so well. I just wish there was more introspection and less of whatever you want to call this madness.

</comment><comment><USER38> A good measure of how free a country is: are its own journalists safe putting their feet there.

</comment><comment><USER39> Many of these comments are touchingly naive. Governments have always maintained absolute control over what they will permit to be said within thir own borders about matters affecting state security, whether this is about home or abroad.

Nothing has changed, just because you have a web browser hasn't given you any more 'freedom'. In the UK we still have D-notices (now DA-notices) and the media still has to abide by their guidelines or discover the consequences.

It is sad though that the "land of the free" needs an English newspaper to tell them what is going on. Your state control of the media must be even stronger than ours.

<comment><USER40> Compliance with D-notices are voluntary. They are notices that indicate concern and puts the press on notice that some of what they report _may_ lead to lawsuits if they go ahead. They're intimidation of sorts, but they don't in themselves stop publication or make publication illegal.

You'll notice that a lot of US media has to a decent extent chosen to ignore the latest DA-Notice relating to PRISM.

</comment></comment><comment><USER41> I don't see what everyone is so shocked by. Sensitive government information is in the wild. The government is using laws in place to try to limit the exposure of said sensitive data. Personally I don't think that journalists are the ones who should have final say over what information should and should not be exposed.

Regardless of which side of the argument you fall on, I don't see what is so revolting about this whole development. Journalist is thought to have very sensitive information about leading world powers. Leading world powers want information on who has the data, where it's been released, and want to limit further exposure. You want to be a journalist involved in releasing government top secret information? This is what you have to be ready to deal with. As a journalist, you're not above the law either.

To be clear, I'm not saying the government is always right either, I'm saying that the government is just doing what it's job is: keep top secret information from reaching sensitive hands.

<comment><USER42> And therein lies the real problem. Very few (possibly none) people think "I will subvert and subjugate the entire populace in evil oppressive ways for my own power". Rather they think "I truly believe my job is important and this is what's best".

The SWAT teams that destroy a peaceful organic farm in Texas are just doing their job to collect evidence according to a warrant. There may be a little petty "fucking hippies attitude" in there, but they aren't thinking "lets scare everyone from ever trying to have a nice big garden again". The police commander/cheif/whatever isn't thinking "lets make sure to cause as much damage as possible", they are thinking "lets do this safely and decisively, criminals need to be shown there is no point in crime". The prosecutor and judge aren't thinking "hey oppressing is fun!" they are thinking "we need to live under the rule of law, lets make sure that happens", and that is just their job. Hell even the people who vote for laws and representatives are thinking "this really is the best way to keep crime down and keep people from trashing the nice places we're trying to make here.

The intelligence apparatus is the same. Sure they are just doing their job to get material labeled as sensitive back under control. They really think it's dangerous for it to be out there. The people who label it such are truly under the impression that it is dangerous to the country they protect for it to be widely known. The people who build the gathering apparatus truly believe it won't be abused.

The abusers themselves aren't looking at it and saying "oh subjugate subjugate subjugate!" They are saying "I know my motives are pure - a little abuse of means is less important than the noble end.". The bad ones are not saying "fuck the people, they are an impediment". They are saying "I can get my own position bettered by ignoring one little rule". They say " My primary goal is personal gain, this twisty interpretation gives me that, and besides - it's legal in that light, so why is it bad?"

It's a death by a thousand cuts - the slow erosion of OK action by a large number of distributed players playing for their own gain. The agency of a group (a conspiracy) is not needed - and in fact would be better - it's easier to target and stop. The people who aren't abusers can't imagine the little abuses that would arise - they say "but it won't be used that way, it's absurd!"

Think of it like engineering a system: the security people fight their own engineering teams as much as they fight the "bad guys". They hear regularly "why would anyone think to do that?" "how could someone know that to abuse it?" "I can't imagine anyone going to such trouble for my little bit" and on and on. It's difficult to make people who aren't used to thinking and seeing it realize that such things happen all the time. Human systems and rules aren't any different.

This is what we need to focus on - closing the holes, fighting the little abuses, and generally making things overall better, rather than looking for the big conspiracies that may not even exist (because they don't actually need to).

</comment></comment><comment><USER43> Wow....I can't quite understand the actions of governments these days.

Did they really think that they could literally destroy the hard drives of the journalists working on these NSA scandals and that story wouldn't become a major story?

As in....really now. Any amount of legitimacy that the Snowden leaks didn't have before, just got significantly bolstered with this type of action.

How is this government still in power?

<comment><USER44> Because when they failed to be not elected (the vast majority of those who voted voted for some other guy, but _somehow_ these guys got to be in charge - first past the post, joy), they thought "stability" was the most important thing, and passed a law that said that there wouldn't be another election for at least 5 years.

</comment></comment><comment><USER45> >That work is immensely complicated by the certainty that it would be highly unadvisable for Greenwald (or any other journalist) to regard any electronic means of communication as safe.

Can anyone comment on how accurate that statement is? I would imagine that two people (Greenwald and Poitras) who frequently meet face-to-face could set up a secure way of transferring data.

<comment><USER46> I wouldn't imagine that. All we really know about the NSA's cryptographic powers are that they have a shit-ton of money and a lot of very smart people. And even if your cryptography is actually as secure as you think, the power that can be brought to bear here is immense.

Could they, from a distance, read the EM emissions of your computer? Could they compromise some physical object on which you set your computer and read the EM emissions from there? Can you afford to have somebody watching your computer every second of the day? Is there some possible attack that you haven't thought of but they, having devoted lifetimes of thought to the problem, have?

Of course, they could also be bumbling idiots, so there may be no need to worry. Certainly, this sort of ham-fisted pressure suggests that there are, at least, idiots involved. But if you have gotten to the sort of prominence where somebody might be willing to burn a billion dollars to cause you trouble, it's really hard to say what's safe and what isn't.

<comment><USER44> No need: Using a Mac? Compromised, the NSA has root access via the Patriot act. Ditto windows. And that's before you get to the idea of remote updates to the microcode supported by modern PCs (and not even running Linux keeps you safe from that, unless you recompile your kernel to turn it off).

Lets also not forget that even the strongest crypto is vulnerable to cartel attacks and the rubber hose technique.

<comment><USER47> Use a one-time pad, run Linux, don't connect to the net a machine with access to the keys or the plaintext. Encrypt your message, copy the file to a formatted flash drive, and send it to your communication partner however you like. Hell, stick it on pastebin, it won't matter.

I'm confused at the perspective being espoused here - "encryption is pointless because the NSA can break anything, and failing that they can always beat you up". Encryption is not pointless, they probably can't even break standard open source crypto, they definitely can't break one-time pads which are trivially easy to make nowadays, and they can always beat you up anyway so you might as well encrypt.

There's no plausible way any of that is less safe than running the gauntlet of border security with the data in your pocket.

<comment><USER46> I think you've exaggerated the perspective being espoused, so your reply is equally exaggerated.

Encryption only solves the problem of a suspicious link with perfectly trustworthy endpoints and a perfectly trustworthy side channel (for keys). Nobody is saying, "don't encrypt." But people are saying, "don't assume encryption is perfectly safe."

Also, the plausible way that physical carriage is safer is that you know when interception happens. If you encrypt everything and then use your husband's pocket as a transport medium rather than a wire, that's better data security because interception is harder, and can't be done in secret.

</comment></comment></comment></comment></comment><comment><USER48> I think it's quite important to share the names of the men from Whitehall to the world. I mean we all _know_ names of only the people defending public interest so far: aka Greenwald, Snowden, Miranda and a few others.

We should be told about everyone from the other side too.

Like we learned about Keith Alexander of NSA, you know. It is of utmost importance to make names of public servants well - _truly public_ \- because they're the ones using tax-payers money and should have been defending public interest instead.

<comment><USER49> Agree. But, FWIW, Alan Rusbridger, the author of the article, explains in the comments that in this case, this would not be legal, as the conversations he is quoting from took place after he had agreed to keep names off the record.

<comment><USER50> When the bastards make the rules at whim, there is nothing to be gained from playing by those same rules.

<comment><USER51> That rule is actually made by journalists, not governments.

</comment></comment></comment></comment><comment><USER52> Wow, a Guardian article that had something positive to say about the U.S. My jaw has also hit the floor!

FWIW, I also agree that shredding hard disks is far overstepping the bounds of what is proper.</comment>


The RE: 4chan email that is going around AT&T HQ - <AUTHOR>

When the AT&T 4chan episode hit the internet I had posted on HN that I would tell my contact there what is going on. My contact sent me this today. It is the email going around at AT&T HQ explaining what the 4chan episode was all about. I thought it was interesting to see the nature of articles that large corporations choose to send around.
---begin---
AT&T: 4Chan Blocked Over DDoS, Not Content Jordan Golson | Monday, July 27, 2009 | 10:22 AM PT | 3 comments
AT&T has finally issued a statement about the company's unannounced blocking of certain parts of forum site 4Chan. A number of parties blamed AT&T for "censoring" the Internet, presuming that the company had blocked access because of the content on 4Chan, which can range from classless to tasteless to everything in between. In fact, AT&T said it blocked one section of the site to control a distributed-denial-of-service attack that was affecting an unnamed AT&T customer. 4Chan is, for lack of a better term, the "Wild West" of the Internet. It is an unedited, unmoderated message board that is surprisingly influential on the Net. 4Chan users have been credited with starting lolcats and RickRolling. Last year, a 4Chan user allegedly hacked into Sarah Palin's Yahoo Mail account and posted the password on 4Chan. The site, especially its /b/ page, is an interesting place — very, very "not safe for work."
According to a statement from the phone giant: Beginning Friday, an AT&T customer was impacted by a denial-of-service attack stemming from IP addresses connected to img.4chan.org. To prevent this attack from disrupting service for the impacted AT&T customer, and to prevent the attack from spreading to impact our other customers, AT&T temporarily blocked access to the IP addresses in question for our customers.  This action was in no way related to the content at img.4chan.org; our focus was on protecting our customers from malicious traffic. Overnight Sunday, after we determined the denial-of-service threat no longer existed, AT&T removed the block on the IP addresses in question. We will continue to monitor for denial-of-service activity and any malicious traffic to protect our customers.
The story got legs on Reddit, where many commentators scorched AT&T, saying the company "censors the Internet," and 4Chan fans formed bulletin boards to organize "retaliation" against AT&T. TechCrunch wrote that blocking a web site without notification is an "extreme breach of user trust." Of course, the blocking evidently wasn't over 4Chan's content, which AT&T assures me it would never do, but instead over network protocols that were affecting other customers.
Net neutrality is all well and good, but when one particular server is negatively impacting other users, AT&T is obligated to respond. Not everyone jumped on the blame AT&T bandwagon, however. Late last night, Shon Elliott from unWired Broadband wrote on the North American Network Operators Group mailing list: There have been a lot of customers on our network who were complaining about ACK scan reports coming from 184.108.40.206. We had no choice but to block that single IP until the attacks let up. It was a decision I made with the gentleman that owns the colo facility currently hosts 4chan [sic]. There was no other way around it. I'm sure AT&T is probably blocking it for the same reason. 4chan has been under attack for over 3 weeks, the attacks filling up an entire GigE. If you want to blame anyone, blame the script kiddies who pull this kind of stunt.
"The above newsclip is distributed to educate and inform.  This does not imply accuracy nor endorsement of the views by AT&T."
---end---
My response:
AT&T did a good thing (stopping a DDoS attack), but they need to be more on top of their "twitter relations," people claimed to have called in to AT&T and received the response that the site was blocked due to content. Now that is may normally the case, so the support person on the phone was probably just hazarding a guess, but once the story caught on fire AT&T's twitter account should have tweeted that taking down 4chan had nothing to do with content. Or at LEAST that they could say they were investigating.
I know it was the weekend, but the story went pretty far, and AT&T is already getting blamed for Google Voice being blocked on the iPhone, so they don't need anymore bad press. If a corporation is taking down a website over the weekend they need to be ready to explain to the public why, otherwise they are just asking for disaster. <comment><USER2> A slight inaccuracy, 4chan is moderated, even the "unmoderated" parts. Linky: <http://www.4chan.org/rules>

<comment><USER3> As far as I know, the 4chan /b/ (random) board is only moderated to the extend they're minimally required by law such us banning underage users (as it's an adult board) and removing child porn.

<comment><USER4> You are almost correct but it's not entirely without rules, 4chans /b/ people keep saying "bring back snacks" where snacks are referring to the ban on:

* "Post ending in x"

* Gay porn

* People seeking attention but not "putting out"

Refence but keep in mind the common sense thing is to not click encyclopedia dramatica links if are at work or want to avoid knowing useless things: <http://encyclopediadramatica.com/Bring_Back_Snacks>

For the record I don't actually visit 4chan, I just happen to know this out of sheer morbid fascination.

</comment></comment></comment><comment><USER5> So they sent out a press release to their own employees instead of bothering to write an email?

<comment><USER6> If you work at any big company you'd know it's often standard practice to send out press releases and market statements internally to inform staff of what's happening.

<comment><USER7> standard practice seems to be to first toss the PR to employees so that they're informed beforehand, and to check to make sure no mistakes were included. then, to the public.

</comment></comment></comment><comment><USER8> There should always be 2 sides to any story. Whenever something happens between a large player and a smaller guy, the masses start to make accusations.

Yes large corporations need to be faster at making public statements, but they also need to act cautiously.

<comment><USER9> I wouldn't say 4chan is the "smaller guy".</comment></comment>


Ask HN: Is Xamarin worth learning? - <AUTHOR> <comment><USER2> Sure. Especially if you were doing cross platform work. E.g. monogame, mobile development etc.

The move to the cross platform .net core would take preference if I was learning .net now however.

</comment><comment><USER3> No.

<comment><AUTHOR> Why not?</comment></comment>


A Tweet Led to a Shortage at Popeyes - <AUTHOR> https://www.nytimes.com/2019/08/29/business/popeyes-chicken-sandwich-shortage.html <comment><USER2> I was really caught off guard when Popeyes had a line like chick-filet/in-n- out disrupting the entire business plaza in the spring valley area of Las Vegas.</comment>


Assistly Comes Out Of Stealth: New support platform - <AUTHOR> http://www.assistly.com/

<comment><AUTHOR> It seems like this market is well covered by SalesForce already (service cloud 2). Assistly doesn't seam to offer anything different...

<comment><USER2> It looks more like an alternative to Get Satisfaction and the like to me. Looks reasonably good from the screenshots. I'll give it a shot if there's a free version/trial version released.</comment></comment>


Antarctica may have set its highest temperature ever recorded Tuesday - <AUTHOR> http://www.washingtonpost.com/blogs/capital-weather-gang/wp/2015/03/27/antarctica-may-have-set-highest-temperature-ever-recorded-tuesday/?postshare=121427729109034

<comment><USER2> "Need another indicator of climate warming in Antarctica?"

It's interesting how high temperatures equates to climate change, but low temperatures doesn't mean anything. Even mentioning it will be followed by a torrent of people commenting on how temperature != climate.

More proof that 'climate change' is a political tool.

<comment><USER3> I saw an article today that mentioned:

"Officially, for the past 11,700 years we have been living in the Holocene epoch. From the Greek for 'totally new', the Holocene is an eyeblink in geological time. In its nearly 12,000 years, plate tectonics has driven the continents a little more than half a mile: a reasonably fit person could cover the scale of planetary change in a brisk eight-minute walk. It has been a warm time, when temperature has mattered as much as tectonics. Sea levels rose 115 feet from ice melt, and northern landscapes rose almost 600 feet, as they shrugged off the weight of their glaciers." [1]

I'm curious how current climate trends fit in with what seems to be a time period that overall has been warming up accompanied by rising seas.

Can someone point me to some resources to help me make sense of human-caused global warming versus normal global warming?

[1] [http://aeon.co/magazine/science/should-we-be-suspicious- of-t...](http://aeon.co/magazine/science/should-we-be-suspicious-of-the- anthropocene/)

<comment><USER4> Maybe

[http://skepticalscience.com/climate-change-little-ice-age- me...](http://skepticalscience.com/climate-change-little-ice-age-medieval- warm-period.htm)

OR

[http://skepticalscience.com/solar-activity-sunspots- global-w...](http://skepticalscience.com/solar-activity-sunspots-global- warming.htm)

Lots of good info on the site. There is a very, very, vanishingly small chance that current climate dynamics aren't caused by humans. Doubling the CO2 in the atmosphere will have consequences. It's inescapable physics.</comment></comment></comment>


Create website with Google - <AUTHOR> https://sites.google.com/new <comment><USER2> I can't help but feel that this is a reactionary move by google against Facebook Work.

Although they aren't similar products, they both are internal networks and Google sites can and is used as an internal site/communication tool at some smaller companies.

On a side note, I don't understand the deal with designers @ Google. They take the ugly(?) material design to the extreme.

I am not a fan of the Google play music app or the web interface.

sites.google carries on the tradition.

Compare it to the deisgns of say the MS Office 365 products, or with Facebook both of which I find pleasure in using.

/End personal design taste rant</comment>


Esper's dark vision for US-China conflict makes war more likely - <AUTHOR> https://www.defensenews.com/opinion/commentary/2020/03/19/espers-dark-vision-for-us-china-conflict-makes-war-more-likely/ <comment><USER2> I estimated a few years back that with China's continuing growth larger than the US's, the crossover date when China would be the strongest nation would be about 2025.

To prevent that, the US in its Thucydides Trap (alternatively, its Wolfowitz Doctrine) would need to initiate a conflict with China about 5 years earlier (so, 2020) otherwise it would have no guarantee of winning that conflict, and would probably be destroyed completely.

Wars in the Northern Hemisphere need to kick off in Spring (Mar 21) at best, or Summer (June 21) at the latest. If the US doesn't start a war with China in the next 3 months, it will have missed the boat.

</comment><comment><USER3> Quite disturbing. We are facing a new Cold War, and a new nuclear threat to humanity once again, actually it never went away!

</comment><comment><USER4> Who will be the first to attempt to invade a virus-riddled country I wonder . . .

<comment><USER5> Apparently the US (as suspected), but the first target could very well be Iran. Disgusting enough that Trump doesn't even think about easing the sanctions so Iran can get medical equipment, and now his generals think about war ([https://www.nytimes.com/2020/03/21/world/middleeast/trump- ir...](https://www.nytimes.com/2020/03/21/world/middleeast/trump-iran-iraq- coronavirus-militas.html)).</comment></comment>


Artist-Made Tools Resist Algorithmic Racism and Empower Communities - <AUTHOR> https://www.kqed.org/arts/13881761/artist-made-tools-resist-algorithmic-racism-and-empower-communities <comment><USER2> It looks like the only way progressives can confront surveillance in 2020 is by attacking it whenever it becomes racist: "algorithmic racism". One would assume that they wouldn't object to the same technology which oppressed in a racially equitable manner.

<comment><AUTHOR> is that even possible? see weapons of math destruction</comment></comment>


Chip Aging Accelerates - <AUTHOR> https://semiengineering.com/chip-aging-accelerates/ <comment><USER2> Chips aging aside, tin whiskers also are one of the main reasons why manufacturers shouldn't use electronics except where it is really really really necessary, especially on things that move at high speeds with humans on board or in vicinity. NASA itself -which use electronics on stuff that gets in orbit- did some good research on the subject. [https://nepp.nasa.gov/Whisker/background/index.htm](https://nepp.nasa.gov/Whisker/background/index.htm)

Tl;dr: Tin whiskers are very thin spontaneous metallic formations which can short nearby pcb tracks or conductors and are believed to be the cause of many failures in electronic devices. There is no evidence of a single cause for their formation but it seems certain that eliminating lead (RoHS legislations etc.) from solder is one of them.

<comment><USER3> Can't they coat the PCBs to mitigate/elminiate the problem?

I remember watching a teardown of a spare-no-expense embedded military computer, and the guy couldn't stop talking about how much "conformal coating" the board had.

Edit: I think this is the video: [https://www.youtube.com/watch?v=55z_0BYb5is](https://www.youtube.com/watch?v=55z_0BYb5is)

<comment><USER4> Typically, the coating is Parylene, which is a room temp vapor phase conformal coating (e.g. roughly uniform thickness independent of the contact angle of the materials). It's used for corrosion, moisture, vibrational. wear, and reduced breakdown voltage, etc.

It is not used for Sn (tin) whiskers at all, since it wouldn't penetrate the pads/package of the IC and wouldn't stop such a high modulus process in any case. Whiskers are most problematic with modern Pb-free solders and fine pitch SMT practices (QFNs & BGAs), but it's worth noting that other materials (like Zn) also have significant issues.

[http://www.paryleneengineering.com/why_use_parylene.htm](http://www.paryleneengineering.com/why_use_parylene.htm)

<comment><USER5> From that page:

> The coating completely penetrates spaces as narrow 0.01mm.

I would have thought that would make it very effective in protecting against whiskers in QFPs that might have a lead pitch of 0.4mm or larger. There's not a lot you can do under the bodies of BGAs and QFNs, but if you're worried about whiskers you're probably using QFPs instead.

This app note suggests that Parylene and a few other dip-type conformal coatings do slow down tin whiskers, but don't stop their spread:

[https://www.maximintegrated.com/en/app- notes/index.mvp/id/52...](https://www.maximintegrated.com/en/app- notes/index.mvp/id/5250)

</comment></comment></comment></comment><comment><USER6> Computing in automotive is a challenge.

In the past, you might have automotive-grade microcontrollers for functions like ABS, which consume less than a Watt maximum. You paid a little attention to having enough ground vias on the PCB for thermal conductivity, and that was about it to qualify the design for the high end of the temperature range.

Degraded lifetime wasn't so much of a concern.

These days, you've got ARM processors with a TDP of 15W or more, and keeping the die temperatures below the maximum when the ambient temperature is 85C, well, that starts to get interesting. Especially if you don't want to use a fan, and the processor is stuck somewhere without adequate airflow.

And then you've got high-end systems with a TDP in the 150W range. Then you've got to have a good cooling solution to run your application at office environment temperatures, nevermind the full automotive temperature range. And what's going to be the lifetime for these parts running at elevated temperatures, even if you are staying within the maximum temperature limits? _Sigh_

<comment><USER7> 15W chips use ~2% of a HP each, and 150W = 1/5 horsepower worth of parasitic load. So using several high end chips should result in significantly worse fuel efficiency.

Are they really necessary or just cheaper in R&D terms?

<comment><USER8> Modern cars have tons of advanced features that require a decent amount of computing. I mean just look at the evolution from radio cassette players to modern integrated GPS/media player/bluetooth/etc systems. Also things like automatic parking assist, rear view camera, computer vision algorithms to detect if the driver is falling asleep. It's feature creep all over the place.

<comment><USER9> > Also things like automatic parking assist, rear view camera, computer vision > algorithms to detect if the driver is falling asleep. It's feature creep all > over the place.

You have a weird definition of "feature creep". Some of those things you listed save money, others save lives. I definitely wouldn't include such things in "feature creep".

<comment><USER8> Sorry, it might not have been the right word. I didn't mean that these features weren't useful, only that in the past decades the past decades the amount of software in cars really exploded from basically 0 to having a bunch of fully featured computers dealing with various subsystems, from fuel injection to radio playback. Software "creeped" everywhere.

</comment></comment></comment></comment></comment><comment><USER10> > Chips developed for computers and phones lasted an average of two to four > years of normal use.

Err what?

<comment><USER11> This is a real thing. For game consoles, which run their chips pretty "hot" in order to meet the performance requirements for high-end gaming, the main SOCs have a design lifetime. There are deliberate trade-offs between performance and the expected lifetime of the part. This lifetime is difficult to estimate prior to ship.

One console made a firmware update about a year after its release that increased its clocks by about ten percent. I'm guessing they did this after seeing telemetry numbers indicating that the box's cooling system was doing a better job than expected, and that there was lifetime headroom available (probably other factors were involved, too, but heat is a major one).

<comment><USER12> I'd be charitable and say that gaming chips run hot because cooling systems are "optimized", not for performance reasons.

And by optimized, I mean shoehorning the smallest / cheapest solution in there that barely meets cooling needs.

<comment><USER11> Consoles are pretty serious about effective cooling. There is definitely price pressure, but that's not a one-sided thing -- you need to make sure that the unit lasts long enough, and that its cooling system doesn't generate too much noise for the environment (typically a quiet living room). And then you need headroom for envrionmentals (e.g., being stuff into the back of an entertainment unit next to other equipment). The current generation of consoles are quiet compared to the last generation, and more powerful.

Cooling "needs" have to take into account the whole product and user experience, not just keeping a single chip from melting down.

<comment><USER13> I think all the console makers learned a lesson from the Red Ring Of Death. No one wants to be having a massive portion of their systems become useless hunks of scrap because of poor thermal design not dissipating enough heat and causing the solder to crack.

<comment><USER11> RROD cost Microsoft a billion dollars, maybe two billion. That doesn't scream "Please shave the cooling system down to the absolute minimum cost on the next console" to the hardware team.

The XBone cooler is a pretty decent one, because the alternative is a ton of warranty work, plus lawsuits and bad press. And worse, a bad customer experience.

</comment></comment></comment></comment></comment></comment><comment><USER14> Two to four year lifespan?

I happen to know that the design life for the Ford EEC IV, the ignition control system for 1980s Fords, was 30 years. That was achieved; many 30-year old Fords are still running with the original electronics.

<comment><USER15> article says "Chips developed for computers and phones were designed to operate at peak performance for an average of two to four years of normal use. After that, functionality began to degrade"

</comment></comment><comment><USER16> Enterprise ASICs (i.e. routers) already run at these very hot (and cold) temperatures on the very latest nodes. They also demand reliability over long periods - although the consequences of failing to meet that are obviously not as severe. Ageing is much worse at 7nm but it's already accounted for during STA. Like everything else, you just assume the worst and it's another chunk of your clock period you never see. But expecting to run above 125C... not sure about that.</comment>


Ask HN: Where do you stand on privacy? - <AUTHOR> During my first quarter-century of life I've already lost money to a professional con artist while starting my first business (@ 20), quit a relatively high profile/responsibility position because of corrupt leadership, resulting in high tensions due to reputation risk for a political family (@ 22), and severed ties with a close business partner due to manipulation and dishonesty (@ 24). Last month a friend called one of my roommates to come over at 3 AM because a guy (casual acquaintance of all of us) was threatening to break down her door and rape her.
And I grew up in one of the smallest most conservative towns around, in one of the most sheltered and safe families I've heard of -- all of this has happened only since I left high school and had the freedom to explore the rest of the world. (Granted, I am not a normal human being. My understanding of risk allows me to safely undertake many activities that some would find terrifying, and I have a strong drive to have new experiences which ensures close contact with a great diversity of people, cultures, and situations).
Just yesterday I began removing all my info from Facebook, and was promptly greeted with 3 messages from my friends asking me who my new love in life is -- Facebook had courteously notified each of my loosely-acquainted contacts that I was "no longer listed as single". Last fall, I discovered that my cousin was divorced when his wife took him off her friend's list (I'd been out of the country at the time, and he wanted to tell me face to face). Surely my experiences here are not unique.
Neither of these online experiences caused me any physical or financial harm, but are they not examples of how these services can have unintended consequences, which could be dangerous when involving the wrong people? Think "Black Swan" here. It only takes one really rotten egg to spoil the rest of your life.
So I'm re-thinking my stand on privacy and have yet to come to any conclusions, but am being conservative in the interim. I used to be entirely "pro-exposure" -- I deliberately marketed myself aggressively online because I figured that that was the best way to "win the game" -- but these days I find myself taking as much personal info off the net as I can. Do you think I'm making a mistake? What sort of "happy medium" do you think might be appropriate? Where do you draw your line on privacy?
Like many people on this forum, there's a good chance that my professional ambitions will necessitate attempts to draw public attention to myself and my work in the future, so this really isn't an issue that I can ignore. I think it's important, and I want to be smart about it. Any ideas, experiences, perspectives would be much appreciated.
Cheers. <comment><USER2> Your experiences sound fairly similar to my own. When I incorporated, I was a freshman in high school, and the lawyer I used to file the papers hired me to build a web site for him. He never paid the bill in full. (I paid his, of course.) I've also had my share of negative experiences with business partners who respectively stole money from customers, stole common shares in my company from me, and wanted to steal information from customers. Each time, I thought I'd learned my lesson, to no avail.

In general, from my perspective, if you find yourself disagreeing with (meaning "being more conservative than") most people in business about what constitutes an ethical solution to a problem, you're probably right. Most businesspeople have no morals. It took a while to get used to that.

I have a biased view on Facebook and privacy, but a legitimate one nonetheless, I believe. You can find it here:

[http://www.news.com/Will-security-become-Facebooks- Achilles-...](http://www.news.com/Will-security-become-Facebooks-Achilles- heel/2010-1029_3-6231585.html?tag=nefd.top)

<comment><USER3> "Most businesspeople have no morals."

Not my experience. You need to meet new businesspeople.

I would say, "Some business people have no morals. They ruin it for the majority that do."

<comment><USER4> I believe I am gifted with a good judge of character. I have only been screwed over by people a few times, and was always expecting it from that person.

I have met 3 successful people in business that are what I would consider honest and straight shooting. I have kept in contact with all 3.

I have met hundreds of unsuccessful people who fall into the 'have morals' category.

The majority of people who I have met who have been very successful are snakes. I have met a few new billionaires, and the tactics that they used even against their employees and friends were very shady. One of them bought out a partner for a very low amount because he knew they were about to sell at a very high valuation shortly.

What I have discovered is most people play the game to win, and rarely play with integrity. I mean this all in a very non-judgmental way (even the snakes part!), people should live their lives by their own moral compass.

<comment><USER3> "What I have discovered"

What I have OBSERVED

Like I said, my experience is different. I have conducted business over the years with many very honorable (and usually rich) people who often were able to conduct business with an understanding or a handshake. Maybe things have changed, but there are still a lot of people who spend a lifetime building an spotless reputation and protect it dearly. I sure hope you get a chance to do business with more of them in your future.

<comment><USER4> No, not what I have observed. What I have discovered. I have met a large enough sample set to make these statements.

I am judging their morals and how honorable they are against a scale that I perceive to be true. Honor and morality is highly subjective.

Even within the most honest folks I have met, very few reward the people they stood on the backs of to make it to the top. Woz gave some of the original Apple developers over a million dollars of his personal stock in the company as they were about to go public because he felt they were under-compensated - how many people would do this?

</comment></comment></comment></comment></comment><comment><USER5> I'm not sure how privacy works online, or who sees what. Having said that, as a hacker I probably understand it better than most regular people. When I decide to put anything ( _anything_ ) online, I keep two things in mind:

1) Anything I put up can be seen by anyone (not just the people I would prefer see it)

2) Anything I put up will be around forever and I can't unring that bell.

With those points in mind, here are some of the things I _haven't_ put online:

-the company I work for

-my home address (I think I've mentioned the city, I have mentioned the metro)

-any pictures of my daughter (maybe she has slipped in the background of some)

-religious affiliation and (for the most part) political views

One the other hand, I've gone out of my publicize (here and on reddit) things I've written on technical and business topics. I want that stuff to be found, but I try to make sure it's something I wouldn't mind someone pulling out 10 or 20 years from now. My goal is that the worst thing that could happen to me based on what I publish online is that I'll be accused of being dumb or naive. I don't want my friendships or family to be affected by some web fad.

</comment><comment><USER3> I value my privacy so much that I have no account on any social network and never will. A very few number of people have my cell phone #. Anyone can send me an email anytime they want.

You say, "I deliberately marketed myself aggressively online because I figured that that was the best way to win the game".

What game? I don't see the logic in that at all.

People pay money to solve their own problems. Provide them with a solution. That's the only "game" I've ever been interested in "winning". The rest is smoke and mirrors.

If you have a popular blog or had written a book and positioned yourself as an expert, that would certainly help your marketing effort. Of you could just as easily spend that time advertising, networking, and pounding the pavement the old fashioned way to generate business. At least that way, you preserve your privacy.

Losing your privacy is like scattering feathers in a storm. Gone, never to be put back into the pillow case again. Ask him:

<http://slog.thestranger.com/2007/11/a_lesson_for_us_all>

<comment><USER6> Or this fellow: <http://www.bash.org/?203247>

</comment></comment><comment><USER7> I've purposely made a lot of my real life information easy to get by anyone who actually knows me. If you knew the right url, you could have my address and phone in a second, which is exactly what I want. However, I generally avoid leaving anything around online that text technology alone could use to connect my comments back to me and I change my online identifiers every year or so. I'm sure someone who got access to all my personal DSL records for a few years could connect the dots but that doesn't worry me much.

Hmm.. this account is about 257 days old now. Thanks for reminding me to abandon it in about 3 months.

</comment><comment><USER8> I don't put anything online unless I'm comfortable with the idea of millions of strangers, and my mom and dad, looking at it.

And I'm a pretty shy person, so not much of my life makes it onto the web.

</comment><comment><USER9> Privacy is a competitive advantage if implemented well.</comment>


Max Schrems files first cases under GDPR against Facebook and Google - <AUTHOR> https://www.irishtimes.com/business/technology/max-schrems-files-first-cases-under-gdpr-against-facebook-and-google-1.3508177 <comment><USER2> Heavily discussed previously:

[https://news.ycombinator.com/item?id=17152496](https://news.ycombinator.com/item?id=17152496)</comment>


Clojure + Processing + Live-coding at the REPL - <AUTHOR> http://automata.cc/live-processing

<comment><USER2> Can you use the OpenGL bindings from Processing in your application ? This would be an interesting application for doing some OpenGL graphics on JVM.

I didn't used Clojure, how difficult will be for a Scheme/Common Lisp guy to start hacking in Clojure ?

<comment><USER3> If you want straight-up OpenGL bindings for clojure, there's Penumbra:

<https://github.com/ztellman/penumbra>

</comment></comment><comment><USER4> Ruby based Processing + Live-coding <https://github.com/jashkenas/ruby- processing/> a really cool project.

</comment><comment><USER5> Hey, I was doing this with clj-processing in summer '09... except my "eval button" was C-M-x :)</comment>


Ask YC: Gmail Down For You Too? - <AUTHOR>

Gmail is down for me.  What about you? <comment><USER2> <http://downforeveryoneorjustme.com/gmail.com>

</comment><comment><USER3> Works for me. For these issues check twitter :)

<comment><AUTHOR> Good call.</comment></comment>