Intel SA-00145: Lazy FP State Restore - <AUTHOR> https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00145.html <comment><USER2> Andy Lutomirski noted on another thread that he unintentionally fixed this two years ago in Linux:

[https://news.ycombinator.com/item?id=17304947](https://news.ycombinator.com/item?id=17304947)

(He switched it to eager FP because it's faster on modern hardware.)

But a lot of people running old LTS kernels may be affected.

EDIT: Looks like Luto's change landed in kernel version 4.6. [https://kernelnewbies.org/Linux_4.6#List_of_merges](https://kernelnewbies.org/Linux_4.6#List_of_merges) [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ecc026bff6e8444c6b50dcde192e7acdaf42bf82)

<comment><USER3> Does that include Red Hat Enterprise Linux? I can't believe how old of a kernel they use still. I have to use it for work, but I run Arch at home which is running 4.16.13. I'm honestly surprised that Red Hat can't manage to keep their distros current with stable package builds.

<comment><USER4> Red Hat backports security fixes, hardware support and certain new features to their kernel packages, staying on a stable kernel version throughout the product lifecycle is what allows them to guarantee a consistent kernel ABI that virtually every other distribution throws to the wayside. Personally I prefer not having third-party kernel modules on my servers break every time I run `yum upgrade` like I do with Fedora (oh VMWare, how I hate you so).

<comment><USER5> I'm using HP B120i fakeraid controller with proprietary driver and it broke after 7.4 upgrade, so while they probably doing a good job about binary compatibility (7.5 didn't broke it), it's not ideal.

<comment><USER4> Unfortunately the kABI does not encompass every symbol exported by the kernel, there is a whitelist maintained in the kernel-abi-whitelists package and scripts to check conformance of a module to the whitelist. Symbols are only ever added throughout the lifecycle of a RHEL release, so anything that conforms to the 7.3 kABI will also work on 7.4 - but if the module uses a symbol NOT whitelisted in 7.3 there's no guarantee the 7.4 update won't break it.

Not that it's much solace when your proprietary blob breaks, but Red Hat does make considerable effort to give vendors a stable target to build against - unfortunately not everyone fully validates conformance which results in the same old problems cropping up from time to time.

<comment> silly-silly That and Spectre fixes forced a BUNCH of third party code that relied on non exported code to break.

</comment></comment></comment></comment></comment></comment><comment><USER6> I posted some details about this to Twitter: [https://twitter.com/cperciva/status/1007010583244230656](https://twitter.com/cperciva/status/1007010583244230656)

<comment><USER7> Here is the text of the tweets for convenience:

So about that "Lazy FPU" vulnerability (CVE-2018-3665)... this probably ought to be a blog post, but the embargo just ended and I think it's important to get some details out quickly. This affects recent Intel CPUs. It might affect non-Intel CPUs but I have no evidence of that. It is an information leak caused by speculative execution, affecting operating systems which use "lazy FPU context switching". The impact of this bug is disclosure of the contents of FPU/MMX/SSE/AVX registers. This is very bad because AES encryption keys almost always end up in SSE registers. You need to be able to execute code on the same CPU as the target process in order to steal cryptographic keys this way. You also need to perform a specific sequence of operations before the CPU pipeline completes, so there's a narrow window for execution. I'm not going to say that it's impossible that this could be executed via a web browser or a similarly "quasi-remote" attack, but it's much harder than Meltdown was. I was not part of the coordinated disclosure process for this vulnerability. I became aware of this issue after attending a session organized by Theo de Raadt at @BSDCan. It took me about 5 hours to write a working exploit based on the details he announced. Theo says that he was not under NDA and was not part of the coordinated disclosure process. I believe him. However, there were details which he knew and attributed to "rumours" which very clearly came from someone who was part of the embargo. My understanding is that the original disclosure date for this was some time in late July or early August. After I wrote an exploit for this, I contacted the embargoed people to say "look, if I can do this in five hours, other people can too; you can't wait that long". While I have exploit code and it is being circulated among some of the relevant security teams, I'm not going to publish it yet; the purpose was to convince the relevant people that they couldn't afford to wait, and that purpose has been achieved. I know from the years that I spent as FreeBSD security officer that it takes some time to get patches out, and my goal is to make the world more secure, not less. But after everybody has had time to push their patches out I'll release the exploit code to help future researchers. I think that's everything I need to say about this vulnerability right now. Happy to answer questions, but I'm not part of the FreeBSD security team and don't have any inside knowledge here -- FreeBSD takes embargoes seriously and they didn't share anything with me. </thread>

One more thing, some advisories are going out giving me credit for co- discovering this. I didn't; I just reproduced it and wrote exploit code after all the important details leaked.

</comment></comment><comment><USER8> Serious question - should we considering speculative cpu execution to be A Bad Idea (tm) and move on from it (since these problems keep coming up), or is the thought that we more or less have been gaining performance on the back on incorrectly written software (which does not take these speculative execution edge cases into account), and the only forward is patching?

I guess a another question I have is can we win the performance back through fixes on the cpu or will speculative execution always be insecure and thus need patching in software?

 __Try as I might, I am not a CPU person.

<comment> Tuna-Fish Completely dropping all forms of speculative execution means dropping overall performance to a tenth of today. There are really hard limits on how fast any operation, especially memory operations, can be done. The way we have made our CPUs faster is by making them do more operations in parallel, at all levels. At the lowest level, in straight line code, this very often requires speculation to achieve.

Speculation is not fundamentally incompatible with security. It's just that literally everyone in the industry never though that leaking information out of speculative context was possible -- and so there is no hardening anywhere. Now that it was proven possible, people are rushing to find all the ways this can be exploited. New CPUs currently being designed will fix all these, and then eventually we will have speculation without security issues.

Except for Spectre variant 1. That will always stay with us, because there is no sensible fix for it. The only real solution to that is to accept that branches cannot be used as a security boundary. This is mostly relevant to people implementing secure sandboxes and language runtimes. Going forward, the only reasonable assumption is that if you let a third party run their code in a process, no matter how you verify accesses or otherwise try to contain that code, you should assume it has a read access to the entire process. Any real security requires you to make use of the proper OS-provided isolation.

<comment><USER2> > Going forward, the only reasonable assumption is that if you let a third > party run their code in a process, no matter how you verify accesses or > otherwise try to contain that code, you should assume it has a read access > to the entire process.

If this is true, it is unbelievably bad for the future of security and computing in general. People throwing around this assertion are, in my opinion, not appreciating how bad it is. We need to try a lot harder before we give up.

Here's why:

1\. Finer-grained isolation makes security better, because it allows us to apply the Principle of Least Authority to each component of the system, and protect components of a system from bugs in other components. To make meaningful gains in security going forward, we need to encourage _more_ fine- grained isolation. If process-level isolation is the finest grain we'll ever have, we can't make these advances.

2\. The scalability of edge computing _requires_ finer-grained isolation than process isolation. The trend is towards pushing more and more code to run directly on devices or edge networks rather than centralized servers. That means that the places where code runs need to be able to handle orders of magnitude more tenants than before -- because everyone wants their code to run in every location. If we can't achieve high multi-tenancy securely -- by which I mean 10,000 or more independently isolated pieces of code running on the same machine -- then the only solution will be to limit these resources to big, trusted players. Small companies will but shut out from "the edge". That's bad.

Luckily, the "process isolation is the only isolation" claim is wrong. It's true that we need to evolve our approach to make sub-process isolation secure, but it's not impossible at all. In fact, it's possible to design a sandbox where you can trivially prove that code running inside it cannot observe side channels.

Here's how that works: Thank about Haskell, or another purely-functional language. An attacker provides you with a pure function to execute. Because it's a pure function, for some given set of inputs, it will always produce exactly the same output, no matter what machine you run it on, or what else is going on in the background. Therefore, the output _cannot possibly_ incorporate observations from side channels, no matter what the code does internally.

So: It is possible to run attacker-provided code without exposing secrets from the process's memory space.

The question is, of course, how do we build a useful sandbox that relies on this property. There is a lot of work to be done there. Luckily, we don't really have to use a purely-functional language, we only need to use a _deterministic_ language. It turns out that the world's most popular language, JavaScript, is actually highly deterministic, in large part due to its single- threaded nature.

We do need to remove access to timers, or find a way to make them not useful. We also need to prevent attackers from being able to time their code's execution remotely. Basically, we need to think carefully about the I/O surface while paying attention to timing. But sandboxing has always been about thinking carefully about the I/O surface, and we have a lot of control there. We just have a new aspect that we need to account for.

I think it's doable.

<comment><USER9> > Here's how that works: Thank about Haskell, or another purely-functional > language. An attacker provides you with a pure function to execute. Because > it's a pure function, for some given set of inputs, it will always produce > exactly the same output, no matter what machine you run it on, or what else > is going on in the background. Therefore, the output cannot possibly > incorporate observations from side channels, no matter what the code does > internally.

This is patently false. The attacker can do timing on his side and see how long it takes your service to return a response. If you let an attacker have any sort of output channel, you give him the power to use _his_ stuff to find side channels, and there is nothing you can do to close it.

<comment><USER10> If the attacker can do timing on their side in a pure function, then by definition the time the response takes is one of the function inputs.

<comment><USER11> Haskell functions are only pure in an abstract sense that ignores micro- architectural side-effects and the resulting timing changes. All of the recent speculative side-channel attacks are the result of the abstraction layer exposed by modern processors leaking. You can't fix this by putting more abstraction layers with neat theoretical properties on top.

<comment><USER12> You can't be guaranteed to prevent the attacker form writing information to those side channels but you can guarantee that the attacker can't read from those side channels because reading from them requires doing timing analysis and timing analysis requires IO - which we've stipulated hasn't been passed to the attacking function.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Status in Debian: [https://security- tracker.debian.org/tracker/CVE-2018-3665](https://security- tracker.debian.org/tracker/CVE-2018-3665)

<comment><USER14> Just what I was looking for. Guess we'll get a new security announce.

Thanks

<comment><USER13> ... for debian 8 only, though. Looks like current stable dodged that bullet quite a while ago.

</comment></comment></comment><comment><USER15> What is the impact of this exactly? If "FP State" just means floating point register values then those rarely contain interesting stuff.

If this also affects register used for crypto acceleration then it could be used for stuff like leaking browser secrets from javascript, right?

<comment><USER6> "FP State" in this case includes MMX and SSE registers; they're handled via the same "lazy context switching" mechanism. Yes, you can steal keys which are being used for AESNI.

Leaking secrets via javascript -- I'll be impressed if someone pulls that off. There's a very tight timing window to exploit this before a trap fires and flushes your pipeline, and I doubt you can get the right instructions in there using javascript.

<comment><USER16> Can't you executive the trapping access in a not-taken speculatively-executed branch à la Meltdown?

<comment><USER6> Sure, but that gives you an even shorter window because the pipeline flushes as soon as the CPU realizes that it mis-speculated.

<comment><USER16> It is my understanding that you can make that window very long by having the mis-speculated branch depend on a value that has to be loaded from main memory.

<comment><USER6> Hmm, good point. Yes, that would probably make this much easier to exploit.

Unless the CPU does a partial pipeline flush upon realizing that there would be a trap. I don't know -- do Intel CPUs check for exceptions in order?

<comment><USER16> Well, they don't for #PF at least, because that's how Meltdown works - I'm not sure if it works differently for #NM though.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Do we have PoC code? Has anyone tried attacking FP/SIMD state on other ISAs like Power or AArch64?

<comment><USER6> I have exploit code -- took me about 5 hours to write after Theo announced all the important details of the vulnerability. I'm not going to publish it yet, though.

AFAIK other systems aren't affected -- is lazy context switching even a thing on them? The fundamental issue here is that one process' data is still in registers when another process is running and we've been relying on getting a trap to tell us when we need to restore the correct FP state.

<comment><USER18> Lazy context switching is a thing on pretty much every architecture with an FPU.

<comment><USER6> Hmm, I thought most RISCy CPUs kept FP values in GP registers?

<comment><USER18> No, I can't think of an arch that does that. Power, SH, Mips, Sparc, Alpha, ARM, and RISC-V all have separate architectural register files for the floating point state.

Some ARM ABIs end up passing floats in integer registers, but that's just for compatibility for code that doesn't assume the presence of an FPU and might be doing everything soft float.

<comment><USER6> Hmm, ok. It's a long time since I've looked at that. Come to think of it, I think it might be 20 years since I opened CA:QA...

<comment><USER17> Yeah I'd say that modern OoO Arm implementations (A57, A72, ...) are worth trying to speculate into trapped VFP state. Lazy FPU is definitly a thing everywhere.

My hunch says that chips affected by 4a could easily be fair game (4a is speculating reads into priviledged regs... I wonder if 4a would work on regs that are trapped, not inconceivable)

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Is it fixable by microcode update?

<comment><USER6> Very unlikely. The key issue here is that traps aren't handled until a long way down the pipeline.

But it's a (relatively) simple OS fix.

</comment></comment><comment><USER20> We'll let this thread take the frontpage spot now that the announcement is official. Previous one: [https://news.ycombinator.com/item?id=17304233](https://news.ycombinator.com/item?id=17304233).

</comment><comment><USER17> What are we calling this Spectre variant?

<comment><USER6> Either "LazyFP" or "Lazy FPU".

<comment><USER17> Not numerating it? Variant 6 is as good as anything...</comment></comment></comment>


Windows Phone's many problems: Should Microsoft give up? - <AUTHOR> http://www.zdnet.com/blog/btl/windows-phones-many-problems-should-microsoft-give-up/73126?tag=mantle_skin;content

<comment><USER2> I read that Nokia president Eltop, a former Microsoft executive, said "there is no plan B".

Nokia has no fall back plan, so if they continue losing money at current rates, by year end 2012, they will not be able to borrow any more from banks. Microsoft may throw even more good money after bad to keep Nokia alive - or to acquire their patent portfolio.</comment>


Steve Jurvetson pushed out of DFJ as lines between personal and prof crossed - <AUTHOR> https://www.recode.net/2017/11/18/16647078/steve-jurvetson-dfj-sexual-harassment-silicon-valley-tech-allegations-fired <comment><USER2> This situation is different than the recent spate of assault accusations to appear in the news. While he might be guilty of poor judgement, everything described in the article is consensual, and that's his personal life anyway. Only the part about women being banned from TED would be wrong, but that's just an allegation at this point and he wasn't specifically named.

His departure must be the result of some internal powergrab within the firm. Which partners benefit as a result of his departure? Will he start his own VC firm now?

<comment><USER3> How come did president Clinton continue but every man now is brought down?

<comment><USER4> Yeah, it's not like he lost his job or anything...

[https://en.m.wikipedia.org/wiki/Impeachment_of_Bill_Clinton](https://en.m.wikipedia.org/wiki/Impeachment_of_Bill_Clinton)</comment></comment></comment>


IRS Rules Bitcoin is Property, Not Currency - <AUTHOR> http://blogs.wsj.com/totalreturn/2014/03/25/qa-the-new-irs-rules-on-bitcoin

<comment><USER2> Avoid paywall:

The Internal Revenue Service said Tuesday that it will treat bitcoin and other virtual currencies like property, not currency, giving a potential boost to investors but imposing extensive record-keeping rules—and significant taxes—on its use.

The announcement in general was expected to be greeted favorably by the fledgling industry, and many had anticipated exactly this result. But the announcement also served as a reminder that new technologies often can't avoid being subject to the old rules for long.

In a notice, the IRS said that it generally would treat Bitcoin held by investors much like stock or other intangible property. If the virtual currency is held for investment, any gains would be treated as capital gains, meaning they could be subject to lower tax rates.</comment>


Twitter is blocked in Turkey - <AUTHOR> <comment><USER2> Many sites including Youtube did get blocked in the past, but safe to say this one is different. It is an explicit attempt to silence non-traditional media by the government. They have been working on controlling the traditional media (TV, newspapers, etc.) for a long time.

Recently, social media has been making a bigger impact, hence the explicit play. Current administration sees itself in an existential struggle and willing to do anything to stay in power. The model that they aspire seems to have switched from the west to Russia/China.

Whether they will be able to succeed in silencing social media will have implications beyond Turkey.

<comment><USER3> I think you are absolutely right.

</comment></comment><comment><USER4> I was there on business in 2010 and I said "man, this is a country on the move! We need to set up an office here." English is taught from early grades, there is good infrastructure in the western part of the country, a young, educated population with a relatively small domestic economy - great place to outsource and a bridge between europe and asia. Things have fallen apart so quickly since then - wouldn't touch it now. Hope things get back on track, as there is so much potential.

<comment><USER5> it was an illusion. yes i am from istanbul.

<comment><USER6> I am from Istanbul too and I agree on this. Only 3-4 of the larger cities look like they're more modern.

<comment><USER7> Istanbul is modern? Sorry but Istanbul is a shithole when compared to some of the other Turkish western coast cities. What makes it interesting in the global scale is the level of its diversity.

An ex Istanbul resident.

<comment><USER6> I did say it was "more modern". I lived there for 22 years as a minority and have been treated like an animal in the zoo, and don't think I'll ever live in Turkey again.

</comment></comment></comment></comment></comment><comment> guard-of-terra Internet is falling apart. I think that appreciation for free speech, press and freedom of expression is lower today than it was for a long time of recent history. "They" just push and push on it and we're helpless. Not even constrained by paying it lip service.

Do we still have non-violent ways to counter this and make our voice heard? Because otherwise violent ways become aspired.

<comment><USER8> I'm getting depressed by these kind of news. My votes do not seem to change anything and I feel hopeless. I can't find anything meaningful to say and I just want to ignore the whole situation but keeping silent just feels wrong.

<comment><USER9> Don't despair completely.

Not Internet related, but there are some large students going on in Taiwan right now [0] based on concerns on how a new trade agreement with China was passed. The bill was passed in a legal way, but many of the discussions around it were behind closed doors and the Taiwanese people feel that the government is no longer "representing the people." The response against has been pretty described as "unprecedented", people who are normally not interested in politics are getting involved to see how they can help.

" _We do not want to clash with the police...we just have to let the government know that never try to fool the people._ "[1]

I find their vigilance inspiring. This level of spontaneous organization seems difficult to carry out in the US though. Taiwan is considerably smaller than the US, and I think that makes a large difference unfortunately.

[0]: [http://www.bloomberg.com/news/2014-03-19/taiwan-students- occ...](http://www.bloomberg.com/news/2014-03-19/taiwan-students-occupy- legislature-over-china-trade-pact-vote.html)

[1]: [http://www.cnn.com/2014/03/19/world/asia/taiwan-student- prot...](http://www.cnn.com/2014/03/19/world/asia/taiwan-student-protests/)

EDIT (to respond to guard-of-terra): > "Turns out they just sit in their office and pretend nothing happens."

Definitely a valid point. We shall see how long this sustains for.To clarify, when I say "getting involved" I mean many people are taking to the streets, not just armchair activism.

<comment> guard-of-terra Don't want to discourage you but we had this "unprecedented, people who are normally not interested in politics are getting involved" thing in Moscow a few years ago and it all died out. Turns out they just sit in their office and pretend nothing happens. And it works unless you actively overthrow them.

Of course they say Russians are a nation of slaves but there you go.

<comment><USER8> The same happened in Turkey last summer too. You might have heard, "the Gezi protests". Looking back, the whole thing just made everyone a bit more extreme on their views. Some government officials just ignored the situation while the prime minister went on a campaign to stop the so called "looters" (yes he did call the protesters that) from "disrespecting the public opinion". Now? It's a flame war between many groups, with some scandals here and there for what it seems to be entertainment only. Everyone, and I mean everyone -including me- seems to ignore everything opposed to what they believe. I try to be objective, but I can't. People believe in "stuff". I'm still getting amazed by how interested we are in believing in something and the lengths we can go just to keep our beliefs' integrity.

<comment> guard-of-terra "Looking back, the whole thing just made everyone a bit more extreme on their views."

Yeah, that's exactly what we had in Russia. The protests were about rigged elections. They also happened to be peaceful. Nevertheless, you can now find a lot of people who call protester names as if that was something provably bad. You can also find a lot of people who outright support everything the state do - thing you could not find before. A lot of people switched from "not caring" to "actively fearing change and swearing on any critique of status quo".

</comment></comment></comment></comment></comment></comment><comment><USER10> Background info:

This is hypothesized to be a reaction to a series of audio recordings that were anonymously released over time. These were ridiculously damning, clearly someone is tapping phones without anyone else's knowledge.

Previously:

[http://www.theguardian.com/world/2014/feb/27/turkish-pm- corr...](http://www.theguardian.com/world/2014/feb/27/turkish-pm-corruption- youtube-recep-tayyip-erdogan) [http://www.al- monitor.com/pulse/iw/contents/articles/origina...](http://www.al- monitor.com/pulse/iw/contents/articles/originals/2014/02/erdogan-corruption- audio-bilal-graft-probe-police-money.html) [http://en.wikipedia.org/wiki/2013_corruption_scandal_in_Turk...](http://en.wikipedia.org/wiki/2013_corruption_scandal_in_Turkey)

Some highlights from the "alleged" recordings so far:

\- PM calling to get news about an opposing party removed from a TV channel multiple times

\- Modifying political poll results to manipulate public opinion

\- Getting people of opposing opinions fired

\- Buying a large paper shredder to destroy documents

\- PM calling son to ask how much money there is at home, son replies saying about 1 trillion, then switches to 3-5 kurush (cents in turkish).

\- PM calling son to say "they're raiding the houses, zero out the money". Son says there's only 30 million euros left.

\- Call to order the judges to be fixed, and says that a specific person is to be imprisoned.

\- Trying to manipulate who goes on the supreme court.

\- Says "ignore the prosecutor who's running the corruption investigations". Orders documents ripped up.

(source: [http://www.hurriyet.com.tr/yazarlar/25981622.asp](http://www.hurriyet.com.tr/yazarlar/25981622.asp))

The official reason for the shutdown is that there were sexual photos of some poor citizen released on twitter against her will, and she complained but twitter refused to take them down. So they decided to shut down the entire website. Yeah, right.

Of course this is ominous that at the end of this month, the local elections will take place. No coincidence.

It's surreal how all this is going down. It's like watching a conspiracy theory movie. Blatant shills everywhere, media manipulation abounds, blatant lies heard on TV from the horse's mouth. We were all skeptical already, it's inevitable when you live in a third world country for years, but wow. When you hear the insiders actually talking to each other, it's a whole new level.

more news on this specific event: [http://www.hurriyetdailynews.com/turkey- blocks-twitter-after...](http://www.hurriyetdailynews.com/turkey-blocks- twitter-after-erdogan-vowed- eradication.aspx?pageID=238&nID=63884&NewsCatID=338) [http://www.aljazeera.com/news/europe/2014/03/turkey- erdogan-...](http://www.aljazeera.com/news/europe/2014/03/turkey-erdogan- threatens-ban-twitter-2014320165956732467.html)

Related old post here: [https://news.ycombinator.com/item?id=7189577](https://news.ycombinator.com/item?id=7189577)

edit: Erdogan stated a few days ago that he didn't care what the world thought, and he would eradicate twitter. He said "how dare they listen to our encrypted phone conversations". It's expected that on the 25th, something huge is going to be released that might sway the elections bigtime. Whoever is posting these are doing it slowly and deliberately, a few days at a time, building up anticipation. So people think that twitter getting shut down is probably related to this.

edit2: All recordings were put on youtube by someone: [https://www.youtube.com/user/haramzadeler333](https://www.youtube.com/user/haramzadeler333)

<comment><USER11> Do you happen to know where I can find translations for the youtube videos?

<comment><USER10> I think this is the best you can do right now:

[https://news.ycombinator.com/item?id=7297985](https://news.ycombinator.com/item?id=7297985)

<comment><USER11> Thank you very much.

</comment></comment></comment></comment><comment><USER12> Imagine if government had the power to flick a switch place perfect earplugs and blinders on every human everywhere for a time for some political agenda.

I wonder how far men would go (short of just killing people) to suppress the right for people to think, in order to suppress some information floating around in the public space.

Would you flick that switch, to power down an entire nation of drones, just to prevent them from finding out that you did something evil?

Take care you NEVER give other men the right to shut down your brain from afar. It WILL be used against you.

</comment><comment><USER13> Earlier today "Turkey's Erdogan Now Says He'll Shut Down Twitter, Too" [http://time.com/32339/turkey-erdogan-twitter/](http://time.com/32339/turkey- erdogan-twitter/)

<comment><USER14> Wow:

"I don't care what the international community says. Everyone will witness the power of the Turkish Republic."

Did he actually say that? In regards to shutting down twitter to stop a scandal? If you truly weren't responsible for w/e scandal, then wouldn't you just not care?

Edit: Also that line seems perfect for a James Bond villain lol.

<comment><USER15> He is really one of those James Bond villains.

The sentence describes his state of mind: He is the best thing for the country, so any resistance, corruption news, and protest cannot come from people of this country. It must be a conspiracy of the west. So, by blocking twitter, he is blocking west's conspiracy. He started believing his own propaganda.

The truth is people who vote for him is mostly poorer Turks, and those people don't care about twitter.

<comment><USER16> It seems there are a couple of countries with the same problem in their leadership. How the F do these people get into power?

<comment><USER17> Many of them actually get power-hungry while in power.

Especially the ones who get elected after spending some time in the opposition under not-so-democratic regimes.

</comment></comment></comment></comment></comment><comment><USER18> I just tried pinging twitter with this tool, the turkey server seems to work:

[http://cloudmonitor.ca.com/en/ping.php?vtt=1395356295&vargho...](http://cloudmonitor.ca.com/en/ping.php?vtt=1395356295&varghost=twitter.com&vhost=_&vaction=ping&ping=start)

but china, egypt, and interestingly panama don't. I'd love to report on this but it's hard to prove this kind of thing. any ideas? (i'm also contacting twitter)

edit: here's my story -

[http://www.nbcnews.com/tech/social-media/twitter-blocked- nat...](http://www.nbcnews.com/tech/social-media/twitter-blocked-nationwide- turkey-reports-n58276)

please let me know if you see anything wrong. (we're aware of the turkish character issues...sigh)

<comment><USER19> Hi, The method for blocking websites in Turkey is through ISP's DNS servers. So when you don't use a Turkish ISP DNS server you are all good with most of the blocked sites. Some websites, especially popular porn sites are blocked through the IP. In such cases you need a VPN or proxy.

The tool you are using to check Twitter servers probably don't use a Turkish DNS server.

If you don't believe, check the website status from the Informational Technology and Communication presidency of Turkey website: [http://eekg.tib.gov.tr/](http://eekg.tib.gov.tr/)

The court orders are displayed both in English and Turkish.

<comment><USER18> Thanks for the information, I'll try to include this in my report.

<comment><USER19> You are welcome. If you have any questions feel free to ask, you can reach me by twitter too: @mrtksn

this is how the DNS Look-up for twitter looks like at the moment: [http://i.imgur.com/yKmmDYc.jpg](http://i.imgur.com/yKmmDYc.jpg)

I used TTNET's DNS server for this. It's the biggest ISP in the country.

</comment></comment></comment></comment><comment><USER20> DNS based block at the moment, IP based one will follow for sure. Please remember that YouTube was also banned for a couple of years in the past [1].

This is related to the corruption scandal in Turkey, Tayyip Erdogan's government is trying to stop the release of leaked tapes on social media.

[1] [http://en.wikipedia.org/wiki/Censorship_of_YouTube#.C2.A0Tur...](http://en.wikipedia.org/wiki/Censorship_of_YouTube#.C2.A0Turkey)

<comment><USER21> That's old. IP and URL based censorship methods (with TCP packet inspection) is in place already in Turkey. We have seen various individual URLs getting blocked related to the the bribery and corruption leaks of PM and other ministers in past few months. So don't be so hopeful. DNS is not a permanent solution, they can block it entirely just tomorrow.

</comment></comment><comment><USER22> Unfortunately Erdogan is out of control, they use DNS based blocking. I'm using a VPN service so I can access Twitter, Youtube and other services easily but in our country probably 0.01% has VPN so this is serious.

</comment><comment><USER23> Ah, blocking twitter, the last refuge of despots[1].

[http://techcrunch.com/2011/01/25/twitter-blocked- egypt/](http://techcrunch.com/2011/01/25/twitter-blocked-egypt/)

</comment><comment><USER24> For the time being, Twitter is accessable from within Turkey using the DNS of the biggest ISPs of Turkey, 18.104.22.168

<comment><USER19> nope, unfortunately it's just your DNS cache.

</comment></comment><comment><USER25> Twitter has been playing a pivotal role in connecting people and getting the word out via an unbiased medium. The effect of twitter has been phenomenal over the past few years in countries that have undergone political revolution. Much of the reality that the citizens need to know are blacked out by mainstream media because it's controlled by the government & corporations who are hand in glove.

Unfortunately, many countries have resorted to a simple fix of policing twitter. Second unfortunate thing is that twitter is still restricted to the tech savvy crowd. There's still a sizable population outside third world that don't know what it is & how to use it. But as long as we have un-policed social media tools, we might see large scale cleaning in the coming decades.

</comment><comment><USER26> The real question is why Erdogan took risk an blocked Twitter just 10 days before elections? The only reason I see is this Twitter account with almost 500.000 followers who is revealing government corruption: [https://twitter.com/FuatAvni](https://twitter.com/FuatAvni) For English version of his tweets see: [https://twitter.com/FuatAvniEng](https://twitter.com/FuatAvniEng) (usually translation is delayed 1-2 weeks)

</comment><comment><USER27> Reminds me of this scene from Pirates of Silicon Valley:

[http://www.youtube.com/watch?v=b27Mf6wTZ3w](http://www.youtube.com/watch?v=b27Mf6wTZ3w)

"It's about power. It's like those weird countries man where the army guys overthrow the president. The first thing that they take over is the way people communicate; radio, tv, newspapers. Information is power."

Turkish army has nothing to do with this case of course.

<comment> guard-of-terra In this case I'm not so sure.

They just say: we can take things from you, and you can't do anything about that. They'll just say that you had them "elected" \- as if someone voting for them gave them rights to take what is not theirs.

They're just trying to teach the young generation helplessness. You can't do anything, better learn your VPN, feel fear and don't touch politics. You're a minority anyway compared to an army of aged citizens so no point in trying. They're vote for any extortion for a small share of profits.

</comment></comment><comment><USER28> Crucial services that can be choked off need to be made seamlessly (zero thinking UX) available via overlay networks (tor, etc).

IOW, if twitter is blocked, there should be a browser plugin / standard thing that fires itself up and automatically reconnects through said overlay network.

Because right now, a single domain is a SPoF that govts can flick a switch and neuter most of their populace's communication.

</comment><comment><USER29> Irony: Twitter ban being spread to the world via Twitter itself. A perfect example of how powerful the internet is despite of the governments.

</comment><comment><USER30> I'm from Istanbul, and I love how this block makes my pro-govt friends say "Why did they do this?". Twitter is (was?) used extensively in Turkey, and I feel like this block will end up with the government shooting itself in the foot by showing their supporters how sad the things they do are. There's no explanation for this block, and there can't ever be.

</comment><comment><USER31> Just when you think Turkey is a 1st world country...

<comment><USER32> Yeah, like the UK, right?

</comment></comment><comment><USER3> This is really a shocking news for all the twitter users from Turkey. They should raise voice against it. by [http://www.etunescafe.com](http://www.etunescafe.com)

</comment><comment><USER33> Sitting in Ataturk airport right now and twitter is working fine.

<comment><USER34> sorry then, we are mistaken.

</comment></comment><comment><USER35> Something has to be done before this sets an example for other governments. Freedom cannot be taken away from anyone.

<comment><USER29> Considering the fact that I'm seeing #TwitterisblockedinTurkey as a worldwide trend despite of the ban I would say they are far from being able to take away the freedom at all. They will also probably realize that not far from now.

<comment><USER36> The rest of the world tweeting about it doesn't mean that much to Turkish people as they won't be seeing the trend lists, and their government doesn't seem to care that the rest of the world is paying attention.

<comment><USER19> Actually, censorship is nothing new for Turkey so some serious portion of the population knows how to overcome it. Probably these tweets are sent mostly by Turks in Turkey :)

However, this is not the whole story. The current ruling party has huge support from the uneducated and/or religious masses that may not be that informed about the technology. Since 3 months new corruption documents are leaked through twitter, which puts pressure on the PM. Just yesterday there was a phone conversation of one of the Ministers leaked and apparently the guy was mocking Islam when he was trying to look like as devoted muslim in front of the public.

</comment></comment></comment></comment><comment><USER37> This is so ridiculous. As if someone can't just set up a domain themselves and release the shit.

<comment><USER38> How do you promote the domain? That's the real question. Twitter makes it quick, effortless and free to disseminate really interesting information. It's slower, more cumbersome, possibly more expensive with other methods, even with something like email.

<comment><USER39> I think this is why we need an open source twitter / equivalent. Just to enable people to communicate even in local networks.

</comment></comment></comment><comment><USER40> Lots of things happening around the Black Sea these days. Sochi Olympics, Crimea and now Turkey.

</comment><comment><USER28> Does anyone know _how_ it's currently blocked? twitter IP's, BGP or DNS?

<comment><USER41> Implemented via DNS, for now... You can see the court orders about sites from TIB's (The Presidency of Telecommunication) query site. [http://eekg.tib.gov.tr/](http://eekg.tib.gov.tr/)

It's a system where every ISP with a license has to abide. They download a list of domains that needs to be forwarded to a server operated by TIB to explain the reason of access blocking. They add these to their recursive DNS servers so traffic goes there. People using another DNS server other than their ISP's (e.g. Google Public DNS, OpenDNS, etc.) are currently able to access.

<comment><USER28> Thanks. Just confirmed through channels also.

Highly recommended that people, regardless of location, use encrypted DNS like OpenDNS to prevent monitoring, malicious in-flight modifications and DNS-based censorship.

</comment></comment></comment><comment><USER42> turkey made a lot of progress recently. too bad it will all go down the drain like this.

</comment><comment><USER43> ^ Requesting namecoin based app

</comment><comment><USER44> sad to see sites being blocked. Can you use Tor to get around this restriction?

<comment><USER38> Technically, yes. Usability-wise, no. Most Turkish internet users are mobile- based, anyway.

<comment><USER45> See [https://guardianproject.info/apps/orbot/](https://guardianproject.info/apps/orbot/) for some easy to use Tor apps for Android.

</comment></comment></comment><comment><USER46> Bullshit. I was tweeting from Istanbul just 3 days ago.

<comment><USER47> This is an interesting comment. I think comments similar to this should be looked at and at least the message evaluated and at most the user profile examined.

I may start browsing comments which have been downvoted the most to create a list on political-heavy threads. However it may be difficult to determine whether the user is a shill or just stupid.

</comment></comment><comment><USER48> Many people would see this as a good thing.

It's just a big site run by some American company. Amazingly, there are other ways to communicate.

<comment><USER7> Turks doesn't have fanaticism towards domestic good. It's a central communication channel that is known to run by people who seems to be fair, them being American is obviously not relevant to them.

<comment><USER48> I agree. My post was rather tongue-in-cheek actually.

Call me cynical, but it just sometimes strikes me as odd how so many people around the world now rely on a company designed to make money by selling your information to communicate with the rest of the world. Especially in today's post-Snowden-revelations world. It's 'free', but that's how you 'pay'.

It turns out it's a simple DNS block which setting to Google's DNS is fixing for people.

</comment></comment></comment><comment><USER49> Then use Twister [1]. Twister is NSA-proof, censorship reistant, and based on the bitcoin protocol.

[1] [http://twister.net.co/](http://twister.net.co/)

<comment><USER38> It is nontrivial to switch millions of people over to a completely new platform and reestablish the literally billions of connections among them, nor is Twister's infrastructure ready to handle the influx.

For now, Twitter over VPN/TOR is the only workable solution.

</comment></comment><comment><USER50> There are two sides of the coin.

1) In this day and age shutting down Twitter wholesale is simply a silly and undemocratic move. Information can not be blocked and will find its way around anyway. Like many people I am against it.

2) There is a blatant violation of personal rights of people by fabricated phone-tapes. You take it to the court and court decides that these are illegal, and ask the service provider (in this case Twitter) to take down those specific twits. They don't comply. What do you do?

Say there's a link for a fabricated phone conversation of Obama betraying the country disseminated using Twitter. The administration take it to the court and win the legal battle. Could Twitter afford not to comply? Can this happen?

This is the gist of the problem people are having over there.

<comment><USER50> So far this comment got downvoted twice. I bet those who downvote are Turkish. They are not explaining why did they downvote. They are as good as those that blocked Twitter in Turkey.

<comment><USER19> I've down voted you because you claim that the leaked phone conversations are fabricated.

few things about it:

1) some of the people involved accepted the authenticity of these recordings(some directly, some logically)

2) the recordings that were denied being authentic were investigated by audio forensic experts and they reported that these are real.

3) the pro-government media tried to scam US based audio studios to get reports to falsify these recordings. for an example, the mainstream pro-akp media used a report by Kaleidoscope Sound which was later denied by the studio. You can read their public statement here: [https://www.facebook.com/KaleidoscopeSound](https://www.facebook.com/KaleidoscopeSound)

4) Other evidence brought to light by the opposition party supports the authenticity of these recordings.

5) These recordings are part of the legal prosecution.

Though there were some recordings that were made illegally, the fact that the ruling party actively blocked the legal prosecution lives no choice for the public to learn about the accusations from the whistleblower.

If the legal process was not blocked it would have been better to wait for the outcome of the prosecution, but in this case we don't have choice.

<comment><USER50> 1) All of the recordings? 2) Again, for all the recordings? 3) I checked it out. They are saying that they do not know if the recordings are authentic. Since accused can not provide the original recordings actually there's no point even seeking for such forensic examination. It has to be inconclusive, that is why illegally obtained wiretappings are rendered always invalid by the courts. In that case spreading recordings that can not stand such scrutiny is an attempt of defamation, and illegal. 4) I can not check that. 5) What prosecution? Does it have a name/date?

Again I am against the blockage. However we need try to understand what could have pushed the Turkish Courts to take such grave measures.</comment></comment></comment></comment>


Apple co-founder Steve Wozniak quits Facebook over data abuse scandal - <AUTHOR> https://www.independent.co.uk/life-style/gadgets-and-tech/news/apple-steve-wozniak-delete-facebook-quit-data-breach-social-media-a8296071.html <comment><USER2> Sigh... Why is this even news? I really don't care about individuals leaving FB especially when they have no connection to FB other than being a user. It's not like Woz was some power-FB user AFAIK.

<comment><USER3> On the one hand, it's only news because it allows the media to continue the narrative they want, which is that Facebook is in trouble.

On the other hand, any publicity about well known (if only by most people through association) leaving not only reinforces the narrative but likely bolsters the reality as it effects people's opinion, so it's important in it's own right because of that.

Not as important as some Kardashian publicly quitting the platform, but important for the same reasons. Public individuals have some small (or large) level of influence over public opinion.

</comment></comment><comment><USER4> Wozniak used Facebook? That's a surprise to me. Is Richard Stallman on there too?

<comment><USER5> Well I thought he was working at FB and hence leaving FB was news.

<comment><USER6> Yeah, the use of the word "quits" implies employment. When I saw this article, I thought "Wait, I didn't even know he was working there".

</comment></comment></comment><comment><USER7> I thought Woz had gotten a job there. Maybe "deactivated" is a better word.

</comment><comment><USER8> I wonder if facebook will retaliate after this massive wave of negative press passes. I mean it's the media bad-mouthing another medium, so in a sense it's anti-competitive.

</comment><comment><USER9> Awesome :)

</comment><comment><USER10> As someone who has not used facebook in years, this drama is getting ridiculously annoying. I wish hacker news had filters like 4chan.</comment>


Solar Energy Prices: This Is What A Disruptive Technology Looks Like - <AUTHOR> https://medium.com/armchair-economics/cbc9fdd91209

<comment><USER2> If something can be said for Mr McConnell's methodology here, it's that he fully disclosed his assumptions, making it easy to identify the faulty one that sent his analysis hurtling 180 degrees in the wrong direction.

Why are you measuring the efficiency of energy sources in dollars, Mr McConnell? No, I don't mean to say Euros or Yen would have been better -- rather, that the reason you find yourself writing about the efficiency of energy sources in the first place is that their efficiency and availability has changed in recent decades and will change even more in decades to come, yes? Which will in turn affect how you and your readers think about your energy costs, correct? Ah.

But it will also affect how energy providers think about their energy costs, won't it? If one of those lines on your graph goes spiraling up or down, it will affect the other lines, won't it? And it won't affect them equally.

Solar panels can be as cheap as they want _while oil is cheap_. Will they still be cheap when oil is expensive? When the cost of transport for the many materials that go into their manufacture skyrockets?

There exists a better measure of energy efficiency than dollars per gigawatt. It's called gigawatts per gigawatt. Input-output ratio. By that measure, as everybody knows, oil kicks everything else's ass. Whereas solar power won't be even be net-positive until 2020 according to Popsci, which, adjusting for Popsci's track record on predictions, projects to around 2075-2100.

[http://www.popsci.com/science/article/2013-04/solar- panels-n...](http://www.popsci.com/science/article/2013-04/solar-panels-now- make-more-electricity-they-use)

Yes, you read that title right. Only as of this month did new panels stop having a net-negative energy efficiency. If that's your idea of disruptive, well, I advise you to quit reheating your Domino's with hydrogen fuel cells.

<comment><AUTHOR> Except you missed the point, which is that solar PV systems have been posting compounding gains in terms of power output per $ for 35 years. No other energy source I am aware of is becoming ~15% cheaper per year. I'll assume that most of the people here are familiar with compounding interest and its implications.

The point isn't that solar energy is disruptive today, it's that it will be in the not distant future. It's sort of like cellphone technology. It started out as a niche technology that hardly anybody used (even in San Francisco circa 2000, I know eons ago, people outside the tech industry didn't use them, now everyone has a smartphone).

I put my money where my mouth is on this, and retrofitted my pre-WWII home with solar some eight years ago. Even then, the amortized cost of the solar array was 50% less than what I paid to PGE for grid power, and I basically have a net zero electric bill.

Maintenance? A couple times a year I go up on the roof and hose the panels down to wash the dust and pollen off (no moving parts, etc). That and I make a point to double check that a meter reader hasn't flipped the safety cutoff switch. Other than that, I don't do a goddamn thing and my house generates about 75-80% of its power.

</comment></comment><comment><USER3> I don't understand why the oil price is so flat. Between the lower and higher point the difference is like one "mouse-cursor" in my computer. The graph is logarithmic, and the difference between the 200 and 300 marks is approximately one "mouse-cursor". So we have a 3/2 variation in the oil price.

But, for example, in this graph <http://www.wtrg.com/oil_graphs/oilprice1970.gif> (taken from: <http://www.wtrg.com/prices.htm> ) the minimum price is ~$15 and the peak prices are ~$75 or ~$95, so the variation is at least 5/1.

<comment><AUTHOR> I used annual averages from the EIA. This smooths out short term variations (I am looking at a 30-40 year trend). Also be sure whatever price history you're using is adjusted for inflation.

That said, even with short term price fluctuations, the decrease in solar energy costs (100:1) trumps the variation in conventional energy commodities (5:1). The point of the article is the long-term "moore's law" trend and how that will play out over the next 10-20 years.

</comment></comment><comment><USER4> Even more so: <http://www.genome.gov/sequencingcosts/>

</comment><comment><USER5> Is there a algorithm or standard method for doing "inflation adjusted terms"?

<comment><AUTHOR> The data I got from the EIA is inflation adjusted using the consumer price index.</comment></comment>


Linus Torvalds on semaphores (1999) - <AUTHOR> http://yarchive.net/comp/linux/semaphores.html <comment><USER2> Here's Dijkstra's original paper on P and V (in Dutch), from about 1963.

[http://www.cs.utexas.edu/users/EWD/transcriptions/EWD00xx/EW...](http://www.cs.utexas.edu/users/EWD/transcriptions/EWD00xx/EWD35.html)

Here is a implementation of P and V, the original counted semaphore primitives, from 1972.

[http://www.fourmilab.ch/documents/univac/fang/](http://www.fourmilab.ch/documents/univac/fang/)

This is UNIVAC 1108 assembly code. Along with P and V is the code for bounded buffers, with the operations "PUT" and "GET". Bounded buffers are what Go calls "channels". Note how simple they are if you have P and V. That code even works on multiprocessors. There's one semaphore for "queue full" and one for "queue empty". PUT does a P on "queue full", puts on an item, and does a V on "queue empty". GET does a P on "queue empty", takes off an item, and does a V on "queue full". It's very simple. That's the real use case for P and V. Linus' note indicates that in 1999 he didn't know this.

(I didn't write those primitives, but I've used that code, and once ported it to a Pascal compiler I adapted to handle concurrency.)

This stuff was all well understood four decades ago. Much of it was forgotten outside the mainframe world, because threads and multiprocessors didn't make it to microprocessors for several more decades. UNIX, for a long time, had very primitive synchronization primitives. Early UNIX didn't have threads, and even after it got threads, it took years before the locking primitives settled down. The DOS/Windows world didn't get them until Windows NT, circa 1993.

It's been amusing to me to see bounded buffers resurface in Go. They're quite useful, and I've been using them in concurrent programs for many years.

<comment><USER3> For those wondering, P comes from 'Passering', roughly translated 'pass' (as a noun), and V from 'Vrijgave' ('release'). Apparently somehow this terminology comes from train systems but there's not a lot of context on that etymology.

As an aside, this paper (it's actually the transcription of a lecture) has some great metaphors that explain problems with concurrence and issues with synchronisation. At the risk of losing much of the nuances, he essentially illustrates the synchronisation using the example of a teacher who needs to find a pupil in a class. When pupils are free to choose a seat, she needs to scan all seats when she is looking for a particular pupil; but when at the same time pupils are free to change seats as she's scanning, there is no guarantee that she will ever find the pupil she's looking for as he might run from the back of the class to the front as soon as she's done scanning the front.

<comment><USER4> From what I've heard, the P comes from "Probeer" (to "try"), and the V from "verhoog" (increase).

This makes more sense to me.

<comment><USER5> That's Dijkstra's later terminology. In EWD35 he introduced the operations as "passering" and "vrijgave". Then in EWD51 he uses "verhogen" and the neologism "prolagen" ("probeer te verlagen"). In EWD74 he switches to "proberen".

These documents are available from the Dijkstra archive: [http://www.cs.utexas.edu/users/EWD/welcome.html](http://www.cs.utexas.edu/users/EWD/welcome.html)

</comment></comment></comment></comment><comment><USER6> _" Dijkstra was probably a bit heavy on drugs or something (I think the official explanation is that P and V are the first letters in some Dutch words, but I personally find the drug overdose story much more believable)."_

Quotes like this are why I always read what Linus has to say, regardless of whether the subject is relevant to my life in the slightest.

Edit: Yes people, those Dutch words exist! I get it! I'm sure Linus was well aware of that when he made this remark. However, the point Linus was making (in a humorous way) was that like most computer scientists / mathematicians, Dijkstra was overly fond of obscure, single-letter names, which nobody ever intuitively understands. Whereas the names "up" and "down" (see the rest of Linus' quote) are far more intuitive. Personally, I would argue that "hold" and "release" convey the intent in a clearer, more abstract way.

<comment><USER7> > "Dijkstra was probably a bit heavy on drugs or something (I think the > official explanation is that P and V are the first letters in some Dutch > words, but I personally find the drug overdose story much more believable)."

I personally have made remarks that were WAY more potentially offensive than that, verbally, with friends. But, accusing Dijkstra of being a drug user, on the Linux kernel mailing list, from a @transmeta address?

That's grounds for termination in many companies, even if you don't get lawyers involved. I think he's only able to get away with it because he's Linus.

<comment><USER8> Technically, you're right. Today, any junior employee that made a joke like that would be crucified and practically blacklisted from the industry. What you should be thinking, however, is not "why does Linus get away with saying things like that?" but rather "why do I consider it normal for someone's livelihood to be permanently destroyed over a stupid joke?"

<comment><USER9> What do you think an appropriate response is?

<comment><USER10> How about public shaming instead of the destruction of a career?

<comment><USER11> That's the idea. Any developer or researcher who uses obscure or meaningless names like "P" and "V" needs to be publicly shamed for it, no matter how well- respected they are. Clarity is vital in this business.

Or were you talking about Linus?

</comment></comment></comment></comment></comment></comment><comment><USER12> Semaphores are basically unused in the kernel these days, abandoned in favor of mutexes. I really recommend reading kernel/locking/mutex.c. You can learn a lot about the details of low-level synchronization, and it's also just a really impressive piece of ruthlessly-optimised code. Tricks like reading the lock's 'owner' field without any sort of synchronization, to decide whether to spin on the lock or to sleep.

<comment><USER13> For the sake of completeness: [https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/mutex.c)

</comment></comment><comment><USER14> it's surprising that Linus answers peacefully and pedagogically ! and thus it's a nice read to refresh the definition of semaphores, spinlocks and mutexes.

Maybe you can edit the title and add a little [199] :-)

<comment><USER15> He did start out by saying Peter Samuelson's CS education was bad, so there was certainly some of the infamous Linus in there.

<comment><AUTHOR> Can you blame Linus for saying that? I can't speak with precision about 1999, but someone assumed to have a CS education should know the difference between semaphore and spinlock.

<comment><USER16> Whether they did or didn't know about the definitions, they should have at least looked it up to confirm before posting to the Linux Kernel Dev list or taking on Linus on the matter.

</comment></comment></comment></comment><comment><USER17> Link is to a collection of Linus's comments, from between 1999 and 2008, about how to efficiently implement mutex and semaphores in the Linux kernel.

Linus is the BDFL of the Linux kernel, and he obviously needs to think about the big picture. And yet in these comments he gets into nitty-gritty assembly language.

I love it when a big picture guy also sweats the details.

</comment><comment><USER18> Nice read, but still some misconceptions and misunderstandings.

If I am in a multiprocessor design, where each processor runs completely independent from each other and there is no central organizational unit like a central OS, like in many embedded designs, there is no such thing as a spinlock. A semaphore is also not used for sending processes to sleep.

Linus explanation makes sense, speaking only about a single OS, which may is comprised of multiple processors. But makes no sense at all for real multi- processor designs. I don't blame him for his narrow view, more his professors that he never experienced real embedded design.

I have worked in the past with multi-processors designs, where one processor was using one OS, like OS9, and the other processor had no OS at all running. Both processors where exchanging data via a dual ported RAM. So basically both processes where competing for the same resource. But even than, using a semaphore does not prevent race-conditions. Even then it is very tricky to use them. Because dual ported RAM allowed real concurrent access to the same resource.

In summary, the semaphore definition is the more broad definition where 2 (or more) processes are competing for a single resource. The naming convention within Linux is more specific for this requirements. The "invented" spinlock is just a renamed version for a fast semaphore, AFAIK. Even a mutex is just a special case of a semaphore.

<comment><USER19> You should realize that the post was written in 1999.

<comment><USER18> And I have done my work on this back around 1997/1998 as part of my diploma thesis.

</comment></comment></comment><comment><USER20> The "benchmarks game" has a thread ring benchmark which can compare conditions/mutexes/semaphores, because for some applications they're interchangeable:

4-thread x64:

mutex (480s): [http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=threadring&lang=gcc&id=1)

sem_wait (476s): [http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=threadring&lang=gcc&id=2)

cond_wait (271s - runs single-threaded?) [http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=threadring&lang=gcc&id=3)

1-thread x86:

mutex (136s): [http://benchmarksgame.alioth.debian.org/u32/program.php?test...](http://benchmarksgame.alioth.debian.org/u32/program.php?test=threadring&lang=gcc&id=1)

sem_wait (131s): [http://benchmarksgame.alioth.debian.org/u32/program.php?test...](http://benchmarksgame.alioth.debian.org/u32/program.php?test=threadring&lang=gcc&id=2)

cond_wait (156s): [http://benchmarksgame.alioth.debian.org/u32/program.php?test...](http://benchmarksgame.alioth.debian.org/u32/program.php?test=threadring&lang=gcc&id=3)

</comment><comment><USER21> If you haven't already, follow the 'index' link (to [http://yarchive.net/comp/index.html](http://yarchive.net/comp/index.html)) and bookmark that. Some very worthwhile reading there.

<comment><USER22> That whole site is fascinating, I just lost like 2 hours of my life. Late 90s threads about TLB strategies from @sgi.com e-mail addresses? F&$% me, I could read that sort of stuff all day. (I started [http://www.snakebite.org](http://www.snakebite.org), for reference.)

</comment></comment><comment><USER23> The possible future improvement that Linus mentions here:

      For example, the per-VM memory management semaphore could very usefully
      be a blocking read-write lock, but without heavy thread contention a
      mutex semaphore is basically equivalent.

has actually come to pass: the mm_struct is now protected by struct rw_semaphore mmap_sem.

</comment><comment><USER24> This is why I _LOVE_ reading what Linus has to say, it's all very practical, and pretty much ignores theory, because he has actual evidence to support his claims.

</comment><comment><USER25> For those that care why Dijskstra used P() and V(): P for Passering; V for Vrijgave (release)

<comment><USER26> Thanks. Fun quote FTA:

"" They originally had operations called "P()" and "V()", but nobody ever remembers whether P() was down() or up(), so nobody uses those names any more. Dijkstra was probably a bit heavy on drugs or something (I think the official explanation is that P and V are the first letters in some Dutch words, but I personally find the drug overdose story much more believable). ""

</comment></comment><comment><USER27> Ah, semaphores. I recall in my operating systems course we had to implement some simple concurrency patterns using semaphores as an exercise -- for instance synchronize a group of processes so that idle processes gather into groups of N, and when a group is ready, N-1 threads of the group does TaskX(), and 1 thread does TaskY(), and when they're done, they go back to the pool. Then we implemented the same using usual mutexes and condition variables. I encourage everyone to do that, to see how one of the approaches is much more complicated than the other. Linus says that almost all practical uses of semaphores is when they are just used as mutexes, and rightly so -- implementing concurrency patterns based only on semaphores is a total pain.

<comment><USER28> > Linus says that almost all practical uses of semaphores is when they are > just used as mutexes, and rightly so -- implementing concurrency patterns > based only on semaphores is a total pain.

Semaphores are not very good in practical programming but they're still valuable as a mental model and reasoning about algorithms.

It is a lot easier to prove by induction that an algorithm implemented with semaphores works than it is to deal with mutexes and conditions in a formal proof.

So semaphores are very useful in theoretical work, mutexes and conditions are more useful in practice.

</comment></comment><comment><USER29> There's a free book available on the subject of semaphores:

 _The Little Book of Semaphores_ by Allen Downey

[http://greenteapress.com/semaphores/](http://greenteapress.com/semaphores/)

</comment><comment><USER30> Semaphores make a lot more sense if you know their literal translation of the Spanish word "semaforo", which is a "signal". It's also used in Spanish to describe a traffic light.

<comment><USER31> I thought it got its name from the Semaphore Flag Signalling System.

[http://www.anbg.gov.au/flags/semaphore.html](http://www.anbg.gov.au/flags/semaphore.html)

</comment></comment><comment><USER32> Semaphore and spinlock sure are different. People rarely use spinlock in user mode programs since there are better synchronizing mechanisms in user mode than busy waiting with spinlock. However, in kernel mode program spinlock is invaluable since it's the only synchronizing mechanism that works in any interrupt level.

</comment><comment><USER31> Wow I remember some of this class from my OS course 10 years after this was written.

We were just told about mutexes, but I never knew that it stood for Mutual Exclusion. (I did correctly intuit that a mutex was simply a specific use case of semaphore though so I'm happy and managed to pass the course in the end :D)

</comment><comment><USER33> Coming from embedded systems and MCU RTOS development, mutexes and semaphores are the name of the game.

</comment><comment><USER34> Man every time I read linux kernel stuff I want to join the dev team. Kernel development sounds awesome/fun. But I can never seem to make time.

</comment><comment><USER35> That was a pretty interesting read.

Random question, does anyone know what some more active newsgroups lists are today or has it all slowed down?

</comment><comment><USER36> Love vintage concurrency techniques :).

<comment><USER37> There's nothing vintage about them. Understanding these is crucial to understand how and why modern concurrency tools work. As I noted above in another comment, the issues that need mutexes or semaphores have never went away, only you might not realize that they're there.

And please, don't come up with async/await, callback/continuation, node.js' async stuff. They are not a replacement for mutual exclusion, etc.

<comment><USER38> Vintage doesn't mean obsolete.

<comment><USER39> Vintage means "from a particularly good year, the like of which we have not seen for some time". Seems about right for an invention of EWD's.

</comment></comment></comment></comment><comment><USER40> nice</comment>


Code Changes to prepare GMail for the Future - <AUTHOR> http://gmailblog.blogspot.com/2007/10/code-changes-to-prepare-gmail-for.html

<comment><AUTHOR> This caught my eye: "One side effect of this change is that if you're using third-party Gmail extensions, they're likely to stop working."

I assume that means GMail Drive?</comment>


Google offers to help others with the tricky ethics of AI - <AUTHOR> https://arstechnica.com/tech-policy/2020/08/google-offers-to-help-others-with-the-tricky-ethics-of-ai/ <comment><USER3> > _Google's new offerings will test whether a lucrative but increasingly > distrusted industry can boost its business by offering ethical pointers_

Isn't google a big part of why there is so much distrust to begin with? Seems like asking your oxy dealer for detox advice.

<comment><USER4> Opiate dealers often stock narcan.

<comment><USER5> But only the worst dealers would sell it, most would just keep it or give it to you. It's the quintessential olive branch — "Keep buying drugs from me, I'm the nice one"

</comment></comment></comment><comment><USER6> A lot of knee-jerk Google-hating in this thread, which I think is unfounded in this specific context. Google's AI safety and AI bias toolchain is by far the most robust that I've seen. Seems to me that they are investing much more heavily than other players.

Note that this is a different issue than data privacy, which Google rightly takes flak on. AI bias is referring to questions like "if I train a network to (sentence criminals, price insurance, ...), how do I detect if the resulting predictions are racially biased?"

The conversation on this subject by lay folks is rife with statistical ignorance, and Google has done good work communicating and clarifying the conversational starting points. This is a hard issue because it takes ethical trade-offs and forces you to specify mathematically exactly how you want to handle inequality, which is a subject that most people haven't thought through rigorously, and would rather hand-wave away with virtuous sound bites.

For example, see [http://research.google.com/bigpicture/attacking- discriminati...](http://research.google.com/bigpicture/attacking- discrimination-in-ml/)

<comment><USER7> That example ([https://research.google.com/bigpicture/attacking- discriminat...](https://research.google.com/bigpicture/attacking- discrimination-in-ml/)) isn't particularly compelling.

As their example shows, there're two traditional approaches: maximize profit and group-unaware.

* Maximizing profits gets the most profit, but treats people differently based on their group.

* Group-unaware treats everyone the same regardless of their group, but can generate far less profit.

The example presents two alternatives: " _demographic parity_ " and " _equal opportunity_ ". Presumably the authors would argue that these may be superior choices because they generate nearly as much profit as profit-maximization while having a plausible argument for being socially responsible.

This seems a bit off.

Fundamentally, we might say that there're 2 kinds of discrimination: fair and unfair. For example, it might be fair to discriminate for objective reasons, but it'd be unfair to discriminate for non-objective reasons (e.g., bigotry).

The advantage of profit-maximization is that it takes full advantage of fair- discrimination while fully avoiding unfair-discrimination; the drawback is that it does discriminate.

The advantage of group-unaware is that it fully avoids all discrimination; the drawback is that it sacrifices fair-discrimination, causing it to yield the lowest profits.

The two alternatives proposed in that example seem to get the best of both worlds because they're basically just cloning profit-maximization, but with slight concessions to plausible-sounding criteria for equality to dodge perceptions of unfair-discrimination.

Here're the tricks:

* In " _demographic parity_ ", everyone has the same odds regardless of group. This would appear to be the same thing as profit-maximization if the risk/rewards were the same, but since it ignores them, it ends up being basically " _profit maximization, but ignoring different risk /rewards_".

* In " _equal opportunity_ ", both groups get the same true-positive rate. This again seems to sacrifice some of the risk-vs.-reward information, but with a slightly different skew.

So for the privileged group (Orange) vs. the disadvantaged group (Blue):

* Profit maximization: $32,400 from 50 vs. 61

* Group-unaware: $25,600 from 55 vs. 55

* Demographic parity: $30,800 from 52 vs. 60

* Equal opportunity: $30,400 from 53 vs. 59

To me, that looks like 3 ways to discriminate, all yielding roughly the same profit and thresholds -- using either of the 2 proposed alternatives gives up a little bit of the profit in exchange for a pleasant-sounding rationale.

What I dislike about this is that it seems entirely superficial. The proposed alternatives engage in roughly the same level of fair-discrimination (and none of them engage in unfair-discrimination, which wasn't given in the example at all) to generate roughly the same level of profit.

\---

> The conversation on this subject by lay folks is rife with statistical > ignorance, and Google has done good work communicating and clarifying the > conversational starting points. This is a hard issue because it takes > ethical trade-offs and forces you to specify mathematically exactly how you > want to handle inequality, which is a subject that most people haven't > thought through rigorously, and would rather hand-wave away with virtuous > sound bites.

That's exactly what this looks like!

In this case, the virtuous sound-bites are " _Demographic Parity_ " and " _Equal Opportunity_ ". They both worked out to be mostly the same as simple profit-maximization, but if someone in a disadvantaged group protests that they're being discriminated against, they'd probably find it difficult to follow the math far enough to sustain their complaint.

<comment><USER6> I appreciate the detailed object-level analysis of that case, but my point was more intended at the meta-level — in order to have a conversation about AI bias, we need language and examples to start from. Google is building these fairness metrics into parts of its cloud ML toolchain, and is investing in peer-reviewed research in this area. That's more than most other companies, and should be applauded.

At the object level, I think you perhaps oversimplify with " Fundamentally, we might say that there're 2 kinds of discrimination: fair and unfair." What we think is fair is the crux of the whole issue, and there is not agreement on that distinction. For example, if we price risk at the "true" risk of default, and this means that fewer black people get loans, is that fair? Some think it is, and some think it is not, even though I hope most would agree there is no racist intent in the decision itself. The economic context in which these decisions are made already contains the impact of past racism, which can be perpetuated or or restituted by decisions made now.

I also disagree with your characterization of these types of parity as sound- bytes. My definition of a sound-byte is a conceptually thin phrase that sounds good, but does not contain much content. These are the opposite; they are specific jargon with a precise meaning that is explained in detail in the article. You could call them "Foo" and "Bar" and their usefulness in the conversation would be the same. (Sure, "equal opportunity" has been deployed as a sound-byte elsewhere without any rigorous definition, but I hope you'd agree that this article at least fleshes out one possible concrete definition, even if you don't think it is the right one. I'm sure they are others - <USER2> now we can further the field by writing them down, citing this piece, and making an objective critique!)

<comment><USER7> > My definition of a sound-byte is a conceptually thin phrase that sounds > good, but does not contain much content.

Yeah, same. And to my eye, this whole thing's paper-thin; it tested my suspension of disbelief to not dismiss it as an obvious joke or bad attempt at a PR stunt, and I'm honestly still undecided.

Part of the issue is that its conclusions are absurd. We can invent ethical constraints to arrive at max-profit by pulling things out of thin air; the procedure's analogous to _p_ -hacking. For funsies:

1\. Come up with a huge portfolio of ethical-sounding constraints. Include basically any ethical-sounding argument imaginable, even if it seems stupid or bad for business (the numbers can be hacked; we just need the pretty words).

2\. Come up with model-transform strategies. For example, instead of calculating the threshold, calculate the log-threshold or the threshold- entropy. (Again, don't think logically -- we're hacking the math, so just come up with random things.)

3\. Come up with model-hybridization strategies. For example, construct a meta-model that uses 50% of the threshold + 25% of the inverse-log threshold + 20% of the inverse-entropy threshold + a flat 5% (because, hey, why not?).

4\. Whenever we want to maximize profit, just do so with simple profit- maximization. No " _ethical_ " constraints.

5\. Repeat Step (4) for millions (or more) different hybridized models, combining all sorts of different model-transforms and ethical-constraints at random. Optimize each for approach to simple-maximization.

6\. Stop upon finding an " _ethical_ " solution that effectively equals simple profit-maximization.

7\. Generate a description of the " _ethical_ " solution, and write a report on how your company's using that particular ethical strategy as part of its latest, on-going campaign to promote social equity.

8\. Just do simple profit-maximization. Or the " _ethical_ " solution; whatever, they're literally the same thing. But if anyone asks, be sure to tell them it was for ethics.

While we're at being ethical, we could even shift some paradigms to envision new modes of engagement with socioeconomically disadvantaged stakeholders to promote a synergistic realignment between our commitment to ethical outreach and disrupting the zeitgeist with our social media blitz to rapidly accumulate surprisingly liquid social capital, indelibly scrawled across the human blockchain of love and mutual respect, validated in the proof-of-work of billions of sapient hearts yearning to emerge from the trials-and-tribulations of our modern era into the welcoming bosom of a new tomorrow.

But who cares about pointlessly defining jargon for logical inconsistencies, though? It's hallow and meaningless; at most, it might fool people before they realize that it's logically inconsistent, but that'd just be lies. It doesn't seem to have an actual use.

</comment></comment></comment></comment><comment><USER8> Gah. I'm experimenting with an iPhone as a second phone to dip my toes into the iOS ecosystem because despite their other flaws they've at least learned the lesson "don't be f __ckn ' _creepy_ " which Google has not.

Being able to count on probably 5 years of OS and security updates instead of the mishmash of abandoned handsets or potential security nightmares with hobbyist firmwares is another nice feature.

</comment><comment><USER9> I made it two paragraphs before I remembered the preface of Google's code of conduct was stripped of the line "Don't be evil." It's now the last line, which to me reads as _doing the ethical thing is an afterthought_. The much more ambiguous "Do the right thing," is in Alphabet's corporate code.

We need the ethics of AI to be an offshoot of open source. It's tough to suggest that in an environment where enough people can pressure a group to adopt objectively unethical policies, but at the very least open source communities do not have the trigger of financial risk.

Google is going to need to make money from this "Ethics as a Service" eventually. And that is an obvious, all too obvious conflict of interest for what Google envisions.

Maybe they _can_ do this. I'm not 100% objecting to the idea. But I'd like to suggest they start with YouTube. Ads on the platform have quadrupled since November 2019. Creators are playing a lottery with monetization every other upload, and they're making as little as 1-5% compared to YouTube's ad-share program 10 years ago. Start there.

<comment> twitch-chat As far as the ethics of AI goes, would "don't be evil" slogan be better than "do the right thing"?

I'm sure people may interpret things differently but doing the right thing would mean Google making AI that has a positive net impact on society, while "don't be evil" could simply mean good for Google but neutral to everyone else.

<comment><USER10> It's easier to know what is wrong that what is right.

</comment></comment></comment><comment><USER11> Reminds me of [https://en.wikipedia.org/wiki/Comics_Code_Authority](https://en.wikipedia.org/wiki/Comics_Code_Authority) \- a sneaky way for a private entity to act as de-facto arbiter and censor.

</comment><comment><USER12> Please don't

</comment><comment><USER13> Google helping on ethics: the blind leading the blind!

Some fun history: I thought the phrase originated with the Christian bible:

> _" when the blind lead the blind, both shall fall into the ditch"_

but Wikipedia [https://en.wikipedia.org/wiki/The_blind_leading_the_blind](https://en.wikipedia.org/wiki/The_blind_leading_the_blind) traces it to the Upanishads:

> _" Abiding in the midst of ignorance, thinking themselves wise and learned, > fools go aimlessly hither and thither, like blind led by the blind."_

</comment><comment><USER7> > Longer term, the company may offer to audit customers' AI systems for > ethical integrity, and charge for ethics advice.

If Google'll act as any other company in a free market, then this'd seem like a neat offering. Maybe it'll be a helpful service, or maybe it'll be a flop; but, worst-case scenario, just don't use their service if it's no good.

But the language of " _auditing_ " and " _charg[ing] for ethics advice_ " sounds almost like someone's laying the groundwork for regulatory authority over AI.

</comment><comment><USER14> Ag yes, that paragon of ethical data use, Google.

<comment><USER15> Next up: "Media accuracy and journalism standards" by Facebook

</comment></comment><comment><USER16> Google offers help with ethics. Love this headline! So funny!

</comment><comment><USER17> Step 1. Do no evil

Step 2. JK

</comment><comment><USER18> Google is in such a strange state. I deal with Googlers several times a week, and their defining characteristic is hubris. It seems like their arrogance blinds them to the fact that their credibility has been in a free-fall for years. Publishing articles like this is just laughably tone deaf.</comment>