"A closure is a poor man's object; an object is a poor man's closure" (2003) - <AUTHOR> http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg03277.html <comment><USER2> My definition of an object is "an ADT that gets to 'decide', using private runtime state, what to do in response to any/all attempts to interact with it."

Which makes for a very simple test: if something is an object, you should be able to send it a message that makes it change its mind about what messages it accepts after that.

There's no obvious way to implement a behavior like this using the native "objects" in nominally-object-oriented languages like C++ or Java. (But you _can_ do so with the closure types from these languages. Or you can look higher on the stack: threads with IPC queues, or POSIX _processes_ , are both objects.)

But implementing this behavior is obvious in Ruby; this is how basic things like Object#extend work.

And implementing this behavior is also obvious, oddly enough, in Erlang: [http://joearms.github.io/2013/11/21/My-favorite-erlang- progr...](http://joearms.github.io/2013/11/21/My-favorite-erlang-program.html)

In conclusion, Erlang is more object-oriented than Java. ;)

\---

And yeah, I know, this definition of OOP-ness sort of flies in the face of decades of things people have said about OOP. Objects, under this definition of OOP, don't obey any of the SOLID principles. They're slippery; unpredictable; "alive."

But this was the original definition! It's the one Smalltalk fits; it's the one LambdaMOO fits; it's what is meant by referring to Javascript as object- oriented.

It's a shame we have this collision between meanings; having two separate words for "OOP like Smalltalk" and "OOP like Java" would have saved language- designers all a world of arguments from people who try to propose extensions to a language in one category, assuming it's in the other.

<comment><AUTHOR> Your definition of an object is similar to one that I've used[1][2][3], which is unsurprising since we got it from the same place.

But, now I define an object as any component with which you interact strictly by way of negotiation (as opposed to command and control), and I finally think I understand why Alan Kay says that the most important thing in OOP is the message passing.

Thus, statically-typed languages like C#/Java/C++/etc. are also capable of being object-oriented as long as you avoid setters and realize that some of the negotiation happens at compile-time.

[1] [https://news.ycombinator.com/item?id=8679224](https://news.ycombinator.com/item?id=8679224)

[2] [https://news.ycombinator.com/item?id=11967631](https://news.ycombinator.com/item?id=11967631)

[3] [https://news.ycombinator.com/item?id=11809477](https://news.ycombinator.com/item?id=11809477)

<comment><USER3> I'm not sure what's the problem with setters? In my eyes, they are just a convenient shorthand for a common message. For example, Objective-C before version 2.0 didn't have setters; so the libraries had lots of methods like "setTitle:", "setDelegate:" etc. Setters are just a convenient shorthand. And in ObjC and in Swift (I don't know Java/C#/C++ so well) you can override setters to do whatever you like, so there really isn't much of a difference between using a setter and sending a message -- it's just syntactic sugar that makes code more readable.

<comment><AUTHOR> By "setter", I mean a mechanism by which code that is external to an object can gain _unmediated_ access to one or more of the object's mutable instance variables.

The big idea is that the sender should not be able to assume that the message, `setFoo`, will have any particular effect with respect to the reciever's state.

<comment><USER4> At least in C# terminology, a "setter" for a public property is mediated access, unmediated access would be a public field that can be set without invoking a setter method.

Getters and setters in C# are method calls, like `setFoo` in Java, so can't be assumed to be simple, they can do anything any other method can do.

<comment><AUTHOR> As it turns out, C# is a language that I know _extremely_ well; I'm aware that setters can have arbitrary code. But, here's the problem: A programming a language is medium for thought, which means that it promotes or prevents the occurrence of certain ideas.

So, even though the compiler will accept any legal void-returning code within the inner braces of

        public Foo Bar
        {
            set
            {
              //Do stuff
            }
        }

The fact that the client code is going to look like `Obj.Foo = whatever` will practically foreclose on all but a small subset of possibilities.

</comment></comment></comment></comment></comment></comment><comment><USER5> I enjoyed building object systems in scheme. The closure way is kind of like focusing on the code without worrying to much about the underlying data structure. The object way is worrying about the data structure and the code is secondary.

<comment><USER6> I think in effect you're saying that they're duals.

I think I would argue that objects bundle data and functions in a useful way (when done right). Closures are more of an expressive convenience for constructing functions on-the-fly. The overreach for objects is to consider them as the only way to organize data and functions.

<comment><USER7> What is a dual?

<comment><USER8> It's a table in all Oracle databases that has nothing in it. Also, it's this: [https://en.wikipedia.org/wiki/Duality_(mathematics)](https://en.wikipedia.org/wiki/Duality_\(mathematics\))

</comment></comment></comment></comment><comment><USER9> I actually studied carefully the Koan from the Japanese Buddhist traditions a few years ago. I can't tell you much about Japanese Buddhism, but I can tell you a lot about the Koan.

Not all Koan share all the same themes with each other, but for my money, a contemporary Koan really should try to embody all of a few important themes that I find repeated throughout the Koan.

A lot of non-experts seem to want to emphasize how often someone is struck, but for me I think this comes from a one of the most interesting themes in the Koan: a sense of humor about failing to attain enlightenment.

When a person attains enlightenment in the Koan, it is often at the moment when they are forced to acknowledge the duality of two opposing views, as if in the fleeting moment between seeing Rubin's Vase or the faces in its negative spaces.

In that sense, at least, it is a perfect Koan.

</comment><comment> white-flame Multiple closures can overlap in their referencing of some set of variables. Objects can't really do this, so they're not _that_ similar.

Only the most simplistic application of closures (one unique set of instantiated variable slots per closure) matches object encapsulation.

<comment><USER10> Objects containing references can share referents.

<comment> white-flame That's a feature of the references, not of objects.

The sharing and overlapping of data slots is however an inherent feature of closures.

<comment><USER10> > That's a feature of the references, not of objects.

That doesn't seem like an unreasonable way to draw the lines.

> The sharing and overlapping of data slots is however an inherent feature of > closures.

Is it? When defining a lambda in C++ you can choose for things to be captured by copy rather than reference. I _agree_ with saying that a function that captures nothing is "not a closure", but I don't know that I'd say the same about something that captures only by copy. Would you say that a language that only supported capture-by-copy "lacks closures"?

<comment> white-flame > That doesn't seem like an unreasonable way to draw the lines.

It's a turing tarpit otherwise, after all.

Regarding C++, I've not used the newer versions, but using that feature to capture only copies would match what I said upstream, as a limited usage of closures that would happen to match objects, while closures themselves can do more. Capturing by copy also means that it's instantiating a new closure variable slot specific to the copy, and is no longer closing over the original variable, so it's an additional step on top of the fundamental closure mechanism which would by nature be agnostic to private vs shared variable slots.

</comment></comment></comment></comment></comment><comment><USER11> While I've seen various methods in scheme to emulate objects with closures, I haven't seen a good one from the statically typed FP camp (the ML family, Haskell, etc). Meanwhile, objects are able to masquerade as closures very nicely in statically typed OO object systems like Scala and C#.

So I lean more towards closure being a poor mans object than vice versa.

<comment><USER12> OCaml, Objective Caml, has a built-in Object system. Objects are immutable which prevents them from carrying identity (though mutation can be opted into to recover this). They're also "just" row-typed records. Classes are a form of constructor function which have inheritance for code sharing purposes. This isn't emulating objects with lambdas, but it is a well-articulated idea of what objects in a pure-ish FP setting might look like.

There have been many attempts to bring various forms of "object" into Haskell. It tends to be either a very simple or a very messy affair. On the messy side, strict purity blows up object identity again and there are only very messy ways to recover it. This makes most OO such a pain that you're more likely to factorize it into smaller pure components which appear more functional.

On the simple side, corecursive patterns are distinctly "object like" and are used _all the time_. Laziness gives you infinite data structures which, in effect, give you a limited form of mutability which can be _incredibly_ helpful while still be easy to reason about.

\---

Finally, I think that practically there is a strong case to be made for introducing actor-like objects into Haskell. It has a great runtime and exception system to support it, and it would allow a bit more structuring "at the highest level" of application architecture. Today these are often managed as imperative programs (built an IO chunk, execute it). This tends to work, but Erlang did it better.

<comment><USER11> _OCaml, Objective Caml, has a built-in Object system. Objects are immutable which prevents them from carrying identity (though mutation can be opted into to recover this). They 're also "just" row-typed records. Classes are a form of constructor function which have inheritance for code sharing purposes. This isn't emulating objects with lambdas, but it is a well-articulated idea of what objects in a pure-ish FP setting might look like._

Ocamls objects are great. I'm familiar with them and a big fan. I wish there was a whole language based around them, ie everything is an ocaml style object.

That having been said, they do feel odd and out of place in Ocaml, and people don't seem to use them much.

</comment></comment></comment><comment><USER13> Both are valuable under the right circumstances. A language that supports both is stronger than a language that only supports one or the other.

In that respect, I think the koan is perfect. :)

<comment><USER14> What does "stronger" mean in this context?

According to this definition, assembly is stronger than Haskell, because it contains stateful objects (registers), whereas a state-containing object cannot be present in Haskell code, although one may appear when you run the code in question.

<comment><USER13> Stronger means higher leverage for the typical programmer. Assembly language isn't very high leverage (I say that having written several games professionally in entirely assembly language).

Assembly language also doesn't natively support either closures or objects, except in the sense that _of course_ you can build them from first principles, Turing Completeness and all that.

Until someone proves to me that Haskell can be used in a reasonable way by a "typical" programmer, I simply won't think about pure-functional when I make general statements about languages.

I fully admit this may be a hole in my brain, but I don't understand how to break down the problems I face on a daily basis into Haskell (or any pure- functional language), so I just talk about the universe of languages that are _not_ pure-functional.

I've looked at Haskell tutorials and gotten at least hip deep into the language, by the way. I just usually end up dealing with problems that require levels of complexity where FP tools fall down. Your mileage may vary.

</comment></comment></comment><comment><USER15> They're both wrong. Closures and Objects are both structures with function pointers and delusions of grandeur.

<comment><USER16> Which is funny because I could never get anyone, when I first started programming in C and everyone was raving about how great 'object oriented programming' was, to tell me what 'an object' actually was. When I started learning C++ I was somewhat disappointed to find out an object was just a struct with some functions attached.

(It's kind of like when people told me about "user defined data types" and I couldn't wrap my head around how you could create a new data type with all its associated syntax. Then when I learned that they were just bundles of existing data types, though 'oh is that all?')

<comment><USER11> I love OO - but C++s objects are so bad that most of the time I just end up using functions-where-the-struct-is-the-first-argument, C style. If I'm feeling particularly energetic I might use the PIMPL idiom. But naiive use of C++ objects is absolutely painful for me.

All language constructs can be broken down into simpler ones, at the end of the day. We might talk about how subroutines are just overrated assembler macros. C++ isn't very clean so the fact that these fancy schmancy objects are just structs + function pointers is very obvious. But when done cleanly, I'd argue that an object is more powerful than the sum of its parts.

<comment><USER17> _But when done cleanly, I 'd argue that an object is more powerful than the sum of its parts._

I think I agree with your assessment of C++. What languages do you feel deal cleanly with objects so they exceed the sum of the parts?

<comment><USER18> I've seen objects recreated in Scheme and Clojure. They are really much more common than most FP people appreciate, just look for noun-ish naming conventions in your abstractions.

<comment><USER11> It's controversial for some people. OO is that horrible verbose enterprise code with all the mutable state. And if you suggest that certain features or patterns approximate OO they get very defensive.

There are certain famous FP language implementers that should probably read what Guy Steele wrote in the topic, because listening to their talks I strongly get the impression they haven't.

<comment><USER18> One could also just use a synonym of object, like "entity." "Hey...we aren't using objects here, we are using entities!" (Facepalm)

I saw Guy's growing a language talk before, but I don't think he's ever chimed in on this debate before. Most people at his level simply except the world without much bias, and are comfortable using object and functional styles when necessary.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> 'Qc Na responded by hitting Anton with his stick, saying "When will you learn? Closures are a poor man's object." At that moment, Anton became enlightened.'

The cruel tutelage of Qc Na.

<comment><USER20> The Zen Slap:

"If you read through enough Zen koans, you're quickly struck, as it were, by the number of those oblique lessons that end with a student getting slapped. It may be delivered by hand, stick, sandal or goat bite, but the slap, painful as it might be, is an act of compassion, a wordless reminder of the things we know but can never seem to remember or practice for very long."

\-- [http://www.siliconbeat.com/2008/10/09/of-course-if-theres- no...](http://www.siliconbeat.com/2008/10/09/of-course-if-theres-no- enlightenment-a-zen-slap-just-hurts/)

<comment><USER21> The TV show NCIS borrows from this philosophy. Gibbs always smacks the back of his underling's heads when they do something stupid. But if he doesn't, they worry he has given up on them. They also made it a point saying that smacking the front of the head would be offensive.

</comment></comment></comment><comment><USER22> Really understanding this means knowing why a closure is a poor man's poor man's closure.

<comment> white-flame When you get to my depth, you'll understand that a closure is a poor man's poor man's poor man's object. And it smells like a turtle, for some reason.

</comment></comment><comment><USER23> The main use of closures today is to create something you can pass to a callback. In theory, you could pass an object, but the callback would have to cooperate by accepting an object and an object method, or a reference to an object method/object pair, in languages which support that. This requires more standardization than one is likely to get.

<comment><USER24> I think of closures as a bit orthogonal to that; the definition of "closure" that I'm used to is a function that "encloses" some data that is not an argument (somewhat like a function with free variables in lambda calculus), whereas a callback might only deal with its arguments. That being said, sometimes I see the term "closure" being used in a way that I associate with the word "lambda", which feels a bit off to me (I'm looking at you, Rust...)

<comment><USER23> A lambda is an anonymous function. A closure is a function which carries with it some state. The two concepts are independent - you can have named functions as closures, and lambdas that aren't closures. Rust does seem to tie the concepts together syntactically.

Languages which support nested functions usually allow those inner functions to be closures.

<comment><USER10> Yeah, any time you have a nested function you raise the question of "what can this close over?" Sometimes the answer is "nothing", and that's not a closure... but it's still an answer to the question so it makes some sense to talk about it in the same breath.

<comment><USER23> Lambdas are just functions with different syntax. Closures imply keeping some state around, and may require that some variables from outer scopes live longer than they otherwise would. (That's what they're usually used for in Javascript, where you want state kept around for a reply event.)

Closures are easy to implement in garbage-collected languages, but get complicated in non-garbage collected languages. When an inner function normally references variables in an outer scope, they're just references to the stack. But if there's a closure that can outlive the outer scope, those variables have to be saved somewhere.

With C++11 closures, you have to specify what you want to keep around and how you want it kept.[1] Internally, C++11 creates an object to store the variables you want to keep around. This can potentially create lifetime problems, because you can save a reference to a variable, then keep the closure object around longer than the variable being referenced. This creates a dangling pointer. Rust has lifetime checking to catch such errors, but C++11 does not. See this Stack Overflow discussion of how not to shoot yourself in the foot with C++11 closure lifetime undefined behavior.[2]

That's why it's useful to realize that closures and lambdas are not the same thing. Lambdas are simple. Closures are complicated.

[1] [http://www.cprogramming.com/c++11/c++11-lambda- closures.html](http://www.cprogramming.com/c++11/c++11-lambda-closures.html) [2] [http://stackoverflow.com/questions/27775233/lambdas-and- capt...](http://stackoverflow.com/questions/27775233/lambdas-and-capture-by- reference-local-variables-accessing-after-the-scope)

<comment><USER10> I certainly don't mean that there aren't contexts where distinguishing the two is vital to understanding. I just mean there are context where you'll need to talk about both, and I think that's part of the reason they sometimes get conflated.

</comment></comment></comment></comment></comment></comment><comment><USER25> In Midori, objects are capabilities. This is possible because type safety is enforced and an object's type cannot be forged.

It's easy to imagine a capabilities system built around closures. For example, a capability to send on a socket could be an unforgeable closure that performs that sending. Are there any systems that work this way?

<comment><USER26> > objects are capabilities

What does 'a capability' mean? People seem to use the term to mean everything from objects to threads.

<comment><USER27> In this context, it means the object capability model.

[https://en.wikipedia.org/wiki/Object- capability](https://en.wikipedia.org/wiki/Object-capability)

Which can be used as the foundation of security.

[https://en.wikipedia.org/wiki/Capability- based_security](https://en.wikipedia.org/wiki/Capability-based_security)

Relevant application in Midori:

[http://joeduffyblog.com/2015/11/10/objects-as-secure- capabil...](http://joeduffyblog.com/2015/11/10/objects-as-secure- capabilities/)

</comment></comment></comment><comment><USER28> Since objects are just specialized applications of closures, it's not too surprising that you can reconstruct the basic idea (closure) from its application (object).

</comment><comment><USER29> I'm curious what's the history behind the use of Zen koans in computing? There seems to be a lot of that.

Maybe because in programming there, often, isn't a clear right or wrong (like in this case) so it pays to structure the problem/solution as a story to make people "feel" rather than reason.

<comment><USER30> If I had to guess, I'd say it all started with GEB by Douglas Hofstadter. It very well explains aspects of symbol manipulation using koans. However, there might be another precursor.

<comment><USER10> ESR attributes the invention of the "AI koan" to Danny Hillis "while a student at MIT". Hillis graduated in 1978. GEB was published in 1979.

[https://en.wikipedia.org/wiki/Hacker_koan](https://en.wikipedia.org/wiki/Hacker_koan) [https://en.wikipedia.org/wiki/Danny_Hillis](https://en.wikipedia.org/wiki/Danny_Hillis) [https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach](https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach)

</comment></comment></comment><comment><USER31> Aren't they both just bundle of function objects + states?

</comment><comment><USER32> Closure is more fundamental though. Add a structure (define an ADT) to a closure and you will have an object.

The classic example is a closure with message-passing, which was the original concept in Smalltalk.

<comment><USER32> Well, it seems that having a structure is the most fundamental principle. While in reality structures formed and maintained according to the laws of physics, in CS we have to explicitly define the rules, which is what Abstract Data Types (which define Interfaces) are. Protocols and Interfaces are mere generalizations of ADTs.

There are, however, at least some "laws of the universe" in CS. These are Interrupts (the _only_ way to do I/O right), specialized Syscalls, etc. One should respect them.

</comment></comment><comment><USER33> One word: inversion of control.

</comment><comment><USER34> (2003) My favorite thing about this thread is that Guy Steele is involved, and Guy Steele has a far better name than anybody else has ever had.

<comment><USER35> I did work with a guy named Max Kalashnikov at Google. That was a pretty "rad" name, too.

<comment><USER34> I've also known a "Ulises Cervantes" and an "Aristotle Socrates," since we're trading awesome names :) Guy Steele is still my favorite, because I can just picture a mustachioed villain shouting "curse you Guy Steele!"

<comment><USER35> The visual here is awesome. Although all good programmers are antiheroes in my world, so I read that as a mustache twirling Guy Steele merrily evading the bumbling cops. Practically the whole point of Real Programming is to evade the limits of The Man. (Cf Mel: [http://www.catb.org/jargon/html/story-of- mel.html](http://www.catb.org/jargon/html/story-of-mel.html) )

</comment></comment></comment></comment><comment><USER36> Scala programmers are laughing from the side lines.

<comment><USER37> As Scala didn't yet exist when this message was written, its author was missing the enlightenment Scala provides.

Would you please explain what about this amuses Scala programmers?

<comment><USER38> Scala is largely a set of iterative improvements with a syntax that's designed to handle data workflows well. It's not a big deal, it's not that exciting, it doesn't use all of the latest technologies, and that's why I think it has a bright future.

<comment><USER37> This doesn't address closures vs objects at all.</comment></comment></comment></comment>


The Loss of Skill in the Industrial Revolution - <AUTHOR> http://growthecon.wordpress.com/2014/08/19/the-loss-of-skill-in-the-industrial-revolution/ <comment><USER2> One problem with deskilling is loss of bargaining power. Because you become a cog in the machine, easily replaced. Good if you own/rent people, sucks if you're the "human capital".

(Companies often reject profit-improving innovations which empower skilled workers. On the flipside, unions — to the extent they exist — also have the incentive to reject improvements which damage bargaining power. That's one problem with capitalism's built in boss/worker antagonism.)

Another is mind-numbing work. Adam Smith rants about how division of labor makes people _" stupid and ignorant as it is possible for a human creature to become... But in every improved and civilised society this is the state into which the labouring poor, that is, the great body of the people, must necessarily fall, unless government takes some pains to prevent it."_ ([http://www.econlib.org/library/Smith/smWN20.html#V.1.178](http://www.econlib.org/library/Smith/smWN20.html#V.1.178))

Another (since the last century) is the rise of managerialism, with its bureaucracies. David Noble points out that tech can deskill workers and strengthen management, or empower workers and peel away management layers.

<comment><USER3> Do you have any examples of companies rejecting profit-improving innovations which empower skilled workers?

<comment><USER4> I was told last week not to create any macros whatsoever. Everything must be either completely manual, or outsourced to India. I work in one of the largest financial institutions in the world.

<comment><USER5> What the hell? I belive this might just be gross incompetence and not a deliberate manouver.

<comment><USER6> As a libertarian, my personal philosophy leads me to be suspicious of _all_ large organizations, and one of the reasons why is that once you become large enough to stop being rigidly held to a standard such as the market's demand for efficiency (there are other possible standards, but that's a popular one; small local governing organizations can also be effectively held to account by their constituents, for instance), you become free to do things like start playing turf games internally without regard to whether it impacts the bottom line. Government, company, non-profit, club, union, NGO, sports organization like the MLB, doesn't matter, as soon as you are free to engage in human politics without restraint you get too many people who begin playing games that destroy societal value for their own local political gains. In this case, doing everything manually is a net loss for society, and a net loss to the company in question, but a local gain for the person making that dictate (or at least perceived to be a local gain, which is good enough), and so the trade is made.

</comment></comment></comment></comment></comment><comment><USER7> "If we were talking about innovations that got more output from less energy, then holding output constant while lowering energy consumption would be what everyone hoped to see. Why should human capital be different?"

Wow. I'm not a Marxist, but perhaps _this person_ may want to start by reading Marx' view about de-humanization in industrial production.

<comment><USER8> Well, to the dismay of people who like throwing labels like "marxist" around, Karl Marx was right about many things (also wrong about many, like every human), and his ideas seem to be more applicable nowadays than they were in his time.

<comment><USER6> Marx identified some problems. This is fairly underwhelming... one need merely step outside, point, and one can probably find a problem. Stepping outside is optional, in fact.

Marx was _mindblowingly wrong_ about the solutions. This is also, frankly, underwhelming... solutions are _really hard_. He got unlucky though... people took him seriously, and a lot of people died as a result. Whoops. May we all never have the curse of being taken serious at such a scale given to us.

<comment><USER9> First, I've never seen Marxism implemented in the real world. At the time of its writing, only the UK, Germany and perhaps Holland fit the required industrialization level. Attempting to implement it in an agrarian society was doomed from the being by the theory. Now I know that many (myself included) take issue with what we now call a living wage, but so far it is the only humane response to dealing with poor people and their growing population due to industrialization. A living wage is essentially Marxism.

Think about the present situation. Technology is making more and more jobs unnecessary. Soon trucking will probably go the way of the buggy whip. That entire industry will shrink. Capitalist will own the trucks, and pay a select few to manage a fleet. The drivers and some of the mechanics are no longer needed to maintain routes. That's a lost of many good jobs.

Now, not to disrespect every trucker, but many are average intelligence or below average. If jobs that survive technology require above average intelligence, the aforementioned truckers won't have a job, even with retraining. Now what should we do with them? Pure capitalism would either have the poor die of starvation/exposure, private charities, poor jails or extermination. Private charities haven't scaled in the past. Poor jails didn't work either. I don't see anyone killing the poor either. I also don't think you'll get mass extinction in a modern society. Thus you're pretty much left with a Marxist endgame. Eventually enough of the means of production is owned by the government, either directly or taxation on the producers, that you have a Marxist state.

<comment><USER10> "Now, not to disrespect every trucker, but many are average intelligence or below average" You underestimate the adaptability of your fellow human. There may be truckers that performed less well academically than you, but perhaps they didn't need to perform academically in order to survive, and preferred a different route. Give me anyone from any background with motivation to learn the skills I can share and I'd guarantee I could get them to a decent level.

<comment><USER8> The thing is, the goalpost is moving. First of all, not everyone has enough of "motivation to learn" (along with things like time; the general case is, the poorer you are, the less time you have available to learn; this also applies to unstable situations like losing a job). Secondly, the "decent level" is the moving goalpost. At some point "decent level" is not decent enough to get the bread.

<comment><USER10> My main point is "motivation to learn" is key. Give me someone with motivation to learn, even if their free time is limited, and I can get them to a decent level in the fields I have knowledge of. As for what "decent level" means, to me in this case it means good enough to be employed based on those skills based on current requirements, and with all the mental tools to further their own education in the future.

Perhaps it's because I recognise my own aptitude is very little to do with being "intelligent" (whatever that means), and much more to do with having an interest in what I do.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> Ayn Rand would love the questions near the end.

Although most markets have historically had a high concentration of output from the top percentile, so its not exactly a far fetched proposition to say that 5% of the workforce led development during the IR.

<comment><USER6> If the vast bulk of technological output amplifications are additive, then we'd expect to need broad distribution of the skills and experience a broad distribution of results.

If even a few technological output amplifications are multiplicative, and especially if they are independent (one can freely choose the 3x and the 4x and the 5x advantage and obtain something like a 60x advantage), then we would expect the skills to end up concentrated and the optimal strategy to be to load up as many multiplicative advantages as possible in one place.

(Note carefully the first few words of each paragraph. It only takes a handful of multiplicative advantages for them to dominate.)

You can obtain this result by playing lots of strategy video games, especially Civilization which has both. Additive results like the simple Granary, which adds a constant amount of resources to the containing town, need to be built in every city to be effective. But things that have a multiplicative advantage need only be built where they are necessary, and if you have certain unique things that can only be built in one place that are also multiplicative, the optimal strategy is to build them all in one place so they reinforce each other, not scatter them about.

So far, none of this has a moral dimension. This is all just simple optimization. Now we bring it back down to reality, where morality intrudes. It should be so obvious that a great deal of technology in the real world has a multiplicative effect rather than an additive effect that I should not have to justify that statement. IMHO, the question is what to do about this rather than whether this is true. And one of the dangers is that it is _very_ easy to end up giving answers that actually destroy the technological advantage in the process, because for as mighty as it all appears to be, it's a lot more fragile than it looks, as you can see once you start seeing it this way. The humans involved in the tech aren't irreplaceable (or it would be _really_ fragile), but there's still limits involved, and destroying or impeding their effectiveness can have non-linear impacts on total productivity. A lot of people model the technological advantages we've obtained for ourselves as a constant; we can do whatever we want and it will always be there, no matter how we impede or help it. It's scary how untrue that is.

</comment></comment><comment><USER12> Could also be phrased "the rise of under-employment" rather than loss of skill. Hundreds of years ago my direct ancestors were making coo coo clocks in the black forest, and on paper that carpentry skillset is completely lost after a couple generations of job titles, although I am actually a modestly skilled wood butcher, I am beyond your average handyman or roofer or maybe even average rougher, but I am good enough to know I'm at least one step, maybe two, beneath the true masters of the craft. I might be a higher skilled carpenter than some of my less competent ancestors, despite it merely being a hobby.

As a close to the heart analogy, everyone here knows that if you graduated with a BSCS and didn't do the IT/accounting or the graphics arts/web design track then the student probably did the stereotypical academic track with all manner of highly skilled senior year classes like automata theory, compiler design, maybe some control theory (although thats more EE). I did well in those classes and like many (most?) people I'm highly underemployed. I would guess that well over half, maybe 90 percent, of my fellow students in automata class and compiler class are just doing CRUD web apps or mobile apps, which hardly require those skill levels / skill sets.

I'd be slightly interested in sociological commentary on societies where underemployment increases. Does it always increase infinitely, or crash after awhile, or just not matter much?

A better proxy for carpentry skill level of a society might be the total sales of tools and supplies. I think the total economic size of the "at least somewhat skilled woodworker" is larger today than in the olden days.

Another interesting aspect is expansion of titles. Everyone in a skilled craft no matter if its programming or carpentry knows some are more equal that others, in carpentry no matter if you all have the same job title, or hobby name, some guys can barely be trusted with material handling and rough carcasses while other guys can be trusted to trim the finest kitchen cabinets, despite all having the same title. And obvious IT/CS analogies.

</comment><comment><USER13> This is particularly interesting in combination with Gregory Clark's ("The Son Also Rises: Surnames and the History of Social Mobility") research into social mobility and relative fecundity by social strata.

</comment><comment><USER14> I think the modern wave of industrialization (post invention of the micro- computer) is starting to reverse the trend, especially as leisure time is potentially increasing.</comment>


O'Reilly Radar:  Startup Centers - <AUTHOR> http://radar.oreilly.com/archives/2006/06/startup_centers.html <comment><USER2> I'd love to see a mashup of O'Reilly's list of cities with the locations of funding applications sent to VCs. You could throw in another one with all the YC/Techstars applicants.

<comment><AUTHOR> Good idea - that would be really interesting to see.</comment></comment>


How I found the home address of the culprit who put a RaspberryPi in our Network - <AUTHOR> https://blog.haschek.at/2019/the-curious-case-of-the-RasPi-in-our-network.html <comment><USER2> Original discussion from 11 months ago: [https://news.ycombinator.com/item?id=18919129](https://news.ycombinator.com/item?id=18919129)

</comment><comment><USER3> I remember when this happened. I'm curious what happened since.

<comment><AUTHOR> not much I'm afraid. Or they didn't tell me. Company doesn't want to pull attention to it so they are hushing it up</comment></comment>


40% of college students are enrolled in places with fewer than 1,000 people - <AUTHOR> http://www.economist.com/news/united-states/21711929-about-40-college-students-are-enrolled-places-fewer-1000-people-some <comment><USER2> I feel like someone smarter/less headache-y than my current self would be able to approach that statistic with Jensen's inequality</comment>


Lomuto's Comeback - <AUTHOR> https://dlang.org/blog/2020/05/14/lomutos-comeback/ <comment><USER2> There's an even better branch-free (super scalar) sorting algorithm: "In-place Parallel Super Scalar Samplesort (IPS4o)" which we started using:

[https://github.com/SaschaWitt/ips4o](https://github.com/SaschaWitt/ips4o)

[https://arxiv.org/abs/1705.02257](https://arxiv.org/abs/1705.02257)

As an example, to sort 10 million random longs on my computer it takes std::sort 766 ms (roughly in line with Andrei's numbers) and ips4o::sort takes 274 ms.

[edit:formatting]

<comment><USER3> You might want to mention if are affiliated with the algorithm you are promoting. ipsofacto ~ ips4o

<comment><USER4> Or maybe the throwaway was named after the comment it was intended to be used to post.

<comment><USER2> No affiliation, just picked the name because it fit... The company I work at was exploring various sort algorithms to use and this turned out to be the fastest.

Another benefit of this algorithm is that it is parallizes extremely well.

</comment></comment></comment></comment><comment><USER5> Hoare partitioning can also be implemented in a branchless manner, my algorithm pdqsort does this: [https://github.com/orlp/pdqsort](https://github.com/orlp/pdqsort)

<comment><USER6> Thanks! I tried that too, but it's slower than Lomuto. Forgot to mention in the article.

<comment><USER5> > I tried that too, but it's slower than Lomuto.

That is very strange. I can not reproduce your results in C++, using your code.

On my machine (Threadripper 2950x, 64 bit Windows 10, GCC 10.1.0 from MSYS2 MinGW64) my algorithm performs best and your branchless version ends up slower than your branchy one:

        $ g++ -std=c++17 -O3 -DNDEBUG -DLOMUTO lomuto.cpp && a 10000000
        min_milliseconds=828.1250
        median_milliseconds=890.6250

        $ g++ -std=c++17 -O3 -DNDEBUG -DLOMUTO_BRANCHY lomuto.cpp && a 10000000
        min_milliseconds=671.8750
        median_milliseconds=671.8750

        $ g++ -std=c++17 -O3 -DNDEBUG -DPDQSORT lomuto.cpp && a 10000000
        min_milliseconds=343.7500
        median_milliseconds=343.7500

On an Intel university machine (Xeon E5-2667 v2 @ 3.3GHz, 64 bit Ubuntu 16.04 LTS, GCC 5.4.0) I get:

        $ g++ -std=c++17 -O3 -DNDEBUG -DLOMUTO lomuto.cpp && ./a.out 10000000
        min_milliseconds=514.8051
        median_milliseconds=536.1984

        $ g++ -std=c++17 -O3 -DNDEBUG -DLOMUTO_BRANCHY lomuto.cpp && ./a.out 10000000
        min_milliseconds=688.2471
        median_milliseconds=698.9603

        $ g++ -std=c++17 -O3 -DNDEBUG -DPDQSORT lomuto.cpp && ./a.out 10000000
        min_milliseconds=340.1935
        median_milliseconds=344.7432

Maybe AMD or Windows doesn't like your branchless code or perhaps GCC is generating inferior code for AMD/Windows, as at least on Intel/Linux your Lomuto branchless code can beat the branchy code. But pdqsort (which uses branchless Hoare partitioning) consistently beats both.

I didn't change the benchmark, I only added pdqsort to yours, my full changes are the simple inclusion of 1 header and 3 lines of code: [https://github.com/orlp/lomuto/commit/29381176f49e3588f6882c...](https://github.com/orlp/lomuto/commit/29381176f49e3588f6882ca992cbb04f0c19c5f5)

In order for any interested readers to reproduce, simply clone [https://github.com/orlp/lomuto](https://github.com/orlp/lomuto) and run the above commands.

</comment></comment></comment><comment><USER7> Notice that this is on _random longs_. Of _course_ branch misprediction and memory bandwidth is going to crush you for a branchy sort... you'll be wrong like half the time, and the comparisons and swapping are trivial! Real world data isn't random, so I'd expect branch predictors to do much better with recognizing patterns. And your sorting isn't going to be as simple as sorting integers according to their values all that often. It's a neat exercise, but absent more realistic benchmarking to the contrary, I wouldn't assume I'll get a faster program by just substituting even a typical quicksort with this...

<comment><USER8> Sorting is the case study but not the insight. I think the cool/surprising part of the article is that doing more work can be faster... Even more memory work (which per conventional wisdom is not trivial!)

<comment><USER7> My comment wasn't inherently about sorting either. Branch-free in general is faster in the _worst_ cases, i.e. if the branch is difficult to predict. Except most real-world branches are predictable... that's why we have branch predictors. So in general you should expect branch-free to be slower, unless you have reason to assume your branches are actually close to random.

<comment><USER9> > unless you have reason to assume your branches are actually close to random.

I mean, to the degree that you need to sort the data at all, it contains informational entropy.

Many "presents as sorted" data structures trade space for time by keeping track of the internal sortedness of chunks of the data, between sortation passes. Heck, any insert-optimized data structure (B+ trees; LevelDB's sorted- string-tables; N-ary tries generally) will get most of its performance gains by being able to guarantee, at node split time, that the children of any particular node are sorted relative to one-another.

So, in many cases, by the very fact that you don't already have metadata attached to your data asserting that it's sorted, it's highly likely to have no branch-predictability.

(Couple that to the fact that load-balancing in OLTP distributed systems favors writes to evenly-keyspace-distributed partitioning keys, ala Dynamo, and you'll frequently find that your inputs to an index-like data structure can be _guaranteed random_.)

<comment><USER7> I'm struggling to follow your comment. I said real-world data have patterns and aren't completely random, so you can in general expect branch predictors to help (unless, obviously, your data is actually known to be random like in the example). You rebutted with "because load balancers evenly spread their key space, you will frequently find that your inputs can be guaranteed random", as if that somehow contradicts the point I was making...? It's also clearly not true that it's "highly likely" your data has "no branch- predictability" just because "you don't already have metadata attached to it asserting it's sorted"... right? That's not only both obviously not the case on its own, but also, why would you even sort data that's already guaranteed to be sorted? I can't make sense of this, so I feel like we must be speaking past each other somehow? But I'm confused how.

<comment><USER9> The insight I might have not made clear, is that if you have "this node and its children are already sorted" metadata, then it's cheaper to check whether a given (fine-grained) input chunk is already sorted—and if so, to directly construct a node from it—than it is to feed it blindly to the sorting algorithm.

If such a pre-check is in play, then the sorting algorithm itself will basically never have the sort of "runs" of sorted values that make some algorithms cheaper. Those "runs" were already plucked out and turned into nodes!

</comment></comment></comment></comment></comment></comment><comment><USER10> I used Lomuto's algorithm in my presentation of Quicksort in [http://canonical.org/~kragen/sw/dev3/paperalgo#addtoc_21](http://canonical.org/~kragen/sw/dev3/paperalgo#addtoc_21) because my emphasis there is on using the simplest possible algorithms. It didn't occur to me that it might actually be faster, or be capable of being made faster, and indeed in that page I said you could improve efficiency with Hoare partitioning.

Alexandrescu's branch-free code in this article is a thing of beauty, in the same sort of rugged way that Stepanov's code is.

</comment><comment><USER11> I love this sort of "Historical Fiction" where interactions and conversations between The Great People of Science are imagined and portrayed. A great example of this is Louisa Gilder's wonderful book _The_Age_Of_Entanglement:_When_Quantum_Physics_was_Reborn

[https://www.amazon.com/Age-Entanglement-Quantum-Physics- Rebo...](https://www.amazon.com/Age-Entanglement-Quantum-Physics- Reborn/dp/1400095263)

<comment><USER12> Oh, I assumed that dialogue between Lomuto and Hoare was real -- at least the gist of it, if not the exact words.

<comment><USER11> The interaction was probably real, since there is a date and location, but the dialog (and certainly the narration) seems to be taking some artistic liberties perhaps? This is also what Louisa did in her book on the history of quantum mechanics; real interactions with imagined dialogs. My apologies if I gave the wrong impression.

</comment></comment></comment><comment><USER13> To anyone interested in a deeper treatment of quicksort and variants thereof, I can recommend Sebastian Wild's PhD thesis on that topic: [https://kluedo.ub.uni- kl.de/frontdoor/deliver/index/docId/44...](https://kluedo.ub.uni- kl.de/frontdoor/deliver/index/docId/4468/file/wild-dissertation.pdf)

<comment><USER14> Off topic: Is it common practice for a dissertation at a German university to be written in English?

<comment><USER15> Yes - to ensure the widest possible audience. English is the lingua-franca of academia and has been for almost a century now.

<comment><USER13> At least in MINT fields, probably less so in the humanities. Correct me if I'm wrong.

<comment><USER16> You're right. German economics is famously insular, for example. Part of that is cultural, part of it is that they don't write as much in English.

</comment></comment></comment></comment></comment><comment><USER17> I've read that some vector instruction sets use masking instructions to avoid branching. You execute both arms of the conditional and discard the unwanted computation, which can be cheaper than an X% chance of a branch misprediction.

Should CPUs include more masking instructions for regular, non-vectorized code as well?

<comment><USER18> 32-bit ARM assembly language provided conditional execution for most instructions. This was dropped in the 64-bit instruction set. ([https://en.wikipedia.org/wiki/Predication_(computer_architec...](https://en.wikipedia.org/wiki/Predication_\(computer_architecture\)), [https://en.wikipedia.org/wiki/ARM_architecture#64-bit](https://en.wikipedia.org/wiki/ARM_architecture#64-bit)) I imagine the architects had a clear picture of the advantages and disadvantages, and made a very well-informed decision. I guess that part of the reason might have been that it's not clear when compilers should emit predicated code instead of branches.

Also, you can get something very similar even for scalar code as long as your machine has a conditional move operation, which they all have:

        /* true path */
        a = ...
        b = ...
        true_result = ...
        /* false path */
        x = ...
        y = ...
        false_result = ...
        /* final result */
        result = (condition ? true_result : false_result)

This works if the two "branches" have no side effects (memory writes, function calls) and cannot raise exceptions. Again, it's very difficult to estimate for compilers (and humans) if it will pay off to run both computations rather than branch.

The trade-offs for vectorization are different from scalar code since vectorizing a loop by a factor N gives a huge win, and even if you waste some time doing some redundant computations, you still have a reasonable chance of being faster than scalar.

<comment><USER19> You can get close to the performance of a cmov instruction by generating a pair of all-1s and all-0s values: a=c, b=c-1, and a result z=a&x|b&y.

Gcc will not under any circumstances produce two cmov instructions in a basic block, so that is your only alternative without dropping to asm. Clang is happy to produce two adjacent cmov instructions. Usually your ALUs are not otherwise so engaged as to make the number of operations involved costly.

<comment><USER18> I don't get this. If c can be 0 or 1, then a can be 0 or 1, so it can be all-0s but not all-1s. b can be -1 or 0, so that part is fine. Can you show actual C or C++ code for a function like this:

        int select(int x, int y, bool condition) {
            ...
        }

<comment><USER19> Sorry, brain freeze. See

[https://news.ycombinator.com/item?id=23237663](https://news.ycombinator.com/item?id=23237663)

The bitwise thing is correctly -c&x|(c-1)&y.

<comment><USER18> Oh, yes, I should have realized that -c is enough to transform from 0/1 to 0/-1. Thanks!

</comment></comment></comment></comment></comment></comment><comment><USER19> If you define

      inline bool swap_if(
        bool c, long& a, long& b)
      {
        long ta = a, tb = b;
        a = c ? tb : ta;
        b = c ? ta : tb;
        return c;
      }

and then use it in a partition like

       right += swap_if(
         *left < pivot,
         *left, *right);

compiled with Clang (which generates `cmov` instructions), the Hoare partition is still faster, and more than twice as fast as `std::sort`.

Using `swap_if` in Lumuto is also faster, but not as much faster. I interpret the difference (vs. array ops) as resulting from reduced L1 bus traffic.

A fully general swap_if,

      template <typename T>
      bool swap_if(
        bool c, T& a, T& b)
      {
        T v[2] = { a, b };
        b = v[c], a = v[1-c];
        return c;
      }

could and IMHO should be peephole-optimized to use a pair of `cmov` instructions, but is not in Clang, Gcc, Icc, or MSVC. But even without such an optimization, it makes Quicksort much faster.

(Gcc, incidentally, is very, very sensitive to details of the second swap_if. Change the order of assigning a and b, or use `!c` in place of `1-c`, and it gets much slower, on Intel, for very non-(to me-)obvious reasons. Gcc also will never, ever produce two `cmov` instructions in a basic block. I have filed a bug.)

If `swap_if` were in the Standard Library, it would probably be implemented optimally on all compilers, and almost half of the Standard algorithms could use it to get, often, ~2x performance.

</comment><comment><USER20> Writing branch-free code (which I find myself sometimes doing to take advantage of simd vectorization) is always really painful. I wonder if someone has made an attempt at better tooling for this.

<comment><USER21> One thing people have made is array languages. The "primitive" operations apply to arrays of data, so branches around individual elements can't be expressed.

I think it's also generally less expressive, though. Writing Lomuto's partition as efficiently would likely be impossible, but writing it branch- free might not be -- you would compose branch-free primitives like

        left = filter(array, array < array[0])
        right = filter(array, array > array[0])

where `filter` is branch-free and takes an array of Boolean values as its second argument.

Shrugs, aside from some standard set of functions/primitives and idioms I'm not sure it would help much, though perhaps a mindset-shift is a more likely solution than something more technical.

<comment><AUTHOR> This sort of vectorised pseudo branching is often used in SIMD algorithms

<comment><USER20> Yes... and the transformations are often non-trivial (especially when considering the possibility of out-of-bounds elements, which can't just safely be multiplied by zero--they might be a NaN, or on the edge of a page). It would be nice if there was a tool (or compiler option!) that could convert my branchy code into an "identical" (assuming floating-point associativity, etc.) branchless version.

<comment><USER22> The Eigen Array [1] object is fairly close to this. Combining the 'select' with comparisons, chained operators etc. Eigen vectorized stuff pretty effectively too.

1\. [https://eigen.tuxfamily.org/dox/group__TutorialArrayClass.ht...](https://eigen.tuxfamily.org/dox/group__TutorialArrayClass.html)

<comment><USER20> Interesting... I've used Eigen for matrix operations before, I did not know they had such a smart general-purpose array implementation!

</comment></comment></comment></comment></comment></comment><comment><USER23> Reminds me of this great article on writing a fast mergesort:

[http://pvk.ca/Blog/2012/08/13/engineering-a-list-merge- sort/](http://pvk.ca/Blog/2012/08/13/engineering-a-list-merge-sort/)

</comment><comment><USER24> If you have a branch free algorithm, you can vectorize it. If he discussed implementing this with SIMD instructions (whether by human hand or compiler auto vectorization) I missed that part. But that seems an interesting angle to me.

<comment><USER25> Not necessarily. A branch-free algorithm can totally have a data dependency from the previous iteration of the loop to the next, making it impossible to vectorize.

<comment><USER24> That's true, although there are sometimes workarounds for that by modifying the algorithm to iterate over a small groups where there is still a data- dependency between iterations, but not within each group, allowing one to use SIMD. e.g. instead of delta compression over an array of integers where you subtract each integer from the prior one, you can subtract each group from the prior one with only a small loss of compression ratio.

The question is, is there such a dependency in this algorithm? I didn't check.

<comment><USER26> Yes, sorting typically (and this one specifically) has long dependency chains. The natural reflex is to split the chains across multiple workers... and that's how you get merge-sort. But even the last pass has a linear dependence chain. Not long ago, somebody posted a clever algorithm based on 4-way swaps, which is a clever way of addressing the issue.

Edit: I can't seem to locate that 4-way swapping algorithm, I swear it was posted to HN in the last year... somebody halp!

<comment><USER25> You might be thinking of this: [https://news.ycombinator.com/item?id=22322967](https://news.ycombinator.com/item?id=22322967)

<comment><USER26> Bingo, thanks!

</comment></comment></comment></comment></comment></comment><comment><USER27> Reminds me of this paper on binary search, which also shows a speed up using branch-free comparisons.

"Array Layouts for Comparison-Based Searching" [https://arxiv.org/pdf/1509.05053.pdf](https://arxiv.org/pdf/1509.05053.pdf)

</comment><comment><USER28> So what's the impact of the data dependency chain on "first -= smaller" to the next iteration?

(I guess at the very least it prevents vectorization, but other than that ... shouldn't be too much, but probably still the new limiting factor?)

</comment><comment><USER29> [https://dlang.org/blog/2020/05/14/lomutos- comeback/](https://dlang.org/blog/2020/05/14/lomutos-comeback/) gives a 404 error. And archive.org ( [http://web.archive.org/cdx/search/cdx?url=https://dlang.org/...](http://web.archive.org/cdx/search/cdx?url=https://dlang.org/blog/2020/05/14/lomutos- comeback/) ) is also getting nothing but 404s, so it's not a problem on my end. I'm not really sure what's wrong with this site, since other commenters seem not to have noticed a problem.

<comment><USER30> It's ok now, as far as I can see.

<comment><USER29> Actually it's it still giving a 404, but (after remembering and looking up the --content-on-error option to wget) it appears TFA is served just fine as if it were a 404 error page.

</comment></comment></comment><comment><USER19> It is true that more swaps may be cheaper than fewer swaps.

The operations to count, nowadays, are not swaps or comparisons, which are very cheap, but instead pipeline stalls, which are very, very expensive.

It is easy to better-than-double the speed of vanilla quicksort with a three- line change in the partition step.

<comment><USER19> Details

[https://news.ycombinator.com/item?id=23237663](https://news.ycombinator.com/item?id=23237663)

</comment></comment><comment><USER31> I'm not sure I've ever sorted an array or list. Rather I put things into sets or maps or built in index---I always wanted to access things later, or have an on-line data structure (which a sorted flat thing is not).

This whole field is a waste of time, and anachronism from when the master copy was paper/pre-computer and computers were just doing the analytics.

<comment><USER19> Putting things in a map or set is almost invariably slower, often much slower, than a smart sort. If performance doesn't matter, then go ahead. Most often sort performance doesn't matter, or anyway most of the time is spent elsewhere. People do use Python or Bash in production, without shame.

But some of us work on problems where performance of the sort does matter.

<comment><USER31> Sure, it is slower, but my point is not that I sort by doing `Map.toList . map.fromList` but that I _don 't immediately turn the map back to a list_. The sorting literature is optimizing a thing I don't need to to do.

<comment><USER4> Just because you're not using it directly doesn't mean that you're not benefiting from those optimization in those layers you are depending on.

<comment><USER31> I'm not using it directly or indirectly. The whole point of this sorting literature is a space-time tradeoff that's completely different from when you want an ordered map/index as the end result.</comment></comment></comment></comment></comment>


You're Moving Abroad If So-And-So Gets Elected? Good Luck - <AUTHOR> https://psmag.com/oh-you-re-moving-abroad-if-so-and-so-gets-elected-good-luck-4fa319a55be9 <comment><USER2> I find America's immigration to be similarly opaque and exclusionary, even with highly sought skills. I can understand why some people would break the law to enter a country, the legal methods are nearly impossible. And then, ironically, US citizens think they can easily get citizenship elsewhere, only to come face to face the difficulties that "illegals" have overcome.

Closed immigration policies lead to very negative effects, people who are easily exploited, have no incentive to invest in their surroundings (because its inherently a temporary place for them) and little incentive to become "legal" after the fact (due to taxes etc).

Its sad we cannot just be citizens of planet earth, instead of jailed inmates of small plots.

</comment><comment><USER3> > You need to be OK with paying taxes in two countries and to disclose all > aspects of your life."

Gotta love that US tax system.

If things go to shit I'm glad I can always bail back to Canada.</comment>


What does Unsplash cost to run? - <AUTHOR> https://crew.co/backstage/dispatch/what-does-unsplash-cost/ <comment><USER2> Previously:

      https://news.ycombinator.com/item?id=11522661
      https://news.ycombinator.com/item?id=11519085

</comment><comment><USER3>
      While Heroku charges a small premium over other hosting services

The "small premium" is not small at all.

<comment><USER4> The amount they're spending on heroku is relatively low compared to the cost of the 150TB of bandwidth their using from imgix.

<comment><USER3> I guess that's the next point then, if you're spending that much on traffic you should probably look at just buying dedicated bandwidth.

I'm sure it doesn't have the same awesome CDN latency and I'm also sure it doesn't matter one bit for multi megabyte images.

</comment></comment></comment><comment><USER5> When I've spoken to folks running infrastructure, for a lot of companies straightforward advice such as this always worked to cut costs: Move from Heroku/whatever other managed provider for your actual database/worker/webserver as costs will rise as you scale Start using AWS Reserved Instances and Spot Instances (or similar) for your workloads

Advice such as this has often saved > 30k/month hosting. The caveat is these were mostly SaaS companies dealing with data.

Interestingly enough in Unspash's case, their actual cost on Heroku/etc is fairly minimal as they don't seem to be doing much with the data. If they start building things that collect a lot more user data, run ads, do lots of stuff with it transforming user data, run analytics, etc., their heroku infrastructure will be a bottleneck. Right now, it's not.

</comment><comment><USER6> How do they generate profit?

<comment><AUTHOR> They have 3-5 feeder sites as free services that bring them traffic for crew.co. That's where they presumably make enough money to finance Unsplash, their office/café/etc.

</comment></comment><comment><USER7> well i still think it's a lot of money to run a site like this!

<comment><USER8> As a referral source for new business in your marketing budget it would be a very good investment even if it only generated $150000 a month in new business. And I have no doubt they are getting a heck of a lot more from it than that.</comment></comment>


Mentored by Silicon Valley Luminaries - <AUTHOR> https://medium.com/career-pathing/e8d49a3e4294

<comment><USER2> Thanks for posting the article. Enjoyed the past few months..! @Nivo0o0</comment>


EA's new 'Star Wars' game is so unpopular a developer got death threats - <AUTHOR> https://www.cnbc.com/2017/11/13/ea-star-wars-game-is-so-unpopular-the-developer-is-getting-threats.html <comment><AUTHOR> EA just posted the most downvoted comment in history of the universe, by orders of magnitude.

[https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b...](https://www.reddit.com/r/StarWarsBattlefront/comments/7cff0b/seriously_i_paid_80_to_have_vader_locked/dppum98/)

People are actually "hate guilding": paying real money for reddit gold in order to send "fuck you" messages to EA that they can't ignore or mute in a locked thread.

We may be witnessing the birth of a new business model.

EACommunityTeam -652k points 1 day ago [gold star] x67

The intent is to provide players with a sense of pride and accomplishment for unlocking different heroes.

As for cost, we selected initial values based upon data from the Open Beta and other adjustments made to milestone rewards before launch. Among other things, we're looking at average per-player credit earn rates on a daily basis, and we'll be making constant adjustments to ensure that players have challenges that are compelling, rewarding, and of course attainable via gameplay.

We appreciate the candid feedback, and the passion the community has put forth around the current topics here on Reddit, our forums and across numerous social media outlets.

Our team will continue to make changes and monitor community feedback and update everyone as soon and as often as we can.

<comment><USER2> HN discussion yesterday [https://news.ycombinator.com/item?id=15686220](https://news.ycombinator.com/item?id=15686220)

</comment></comment><comment><AUTHOR> Rumor has it that The Sims 5 will come with only pay toilets, and require an in-app purchase to kill a character.

</comment><comment><USER3> Hell they even removed the refund button from their portals.</comment>


Study: High-Skilled Immigrants Are Neither Better Nor Brighter Than U.S. Workers - <AUTHOR> http://techcrunch.com/2013/03/01/study-stem-immigrants-are-neither-better-nor-brighter-than-u-s-workers/

<comment><USER2> Personally, I think Matloff is rather biased in most of his studies. However, I will say this, I was on an H1B for 5 months or so of my life. I can agree with the perception that one feels like a indentured servant. There is always that worry in your mind's eye that one can get fired and would be kicked out of the country. My company went through layoffs two weeks before my green card interview and that was an insanely stressful experience because I was in the situation where if I had gotten laid off, I would probably have to leave the country. If I left the country, my green card application would have been deemed abandoned which means I would have to start over. If you want real immigration reform, make the H1B visa a portable work visa for 5 years. Of course, this doesn't serve the owners of companies. They would rather have the current situation that probably depresses wages but certainly leads to a situation where the employee can be exploited by unscrupulous managers.

<comment><USER3> Just curious... How long did it take to get the green card once you got the H1B visa?

<comment><USER4> IANAL.

Asking someone else is not useful to you, because there are several buckets into which one can fall, that have different waiting times.

As I understand it, there are two questions of timing: how long will an applicant have to wait until given an immigrant visa number, and how long will it take to turn the immigrant visa number into a permanent residency. The latter is, I believe, consistent across buckets but varies over time, depending on the wheels of bureaucracy. It's a year or two, I think? The former, however, depends on your level of training and country of birth.

The wikipedia page is pretty good, in particular a few sections:

[http://en.wikipedia.org/wiki/Permanent_residence_%28United_S...](http://en.wikipedia.org/wiki/Permanent_residence_%28United_States%29#Application_process)

[http://en.wikipedia.org/wiki/Permanent_residence_%28United_S...](http://en.wikipedia.org/wiki/Permanent_residence_%28United_States%29#Immigration_eligibility_and_quotas)

In the latter, note the difference between EB-3 and EB-2, and note the bolded footer (b).

EDIT STARTING HERE:

Oh, this helps too:

<http://www.travel.state.gov/visa/bulletin/bulletin_5885.html>

Depending on country of birth, if you have a BSc you're looking at 5-11 years of backlog. If you have a higher degree or 5+ years of "progressive" experience you're looking at 0 years of backlog, except 5 years for Chinese and 9 years for Indians. If you are doing family-based immigration, some category/nationality combinations are only just getting around to applications created in 15OCT98 (Philipinos and Mexicans are worst off here).

</comment></comment></comment><comment><USER5> They determine who belongs to the "best and brightest" by looking at wages.

An alternative hypothesis is that in an industry where getting a raise means switching jobs the H1B program is successful in suppressing wage levels of foreigners.

<comment><USER6> While determining one's skill by how much he/she gets paid sounds ridiculous, this makes sense if you consider macro trends, as this article is talking about.

If an industry (like finance) routinely pays multiple times the compensation of another industry (say, engineering) that requires similar skillset, it's rational that more of the top graduates will choose finance over engineering.

<comment><USER7> No it doesn't make much sense even if you consider macro trends. Bright Chinese or Indian workers will almost always be ready to work for lesser wages than a 'Native American' (if there's something like that) will be ready to work for. There's a macro economics involved here, but it's not what you think.

<comment><USER6> Trust me, in a country with reasonable immigration laws, like Canada, highly skilled foreign workers do not make less money than locals.

The biggest problem with H1B, which Matloff is critical of, is that it puts foreign worker and local worker in uneven legal grounds, thus unable to demand the same wage.

</comment></comment></comment></comment><comment><USER8> Can we talk about racism? If you're brown or yellow or black... chances are you don't make the company club (as easily) as well, white people... I've seen it with my own eyes...

If you're not well networked in a corporation, but still work very well, you __don't get a higher wage or as much opportunity to a promotion __..

non-white ethnicity is a huge network barrier. It's not like people wanna golf with you and then go have a fancy chicken tiki massala or eat injera.

Also, could foreign workers be much more ok with simple lives rather than being ambitious corporate thugs?.. but what's the measure of ambition for higher salary btwn immigrant and native wokers?

If I came from a village I'd settle for any salary. Using salaries for understanding is absolutely pathetic..

<comment><USER9> Until a few months ago I was on the 6th floor of an office building. There was my office, which was a diverse company (minority woman owned, lots of black, white, hispanic, Asian, etc). Then next to us was a .NET shop. All they do is .NET software for the government. ALL Indian/Pakistani. They had one white guy working for them, and he and I talked on the elevator. He said that the only reason he was hired was because they couldn't find an Indian guy that knew his particular specialty. 2 weeks later I saw him on the street corner, and they had fired him and replaced him with an Indian guy. We shared a floor with this company for years, and they wouldn't even talk to any of the non Urdu/Hindi speaking people in my company.

This is the part where you say that I don't have all the information, and maybe he was a terrible worker. That's also the part where I say that you likely don't have the full picture of the "company club" either. The cultural divide can be huge. My former coworker grew up in Punjab. He was very apprehensive when he tagged along with me on a roadtrip (he loved football) and my wife wanted to drive. He told me "the women in my family don't drive." This is the kind of stuff that makes it hard to get into the "company club."

<comment><USER2> Sadly enough, I have seen this sort of stuff happen. I have worked in teams which are overwhelmingly South Asian, there are definite situations where they find it hard to have casual conversations in English with American counterparts. They form self-selecting groups and there are conversations which are peppered with Indian languages. Now, this is in someways a reverse company club. I have no sympathy for people who then complain that they don't have enough visibility in the company and don't know how to get ahead in the game.

</comment></comment></comment><comment><USER10> They are cheaper. Many people are willing to work for cheaper if that means that a company will file for them to stay in the US. Companies exploit this.

</comment><comment><USER11> My graduate STEM classes did not have many US citizens. I bet it's the same in most universities. I don't think that H1B workers like myself are brighter or smarter than someone local, but the argument that technology companies are losing jobs to Wall Street as a validation criteria for his hypothesis is absurd. Just because Gates says that we need to keep the "best and the brightest" doesn't mean that we're all Nobel laureates in STEM. We have certain technical skills which cannot be fulfilled by Americans (due to lack of American STEM students in universities). That's all.

<comment><USER2> This is true for my graduate school experience. On the other hand, I have a rather controversial viewpoint: People trained for four years in an excellent undergrad program are way better than people who came from an incompetent undergrad program and have 2 years of a good grad program. Furthermore, the demographics for undergraduate STEM programs has a huge majority of US Citizens. So I am not super sure there are technical skills that graduate students have that cannot be fulfilled by undergraduate students. I think what has happened is that industry (Computer Science/EE) has become obsessed with people with graduate degrees. I work as a data scientist and most of what is done in this field could be done by a competent undergraduate student with some math, computer science.

<comment><USER11> But compare the number of undergraduate American students in STEM vs number of students in BBA/pre-med/law/dance/arts/humanities?

There in lies the problem.

<comment><USER12> The number of Americans in Comp Sci versus English doesn't matter. What matters is supply of Americans with BSc Comp Sci versus demand for Americans with BSc Comp Sci.

<comment><USER11> There is always a demand for software engineers/designers/programmers. According to recent reports there are 3 million open tech jobs in the US. Assuming a pessimistic 50% variation in that number, you still have 1.5 million jobs. The popular dislike for "geek/nerd" STEM subjects in American schools doesn't help. That's unfortunately a societal problem that needs to be overcomed ("I don't know math", "that's for nerds" etc. fine flip my burger then).

<comment><USER12> _According to recent reports there are 3 million open tech jobs in the US._

Really? Source?

 _("I don't know math", "that's for nerds" etc. fine flip my burger then)._

Not everyone can or should be a programmer.

<comment><USER11> Sure not everyone can or should become a programmer/software engineer. But you would expect a progressive forward thinking society like the United States, who has pioneered most of the technological advances in the 20th century to have a steady flow of STEM graduates, wouldn't you?

I believe there was 60 minutes episode for this. [http://www.bloomberg.com/news/2012-07-25/companies- say-3-mil...](http://www.bloomberg.com/news/2012-07-25/companies- say-3-million-unfilled-positions-in-skill-crisis-jobs.html)

This story includes all open jobs which require "high-tech skills", not necessarily being software engineers/programmers, and they don't have a breakdown by job function, unfortunately. Maybe I was misled by the over use of IT and high-tech that led me to believe they meant IT jobs only.

This snippet is from Obama's Google+ Hangout: \</comment></comment></comment></comment></comment></comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-- Perhaps the most notable exchange during the Hangout was when participant Jennifer Wedel of Texas, whose husband has had trouble finding permanent employment for the past three years, interrupted the president.

Obama had said he had a list of folks looking to hire engineers, her husband's specialty. "There's a huge demand for engineers around the country right now," he added, when Wedel jumped in.

"Um, I understand that, but how — given the list that you're getting, I mean we're not getting that — You said in the State of the Union address that business leaders should ask themselves what can they do to bring jobs to America," she said.

The president had a very personal response which Wedel seemed satisfied with.

"If you send me your husband's resume, I'd be interested to find out what's happening," Obama said. "But the word we're getting is somebody in that high tech field should be able to find something right away."

"I'll have to take you up on that," she replied. \</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-- Although he was getting calls from companies he couldn't move away from North Texas: [http://www.mcclatchydc.com/2012/04/09/144558/texas- engineer-...](http://www.mcclatchydc.com/2012/04/09/144558/texas-engineer-who- sent-obama.html)

And in typical brain-dead, southern conservative fashion his wife blames H1B's for his inability to get a job (I lived in Texas for over seven years, I know these things first hand).

</comment><comment><USER13>
        Instead, foreign immigrants have saturated the market,
        pushing science, technology, engineering, and mathematics
        students onto other career paths

This is contradictory. If immigrants are no better than american workers, and there is no shortage of the latter, why would immigrants be hired through complicated and costly processes in the first place?

<comment><USER9> Because once they are hired, they can be treated like modern day slaves. If they lose their job, they have to leave the country. This lack of labor force mobility makes them immediately cheaper. Why pay a competitive salary if you don't have to compete for this person?

<comment><USER14> Well, even if they are not treated that way, the bondage feels that way. The boldness while doing your job well goes away. We cannot ask for promotions since there is no point. I worked with a huge pay cut for years and today I question if my decision to come to US was a right one. I could have been in any of the other open developed countries instead and would have happily settled down.

</comment></comment></comment><comment><AUTHOR> If there was a real talent shortage and a real Visa need, then we would allow H1-B's to self sponsor. Let the company elicit one visa and employee, and have the Fed gov garnish 5-10k over the first year from the employees' wages. After that he/shes a free man/woman to look where they choose. Stay employed, stay out of trouble, and pay another 5-10k and get an additional 3 years. The visa is tied to the person, which is the way it should be. That's a semi compromise, but I still would keep the # capped at < 100k new per year. As it stands with 8% UE we had an increase of 110k in 2010 to 129k in 2011 , [http://en.wikipedia.org/wiki/H-1B_visa#H-1B_Applications_App...](http://en.wikipedia.org/wiki/H-1B_visa#H-1B_Applications_Approved).

</comment><comment><USER15> They don't have to be better or brighter to make everyone better off. If there are jobs and they fill them: they create wealth for the US economy. There's no fixed sized pie in the economy. They fill a job: the pie increases. Economics 101 for HN

<comment><USER16> The size of the pie isn't the only thing that matters. Consider two scenarios of GDP allocation:

Scenario A Shareholder: $100,000, Native programmer: $100,000

Scenario B Shareholder: $150,000, Native programmer: $75,000, H1B programmer: $75,000

There are good reasons in a democracy to favor scenario A over scenario B. It just depends on the following premises:

1) the income of non-Americans is irrelevant to American policy;

2) we care more about the income of the median American than the aggregate GDP;

3) we don't not care about the standard deviation of income

Econ 101 or not, these are not unreasonable premises.

<comment><USER17> How about factoring in the tax revenue from American enterprises into the equation?

<comment><USER16> Corporate taxes are less than 10% of U.S. revenue and falling. Not a particularly compelling reason to push for policies that increase corporate profits.

<comment><USER18> I think parent was talking about the increase in income and consumption taxes as a result of the new worker paying income tax, capital gains tax due to the higher returns to capital, and the increase in consumption all around (resulting in increased sales taxes). Corporate taxes are sort of irrelevant and are arguably double-taxation since corporate earnings will ultimately be subject to those other taxes.

<comment><USER16> Corporate taxes are not arguably double taxation. They just aren't double taxation, period.

Here is an easy way to figure out if something is double taxation: income that counts once for GDP but is taxed more than once. If I make $100,000 in NY as an Illinois resident, and both NY and Illinois tax me on that income without giving me credit for taxes paid in the other jurisdiction, that's double taxation. My income is only counted once in GDP, but is taxed more than once.

Corporate taxes are just the natural consequence of passing money through a distinct legal person (the corporation) in a system that taxes income to legal persons. Say there is a 10% income tax. I make $100,000. I'm taxed 10% on that, leaving $90,000. I give $50,000 to you. You're taxed 10% on that, leaving $45,000. Total income, for the purposes of GDP, is $150,000. Total taxes collected is $15,000, or 10% of GDP. No double taxation!

So why does anyone create corporate entities? To get the benefit of the corporation being a distinct legal person: limited liability. It's symmetric: the corporation is treated as a separate person for liability purposes, and is treated as a separate person for tax purposes.

</comment></comment></comment></comment></comment></comment><comment><USER19> wish someone would look at the human side of the story - you give so many years here, have kids who are citizens, you identify with the country and the culture, you came here legally, you worked hard, you are still waiting to call this home (at the current state with wait times upto 50 years)

<comment><USER20> Wait times for H1B visas are not 50 years. You can become a citizen after 8 years of having permanent residency.

<comment><USER2> 1\. Wait times for an Indian/Chinese born person under the EB3 quota are varied but the backlog as it stands now looks like it might hit 25+ years. I am not offhand sure about the 50 years number but have heard of it mentioned earlier.

2\. One becomes a citizen after 5 years of permanent residency. (3 if one is married to a US citizen.)

</comment></comment></comment><comment><USER7> Shit Article. Since they're earning less wages (or they're ready to work at lesser wages than U.S. workers) that means they are "neither brighter nor better"?

</comment><comment><USER21> This study appears to ignore non-wage compensation (e.g. stock grants). I know that for me, as a H1B worker who graduated from a US graduate program and working for one of the large companies named, that stock based compensation is a substantial fraction of my total pay.

</comment><comment><USER22> Bingo, study showed that because foreign workers are paid no less than us workers, it is unlikely they are discriminated against. That's a good start. Now it would be nice if they help address the shortage of _skilled_ workers (foreign or not) by reforming the visa system

</comment><comment><USER23> Yes, but they are "cheaper", which is all that matters, unfortunately.

<comment><USER7> Oh, cheaper, and many times better and brighter, _contrary_ to what the article says.

</comment></comment><comment><USER24> But are they better or brighter than low-skilled immigrants?

</comment><comment><USER25> The graph seems to be assuming that an immigrant becoming a citizen suddenly makes them America's "best and brightest" and are no longer included in the temporary workers' column. I don't seem them getting filtered out to make the comparison fairer.</comment>


Show HN: remote access to your Android phone with Shynk - <AUTHOR> https://shynk.com/

<comment><USER2> I really do love the idea behind these kinds of services - SMS from a desktop browser window? Awesome.

But I _really_ don't want to pass all of my personal traffic off to a rndom new startup's server. How can you convince people to use this and trust you? Does this kind of thing rely on the serial over sharers to just connect their Google account up, and willingly pass off their private messaging?

Maybe I'm being overly critical, but this is precisely the kind of thing I would want to run myself, not give to someone else to run.

 _edit_ It seems I jumped the gun a little - their FAQ says that they explicitly don't store the messages on their servers, making them a dumb pipe to just send data. Still, my instinct is to wonder who else can see my messages :/

<comment><USER3> _But I really don't want to pass all of my personal traffic off to a random new startup's server_

And rightly so. Just use EasySMS for the SMS thing (it runs entirely on your phone, no third party involved) and one of the six dozen Filesharing/Samba apps that run over Wifi _without_ going through a remote party (personally I'm using the samba app with the red icon, don't recall the name right now).

Sorry @shynk, but your app is broken by design and the proper solutions have been available in the app-store for years.

<comment><USER4> Another option for SMS from your computer is to use Google Voice. If your with Sprint you can link your number and call and text from your browser.

</comment></comment></comment><comment><USER5> How does this differ to something like Airdroid (www.airdroid.com)? It's not really clear to me and I can't run it at the moment.

Airdroid is almost perfect and the screenshots on the Play Store don't show your app in a good light.

<comment><AUTHOR> That is the most common question I see (it's been asked on reddit a lot, too). It is similar. One of the main differences is that Shynk works over mobile networks, not just over wifi. This lets Shynk provide other functionality, too. e.g. sharing files publicly and connection tunneling.

As I continue working on Shynk, you'll see their feature sets diverge a little more. I'd like to implement some more features that take advantage of the fact that Shynk uses the public internet.

As for the screenshots: they're not too beautiful, but the app itself is a pretty minimal part. After you log in, you should be able to forget about the app and just use the website's interface.

</comment></comment><comment><USER6> Nice job, seems to work well. I love airdroid but it annoys me that I am constantly having to reconnect after I walk out of wifi range. If this gets around that issue then I'd swap.

One comment I had is that if I was to download this app on my device, the initial experience could be improved somewhat. The first screen is a login one and it says to go to your website, sign up and come back to the app. I did this on my mobile browser (chrome) and it was a bit clunky to sign up (I had the feedback/need help button hover over the form a bit). Perhaps instead of that, you could have a registration function right in the app? even if it was just a web view with a mobile optimized signup web page loaded. That way I don't have to leave the app.

Not a major thing but thought I'd offer you some feedback.

<comment><AUTHOR> Thanks for the feedback!

It should get past the wifi range issue, as Shynk was designed with that in mind. It will automatically reconnect when it can (even if that means using 3G).

In-app registration is definitely something I've been planning on implementing, but it hadn't been asked about before, so I hadn't prioritized it.</comment></comment>


Side channel that leaked data from Intel CPUs patched by silent Windows update - <AUTHOR> https://arstechnica.com/information-technology/2019/08/silent-windows-update-patched-side-channel-that-leaked-data-from-intel-cpus/ <comment><USER2> I know these problems are serious, but I feel there is a fun and reassuring aspect to them. Back in the 90s and early 2000s we thought anything can be hacked by any clever person. Then the 2010s happened and almost all software, even Windows, became near unassailable fortresses, especially systems like iOS with their tight control of the OS. Sure, a 0day drops every now and then, but we all know that it's basically impossible to attack a modern well configured system.

These hardware bugs turn that idea on its head, suddenly the whole ghost in the shell hacker style dream is again a possibility. A motivated person or group of persons might go and hack any system out there. And that's really a bit reassuring, it's a little bit scary to think about how our lives might be ruled by these systems that are unassailable. I'd like to at least stand a chance when technology turns on us.

<comment><USER3> Keep in mind that all these speculative execution exploits rely on being able to already execute code on the target, they can only read data somewhat slowly, and where the data you want to read is in a 64-bit address space is not easy to find.

The media loves to blow things out of proportion, but all the specexec attacks are really not as big a deal as e.g. remote code execution. In some ways, they are the real-world equivalent of "you can sometimes hear things your neighbours say, with a sensitive microphone and lots of patience."

<comment><USER4> How lucky then that we all live in the cloud now and running code on other people's computers is the only place anybody runs code anymore.

<comment><USER2> In addition to that all sorts of vendors run their code on our systems to provide us with their features. And then there's the idea that some of these bugs have been demonstrated to be exploitable from within a JavaScript VM!

</comment></comment></comment></comment><comment><USER5> As mentioned in the article more technical details can be found in the linked article: [https://www.bitdefender.com/business/swapgs- attack.html](https://www.bitdefender.com/business/swapgs-attack.html)

Worth pointing out that Xen is not vulnerable to the swapgs attack due to a lucky design decision from a decade ago: [https://lists.xenproject.org/archives/html/xen- devel/2019-08...](https://lists.xenproject.org/archives/html/xen- devel/2019-08/msg00507.html)

</comment><comment><USER6> Even though it's supposedly not feasable to exploit on Linux, there are fixes in the upstream kernel now and the changelog entries have some technical details. If I'm understanding this correctly, AMD systems don't speculatively execute GS-based accesses after speculatively excuting SWAPGS like Intel ones do, but that isn't enough to fully fix the problem on Linux because there's a conditional branch over the SWAPGS instruction which can be speculatively executed. Since that execution path doesn't have a SWAPGS all processors will quite happily continue speculatively executing code that fetches data via GS.

<comment><USER7> What does speculative mean in this context?

<comment><USER8> CPU speculating that it knows what the state will be after SWAPGS and continuing to load instructions into pipeline and partially executing them. In case of either a different branch bring taken, or SWAPGS causing a fault the pipeline will be rolled back, but side effects of the partial execution (like loads into cache) will not be.

</comment></comment></comment><comment><USER9> It seems rather unfair to make the statement that "Microsoft silently patched the vulnerability during last month's update Tuesday" where the phrase "patched the vulnerability" links to Microsoft's CVE article on the problem.

<comment><USER10> I agree its unfair.

Microsoft also played ball with Linux vendors alerting them to this vector. This allowed them to get the swapGS fixes tested and sane.

This interaction with the opensource community has significantly increased my respect for Microsoft.

</comment></comment><comment><USER11> Whitepaper is behind a wall, so I can't read it, but I assume it is another in the large and growing class of explits that cannot practically be exploited like Spectre or the load-store buffer that didn't stand a snowballs chance in Hell or creating a workable exploit that didn't' require immense cooperation from the target (literally accessing the same address in a loop and and nothing else).

Still no spectre exploit (or even attempt) found in the wild. There should have been something by now.

If anybody has access to the POC code I would love to see it. Until (and probably after I see how laughable it is), I'll assume this is just a press release by some security company.

<comment><USER10> I believe that this (SWAPGS) vector is impractical but previous spectre proof of concepts exists if you look hard enough.

<comment><USER11> I have never seen one that would work in the wild. I don't think it is possible. We've had plenty of time to see something somewhere, but nothing. I gone through the papers, read other code, I've seen nothing that didn't require help from the target.

I don't accept the general hand waving "they keep getting better" because they haven't.

Most of these the exfiltration rate is so slow that simply xor'ing the secret with a random seed that changes locations or moving the memory around itself would prevent any attack.

<comment><USER5> Meltdown rate was quite a bit higher than the other exploits.

<comment><USER11> meltdown is different and clearly a microcode bug that is easily fixed. i don't include it with the other side-channel "attacks" that seem to come out every other week.

</comment></comment></comment></comment></comment><comment><USER12> So I have not yet installed the July 'security only' update because it contains W10 style telemetry (I'm on Windows 7 group B for the fellow AskWoody-ians) which I abhor even more than being vulnerable to a hack.

Any way to get the good without the reprehensible?

</comment><comment><USER13> Noone is concerned by the concept of "silent" Windows update?

</comment><comment><USER14> What's the performance hit on this one?

</comment><comment><USER15> What i understand is , you are already hacked when they perform this hack. They already have access to your system.</comment>


List of Hardware Startup Meetups around the USA - <AUTHOR> http://www.reddit.com/r/hwstartups/comments/13em7w/list_of_every_hardware_startup_meetup_i_know_of/

<comment><AUTHOR> Also of note, a while back I posted when I started r/HWStartups:

<http://news.ycombinator.com/item?id=4286840>

Now we've got around 600 members and rising traffic!

<http://www.reddit.com/r/hwstartups>

Would love to hear about anyone interested in starting a meetup in your town!

</comment><comment><USER2> You're right, Pittsburgh should be on this list!

<comment><AUTHOR> Dude - talk to Jim - he wants to do it! We did the 3D Cookout and it went really well!

</comment></comment><comment><USER3> Love these meetups!</comment>