Arguments in favor of pure functional programming - <AUTHOR> http://apocalisp.wordpress.com/2009/04/27/a-critique-of-impure-reason/

<comment><USER3> I am deeply suspicious of any attempt to define the next "one true way".

Perhaps it is because I have lived (and worked) through Structured Programming, Object Oriented Programming and now see Functional Programming promoting a (valid) bid to make a contribution to the general improvement in the standard of written programs - this time by attempting to eliminate the side effects of "state".

Each of the previous "one true ways" has made a welcome contribution but only a contribution.

We must remember that all such programming techniques are intended to help us, the poor imperfect programmer, write code with fewer bugs. The final code is always translated into a sequence of simple procedural instructions for the processor to execute - procedural programming is the foundation upon which all other edifices are constructed.

<comment><AUTHOR> I don't think the article promotes functional programming in favor of structured programming or object oriented programming. Rather it explains why _pure_ functional programming should be favored over 'impure' functional programming.

I think your point about 'us poor imperfect programmers' is well supported by the article, because it argues that purely functional programs are easier to understand. To quote the article:

 _This is why you have bugs, why programs are hard to debug, and software is difficult to extend. It's because programs are difficult to comprehend to the extent that they are impure. And the reason for that is that they are not sufficiently abstract. A writer of impure code is not operating on the level of abstraction of programming. He's operating at the level of abstraction of the machine._

<comment><USER4> > [Quoting the article.] It's because programs are difficult to comprehend to > the extent that they are impure.

That's wrong.

> And the reason for that is that they are not sufficiently abstract.

Abstraction has nothing to do with functional purity.

> A writer of impure code is not operating on the level of abstraction of > programming.

Nonsense.

> He's operating at the level of abstraction of the machine.

The real world has mutable state and all abstractions leak.

I write functional when I can and single-assignment (mechanically translateable into pure functional, but is often cleaner in procedural languages) almost all of the rest of the time, so I live on the functional Kool-Aid(TM).

The above is the sort of crap that makes it harder to get people to write functional.

<comment><USER5> Argumenting your "That's wrong" "Nonsense" and such interjections would help me understand the usefulness of your post

<comment><USER4> Functional purity is mostly unrelated to comprehensibility - <USER2> programs can be pure, or not, and the same goes for incomprehensible probgrams. The relevant background knowledge is some experience with programs.

If someone knows what both functional purity and abstraction are, they don't need to be told that they're different and one can have either one without the other.

"the abstraction of programming" is on the edge of bather and the part that isn't blather has nothing to do with impure programming.

And, if you don't understand that abstractions leak and that the real world (and machines) are stateful/not functional, you're in for it.

What part of that is hard to understand?

<comment><USER6> Pure functions are easier to think about than impure functions. It is easier to think about simpler things. Pure functions are simpler than impure functions because their result depends only on their parameters(complexity being correlated with the number of things on which a thing depends).

If purely functional programs in some language to accomplish some goal are harder to understand than stateful implementions, the appropriate response is not to criticize functional programming, but instead to try to find ways to make that language more understandable or more suitable for accomplishing that goal.

"And, if you don't understand that abstractions leak and that the real world (and machines) are stateful/not functional, you're in for it."

I think there are actually conceptual advantages to thinking of the universe functionally(as a tail-recursive function with all the "state" contained in the parameters), in case a method for time travel is ever developed. It seems as though it would be easier to reason about than it would be if the universe is treated as stateful.

<comment><USER4> > Pure functions are easier to think about than impure functions.

Many pure functions are easier to think about than their impure equivalents, but it's certaintly not the case the all pure functions are easier to think about than their impure equivalents.

> Pure functions are simpler than impure functions because their result > depends only on their parameters.

And, because pure functions have no state, many pure functions have to drag along state through their parameters that they don't need/use but that their callees need/use.

> (complexity being correlated with the number of things on which a thing > depends).

The complexity of the things matters a lot as well.

> If purely functional programs in some language to accomplish some goal are > harder to understand than stateful implementions, the appropriate response > is not to criticize functional programming, but instead to try to find ways > to make that language more understandable or more suitable for accomplishing > that goal.

"the appropriate response"? Sorry, I have work to do, work that doesn't include making advocates of functional programming happy, let alone spending time on things that they haven't been able to do. (Some of the developers of functional programming systems are quite brilliant so the fact that they haven't solved certain issues is significant.)

> I think there are actually conceptual advantages to thinking of the universe > functionally(as a tail-recursive function with all the "state" contained in > the parameters), in case a method for time travel is ever developed.

Talk about premature optimization, not to mention the bogus assumption that a universe with time travel can't be stateful.

<comment><USER6> > Many pure functions are easier to think about than their impure equivalents, > but it's certaintly not the case the all pure functions are easier to think > about than their impure equivalents.

I disagree. Most cases of the pure function being harder to think about than the impure equivalent are a result of either demanding impure semantics from a pure function(pure functions aren't always the best choice for implementation) or bad programming. I think that even programs that are best implemented statefully such as databases are easier to think about if they are defined conceptually in a pure way(as functions taking an additional parameter to represent the database state when it is called) regardless of how they are actually impelmented.

> And, because pure functions have no state, many pure functions have to drag > along state through their parameters that they don't need/use but that their > callees need/use.

I don't understand this argument. Calling another function on data from their parameters is the way functions need/use their parameters.

> "the appropriate response"? Sorry, I have work to do, work that doesn't > include making advocates of functional programming happy, let alone spending > time on things that they haven't been able to do.

I'm not saying that you have some sort of obligation to make it possible to more understandably solve some problem purely. However, as I am one of those functional programming advocates, I think that you would be helping yourself by solving problems purely when impure semantics are not a necessary requirement of the problem(I would also expect that fewer problems than you might think require impurity). What I meant by what you quoted is not that you specifically should make a better functional language, but that flaws in the existing languages are not necessarily a condemnation of the concept of pure functions.

> (Some of the developers of functional programming systems are quite > brilliant so the fact that they haven't solved certain issues is > significant.)

Significant, but hardly an indication that those issues are unsolvable. C was developed by quite brilliant people, but there were many issues they did not solve.

<comment><USER4> > Most cases of the pure function being harder to think about than the impure > equivalent are a result of either demanding impure semantics from a pure > function(pure functions aren't always the best choice for implementation)

So much for the claim that purity is always easier to understand....

>> And, because pure functions have no state, many pure functions have to drag along state through their parameters that they don't need/use but that their callees need/use.

> I don't understand this argument. Calling another function on data from > their parameters is the way functions need/use their parameters.

Callees provide a service. I don't care how they provide said service, but if they're "stateful", functional programming exposes that.

> However, as I am one of those functional programming advocates, I think that > you would be helping yourself by solving problems purely when impure > semantics are not a necessary requirement of the problem(I would also expect > that fewer problems than you might think require impurity).

Read up the thread. I'm a believer, and I'm telling you that the way that fp folk advocate is counter-productive. (Telling people that they shouldn't reject fp because some future language might be useful for their problem is especially dumb.)

> C was developed by quite brilliant people, but there were many issues they > did not solve.

C wasn't developed by folks who were trying to introduce a new type of programming language. C was, in some sense, an afterthought for them because their argument was about systems building, not programming languages. Also, it was a different world then - C didn't really have competitors.

While FP can be a fantastic way of programming, FP advocates are mindbogglingly ignorant of people and programming in the real. (Perhaps the ability to write small programs for some non-trivial problems is an obstacle to understanding.)

But, feel free to ignore me.

<comment><USER5> >FP advocates are mindbogglingly ignorant of people and programming in the real

I think i agree with you on some points, but this just sounds like mindless generalisation.

Not all fp advocates are like this, if it were so, we wouldn't see so much real world solutions to the use of fp in non academical languages

<comment><USER4> > Not all fp advocates are like this, if it were so, we wouldn't see so much > real world solutions to the use of fp in non academical languages

There is the possibility that FP gets wins despite the actions of FP advocates.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> As long as IT services business, the most cost-effective solution will always be considered best.</comment>


Reasons why humans won't be cloned - <AUTHOR> https://mindmatters.ai/2019/07/three-reasons-why-humans-wont-be-cloned/ <comment><USER2> This article uses something that is _not proven to exist_ —a soul—as evidence that humans can't be cloned. One could replace the word "soul" with "unicorn" here and make just as strong an argument.

</comment><comment><USER3> A clone is an identical twin born decades later, similar to having a frozen IVF identical twin embryo thawed out and raised decades later (which is currently possible).</comment>


Why It's Time to Build Software in Russia - <AUTHOR> https://blog.toughbyte.com/why-its-time-to-build-software-in-russia-e886b7aa4fb2#.5zqird2od <comment><USER2> It would be interesting to know which known startups have a dev team in Russia. I've heard that Uber is one, any others?

<comment><AUTHOR> A lot of big companies have R&D offices in Russia, e.g. Intel or Deutsche Bank. I don't think there are many startups here yet</comment></comment>


Bootcharting - <AUTHOR> http://alexander.holbreich.org/2011/07/bootcharting/ <comment><USER2> Why the blog spam? Just link to <http://www.bootchart.org/>

</comment><comment><AUTHOR> Because it shorter. You see the the poin at one page. Hot to install and how to disable. And you have disscussion. E.g. you also can find a link to bootchart2 in a comments: <https://github.com/mmeeks/bootchart>.</comment>


Making a total hash of it - <AUTHOR> http://www.economist.com/science/tq/displaystory.cfm?story_id=10789298

<comment><USER2> And, of course, this is old news, and, of course, it only works with MD5 (and --- we expect, but do not yet know --- SHA1, but not SHA256).

Your takeaway from this article is, "don't ever use MD5 again". It has no value. Use CRC's for error detection, and use a SHA hash for security.

<comment><USER3> This is probably not old news to most readers of The Economist.

<comment><USER2> It's also not useful to most readers of The Economist, since nothing they commonly use relies on MD5.

<comment><USER3> The Economist is basically general interest stuff for smart people. They seem to like that sort of thing.

<comment><USER4> Yeah, there is a bunch of news there that's not personally that relevant to me, but I like reading it just the same, being a bit of an information junkie.

</comment></comment></comment></comment></comment><comment><USER5> I can't believe it's 2008 and we're still reading _new_ stories about MD5 collision-resistance. We get it, anybody can manufacture N files with the same hash and different content. Don't use MD5 as a piece of your vote commitment scheme and move on. All of this was news years ago.

edit: MD5 is still good for random other stuff. If you MD5 something, it's not _that_ easy for someone to find a collision with it. I'm anxiously awaiting the revelatory article on rainbow tables in The Economist.

<comment><USER2> What is MD5 good for? It's slower than a CRC and less secure than SHA1. It's a crappy universal hash (bad constant factors) and it's usually much too large for a unique identifier.

<comment><USER5> My intent isn't to bat for MD5, really. But since you asked me, I'll point out that MD5 is actually not that bad if you want a deterministic string of randomly distributed bits. If you don't need all 128 bits, nobody's forcing you to keep the extra ones. That said, you might as well use SHA1.

<comment><USER2> For what it's worth, almost any random number generator is better than MD5 at giving a deterministic random bit string, given a seed. You might start with OpenBSD arc4random.</comment></comment></comment></comment>


Women 2.0 Labs pre-incubator program applications close Sept 4th - <AUTHOR> http://www.women2.org/women-20-labs-new-class-this-fall-2010/

<comment><AUTHOR> Apologies for submitting an article with the W-word, but these folk are particularly after more engineer applications (and it's not just women, most teams from the first batch had 1 or 2 blokes too).

It looks like an interesting cross between Startup Weekend and YC, both in content and time, though with more focus on getting an idea validated and customers waiting than on hacking up something cool quickly.</comment>


The Beginners Guide to Coding for SEO - <AUTHOR> http://www.seotakeaways.com/beginners-guide-coding-seo/

<comment><USER2> The content of this article isn't useless by any means, but there was a lot more you could have went into.. This was more of a general programming guide for making SEO tools, but what about coding best practices that help with SEO?

Example: If you're using Ruby on Rails, get rid of the existing URL naming conventions. Instead of naming your pages /page/<ID> or even worse page=id=<ID>, name them page/FRIENDLY_URL_WITH_KEYWORDS.

Example 2: By default, add a NOINDEX tag to all user generated pages when they're first created. Create a Cron Job that runs every night to check if each page has > X number of words. If so, remove that NOINDEX tag. You don't want Google to think your site is full of "thin-value" pages.

Just some non-obvious things you could've included...</comment>


Mars – A cross-platform network component developed by WeChat - <AUTHOR> https://github.com/Tencent/mars <comment><USER2> Pardon my ignorance, but I tried reading through the page and could not figure out what this is or does. Can someone point me to a quick summary or overview?

<comment><USER3> It has a lot of overlaps with other network frameworks such as OkHttp, AFNetworking, cronet, etc., but I think it main contributions are these two components:

Xlog: in memory mmap log for fast logging and also supports encryption. Xlog log enables them to log everything fast and "safely" even in production build to quickly identify bugs.

STN: short and long TCP connection manager, which is optimized for various networks in China.

Plus other network utilities to make development easier.

<comment><USER4> Out of curiosity, is XLog better than the app simply logging everything to STD out, then let the invoker worry about the storage?

<comment><USER3> Depends on what the invokers wants. If the invoker only wants to log things to disk and read those logs later, surely he can do it himself. What XLog provides for the invoker under the hood is essentially performance: the invoker can log lots of things and does not need to worry about IO speed, disk space, and log safety. And of course it's cross platform. A detailed performance analysis is here (in Mandarin Chinese): [http://mp.weixin.qq.com/s/cnhuEodJGIbdodh0IxNeXQ](http://mp.weixin.qq.com/s/cnhuEodJGIbdodh0IxNeXQ)

</comment></comment></comment></comment><comment><USER5> This looks like a cross between OkHTTP and a jobqueue (like [https://github.com/WhisperSystems/jobmanager](https://github.com/WhisperSystems/jobmanager) or [https://github.com/yigit/android-priority- jobqueue](https://github.com/yigit/android-priority-jobqueue)).

But I think this is the first cross platform framework that does this.

<comment><USER6> Can you explain what is the definition of cross platform here, I mean in which part of it is "cross-platform"? Is Realm also consider a cross-platform database solution for mobile?

<comment><USER5> The library itself claims to be cross platform. It has libraries available for iOS/Android/windows mobile.

I'm not sure about Realm..But I was talking in context of the job manager and network libraries.

But I suppose you could SQLite as cross platform.

</comment></comment></comment><comment><USER7> I hope this contains the code that makes Wechat connections work so well on the often intermediate/slow/bad connections in China. Because then most companies can learn something from it (looking at Skype, Slack etc here which are horrible with bad connections).

<comment><USER8> Im surprised no one brings up how bad Slack's infrastructure or backend design is. It's my slowest app from launch to signed in and can see the latest message.

In the old days MSN messenger was faster.

<comment><USER9> So it's not just me! I have issues with Slack quite regularly, from messages simply not appearing on web (usually iOS works fine) to the app not being able to reconnect without a hard refresh. Add to that the slow 'boot-up' and memory usage and all in all the experience is quite disappointing, especially for such a loved product where good connectivity should be a core requirement.

<comment><USER7> I think I read somewhere the client is hybrid? That does not explain the very bad connectivity but could explain boot up and memory. I find it quite weird that so many techies adore it to be honest.

<comment><USER9> Oh yeah, it's most definitely a 'web app' packaged as an executable.

I mean, I get why people like it. When it works it's a pretty decent experience (and I grew up with IRC).

</comment></comment></comment></comment></comment><comment><USER10> Great to see a open source project from WeChat. WeChat is extremely viral in China. I am sure there is some amazing engineering work underneath.

<comment><USER11> The language barrier has been their main issue, they'd definitely would love to collaborate more with teams outside of China I know. They have an astonishing amount of dev resources as well, they wrote an entire cloud platform (they're at maybe 10% of AWS) from scratch, in C.

<comment><USER12> > entire cloud platform (they're at maybe 10% of AWS) from scratch, in C

Impressive. Any link to this? I'd love to take a look.

<comment><USER11> Their public cloud is [http://qcloud.com/](http://qcloud.com/), I don't think everything they've written on the backend is public however.

<comment><USER13> It has nothing to do with WeChat. qcloud is developed by TEG (Tech. Engineering Group), and the cloud service for WeChat is developed by WXG (Weixin Group).

</comment></comment></comment></comment></comment><comment><USER14> What is China's Super-App "WeChat" and how China is changing your Internet [https://www.youtube.com/watch?v=f-YuU3J-YV0](https://www.youtube.com/watch?v=f-YuU3J-YV0)</comment>


Ask HN: App Landing Page Service - <AUTHOR>

A couple of months back there was a post about a startup providing landing pages for mobile apps.  I now need to avail of such a service but cannot find the link.<p>Does anyone have recommendations for such a service or even an HTML template that is fairly customizable? <comment><USER2> ooomf.com

<comment><AUTHOR> Looks great and includes tracking/sign-up perfect! Thanks.</comment></comment>


Paul Graham spills: Why some companies get his cash and others don't - <AUTHOR> http://www.entrepreneur.com/article/219377 <comment><USER2> _Knowing the business. If we ask a bunch of questions and they have the answers at their fingertips because they understand the domain really well, that's a good sign._

An old boss of mine is a design judge for a student engineering competition. Often times he will ask a team for a specific piece of information about their project, only to have the team pull out a giant binder and start shuffling through looking for the answer. At this point he tells them "If I ask you to tell me your girlfriends phone number and you pull out your phone book and start searching for it, you're probably not in love"

I think the same sentiment applies here

<comment><USER3> I love the comparison, but I would imagine that due to mobile phones and contact lists, it's quite possible to be in love without knowing any phone numbers.

Personally I know the numbers of the (at a guess) 20 people I call most often, but I always assumed that, in 2011, I'm the exception not the rule.

Not particularly relevant, his point stands either way, just interested to know if other people on HN still find themselves memorising phone numbers.

<comment><USER4> The only numbers I know are the ones that haven't changed since before I was a teenager (basically just my parents' numbers and 911), my own Google Voice number, and Bing 411. That's partially because I can always look up numbers in my contacts, on Facebook, or online, and partially because I make phone calls less than once a week.

<comment><USER5> I don't bother remembering a phone number, unless calling that number will result in the person on the other end bailing me out of jail.

So far, 100% success rate.

<comment><USER6> wrt being bailed out of jail?

</comment></comment></comment></comment></comment><comment><USER7> "But if you start discarding determination, you very quickly get an ineffectual and perpetual grad student.": Love this line!

<comment><USER8> There are a few transcription errors in this. What I actually said was "ineffectual perpetual grad student."

<comment><USER9> I don't understand the difference between the version with and the version without "and" other than pedantic correctness of the quotation itself.

 _EDIT:Fixed typos._

<comment><USER10> "pedantic", not to be pedantic

<comment><USER9> Fixed. Thanks.

</comment></comment></comment></comment></comment><comment><USER11> It's nice and concise.

> If you imagine someone with 100 percent determination and 100 percent > intelligence, you can discard a lot of intelligence before they stop > succeeding.

How much? What percentage lobotomy are we talking about?

I totally agree btw: provided you are focussed on meeting a need ("make something _people want_ "), and are willing to iterate trial-and-error (that's the determination part), massive intelligence isn't required: the inside of your software doesn't need to be great; and it doesn't need to implemen some fundamentally new technology; it's what it _does_ that counts.

> Or when Google started, there were eight to 10 successful established search > engines already...

I read a great interview (maybe a co-founder of tripod...?) who went through the evolution of search engines over a decade, each time saying that it was too late to enter the field, and playfully finishing with "[of course, now with google, it really is too late]". I've searched and searched for this but can't find it - anyone recognize it?

<comment><USER12> > I read a great interview (maybe a co-founder of tripod...?) who went through > the evolution of search engines over a decade, each time saying that it was > too late to enter the field, and playfully finishing with "[of course, now > with google, it really is too late]". I've searched and searched for this > but can't find it - anyone recognize it?

Well, I guess it's not too late to enter the search field, then.

</comment></comment><comment><USER13> Interesting article that I think most of the people who apply to YC already understand (Hopefully)

As someone who does not have a co-founder and has now applied for the second time I have decided I am moving to the bay area in June whether I get into YC or not and I am either going to find a co-founder to continue with or go to work at another startup while retaining my rights to the work on this project until I meet some more interesting people interested in working long hours for low pay and having lots of fun while doing it.

I cant imagine my life any other way but in a culture where I am creating every day, life is just not the same for me otherwise and working a 9 to 5 like my parents did when i was growing up has never been an option as far as I am concerned.

<comment><USER14> What are you working on right now, Zach?

Why not start living the crazy-startup-founder lifestyle today? That's what I'm trying to do. [1]The time I spend at my current job is like sleeping to me. It's time during a 24 hour period that I just plain will not get back. I have to do it or I will die.

But the hours I spend not sleeping are hours that I spend reading, and hacking, and writing, and more reading, and more hacking.

As much as I'd prefer the weather (and the availability of coffee) in the Bay Area to the weather in Phoenix, I can't make the move happen right now, so I'm just doing the "crazy hours" thing anyway :)

[1]: I just want to clarify that I don't mean I'm slacking off at my job, if anything, being in "work" mode 100% of the time, as well as having a passion for learning new stuff has _benefited_ my employer. When I say "sleeping", I just mean something that I don't have a choice about doing.

<comment><USER13> I am working on my current startup idea, right now it is simply a side project. I work for myself as it is so sleep is certainly a commodity in my life. I am hacking just about all day every day and enjoying the heck out of it.

unfortunately I am in Louisiana and with a shortage of hackers to network with here moving to the bay area just seems to make sense for my work as well as my sanity :)

<comment><USER13> unfortunately no ... no hackerspace, no local coders to speak of really (probably less than 20 programming jobs in out city of 250,000 people)

most of those are older married guys to boot. I started early and honestly had no idea you could even make a living doing this so i just did it for fun and kind of accidentally built something people wanted.

Basically now that I have done that and I realize what is possible as a hacker and in business I plan on repeating, hopefully on a larger scale

<comment><USER13> What I was referring to was a company I have already sold.

It just happens to be what made me realize you could make a living writing software.

Unfortunately I am still under an NDA with the first exit so all I can really say is that it was mostly sharepoint plugins in the legal industry. What I work on now is completely at my discretion and I only do what I like, most of what I do these days is for fun and for the experience.

I have done some work lately for political campaigns and businesses(custom robo-calling apps, calendaring systems for a group of sleep clinics, a product and inventory management system for a manufacturing company etc) but thats just how I make money to pay the day by day bills, my projects are things I am passionate about and love to do, I guess I feel like moving to the bay area would help me get away from the stuff im doing now since I will know less people and I think it would force me to build something I truly WANT to work on instead of constantly accepting projects because there is good money in it and the people know me.

</comment></comment></comment></comment></comment><comment><USER15> To be concise is more indicative of mastery rather than proficiency. You can understand something very well and be able to navigate your way through your own domain of expertise at whim. But if you can do that AND also communicate your domain to those for whom your domain is horribly foreign then you've demonstrated that you've not merely mastered the domain but that you have a grasp on a more foundational framework that translates to virtually limitless perspectives.

Proficiency is to navigate your world, Mastery is to successfully guide a complete alien through that world and have them see it, even if just dimmly, with the eyes of a native.

<comment><USER4> I don't think that what you're calling mastery and proficiency are matters of degree like you seem to be expressing. Ability to communicate can be completely separate from understanding, because (particularly if you're of above-average intelligence) most people don't understand things the same way that you do.

<comment><USER15> I don't think that the underlying inherent knowledge that accompanies the capacity to successfully transmit understanding can be set apart from a more complete understanding.

For example if you can produce any number of outcomes in a single language but lack the capacity to do the same things in another language there's a level of abstraction that you simply have not witnessed which itself gives greater dimension to the fundamental capacities that are being exercised.

If all you know is English, even if you speak it very very very well you can go your whole life missing out on seeing language itself rather than just a one dimensional perspective on a single language. You can have all the parts held in theory but until you can see equivalents in another language in action you lack understanding of English.

The mere capacity to communicate something real demands an empathy that speaks to experience on a higher level. If you can't, in some degree of abundance, find analogous material from which you can more clearly communicate your realm of knowledge to the uninitiated then it says something about the true depth of your knowledge.

Just as you can't get real depth perception without a plurality of vistas neither can you truly see what you're doing unless you have experience, and thereby the capacity for empathy and making links with that empathy, from far outside of your field of focus.

</comment></comment></comment><comment><USER16> Printable Link: <http://www.entrepreneur.com/article/printthis/219377.html>

</comment><comment><USER17> True: "If you really understand something, you can say it in the fewest words, instead of thrashing about."

<comment><USER4> Not necessarily. My friend and cofounder is one of the smartest people I know, but even when he talks about things I know he is an expert at (as demonstrated by his skill), he's often a bit verbose.

Personally, I can generate concise statements (and I'm good at it when I try), but the first couple of times I verbalize an idea I'm usually somewhat verbose. I make an explicit effort to figure out a concise way to say things once I've noticed that I'll probably be expressing the same idea again (and when I write). Perhaps other people become significantly more concise over time without effort, but I'd estimate my baseline compression rate is only something like 10% (whereas when I explicitly compress I can often get 30% or better).

I think the difference is that some people think very verbally and linearly, whereas I tend to think more visually and abstractly. A lot of the time, I have to translate the ideas in my head from these somewhat amorphous concepts or visual ideas into English (or German, or code, or whatever).

<comment><USER18> What you're missing is that PG says "can." When you're in a ten-minute interview that could change your life forever, you're going to be concise when possible.

That being said, I don't believe that one's 'natural' verbosity has any correlation with intelligence. Minds aren't that simple.

<comment><USER4> Sure; in the particular context of a ten minute interview, you'll have prepared and it ought to be relatively easy to be concise most of the time. But if they throw you a curveball, then I don't think it ought to reflect negatively on you if you're a bit verbose (though it might reflect negatively on you that you didn't anticipate whatever the question was).

I replied because the quote was taken out of context, and I don't think that the idea applies in the general case.

</comment></comment></comment></comment><comment><USER15> To put it concisely :)

A demonstration of empathy that translates to real communication with the particular lay person you are engaging with regard to your domain of expertise shows mastery rather than mere proficiency.

<comment><USER19> Agreed, the important part about being concise is not that you can boil down a complex idea into a short statement, but that you can give the questioner the information relevant to them. When there is too much relevant information to be concise, that is in itself a relevant (and concise) answer, the art is giving the questioner just enough information to ask another question.

Personally I find doing that sort of thing _extremely_ difficult, and doing it in real-time as part of a verbal conversation orders of magnitude more so.

</comment></comment><comment><USER20> > What do you look for? Determination. When we started, we thought we were > looking for smart people, but it turned out that intelligence was not as > important as we expected. If you imagine someone with 100 percent > determination and 100 percent intelligence, you can discard a lot of > intelligence before they stop succeeding. But if you start discarding > determination, you very quickly get an ineffectual and perpetual grad > student.

This is awesome.

</comment><comment><USER21> a great related article at Wired: [http://www.wired.com/wiredscience/2011/03/what-is-success- tr...](http://www.wired.com/wiredscience/2011/03/what-is-success-true-grit/) what PG calls determination I think Wired (quoting Angela Duckworth) calls "grit"

<comment><USER21> and the survey measuring how gritty you are is here: [http://www.sas.upenn.edu/~duckwort/images/17-item%2520Grit%2...](http://www.sas.upenn.edu/~duckwort/images/17-item%2520Grit%2520and%2520Ambition.040709.pdf)

<comment><USER22> Clicked it, it didn't work, gave up, moved on.

<comment><USER21> sorry, here's the page that holds three surveys: <http://www.sas.upenn.edu/~duckwort/gritscale.htm>

</comment></comment></comment></comment><comment><USER23> When an entrepreneur is determined and passionate they will find a way to succeed. Without both of those, the "intelligent" individual will just move on to another idea or company.

</comment><comment><USER24> Great line & sentiment --> "And you can see where that line is? Well, this is why we pick based on founders."

</comment><comment><USER25> Tried to read but I am distracted by all the clutter!

</comment><comment><USER26> As a stupid, but determined person this gives me hope :D

<comment><USER27> Well you need to be a bit stupid to want to start a business. You would be considered stupid if someone told you you have a 5% chance at every kissing this girl, yet you decide to spend the next 3 years of your life pursuing her and her only. This is exactly what we do.

<comment><USER28> _You would be considered stupid if someone told you you have a 5% chance at every kissing this girl, yet you decide to spend the next 3 years of your life pursuing her and her only._

If that girl is worth more than 20x as much to you as any other girl, a 5% chance is worth it.

<comment><USER29> Most people have some risk aversion. For example, few people would trade $10 million for a 10% chance at $100 million.

<comment><USER27> Interesting. I am assuming you would. Can you briefly explain your reasoning?

<comment><USER30> Unless you really, really need 100 million, no reasoning will make that a good bet. The expected value is 10 million, and since that's what you're putting at stake, you have nothing to gain. Whereas the utility value of a guaranteed 10 million, assuming you're not very rich already, is huge. It's more than an average lifetime of hard work.

</comment></comment></comment></comment></comment></comment><comment><USER29> Old news.</comment>


Why every programmer should learn to optimize algorithms - <AUTHOR> https://www.programiz.com/blog/scalability-algorithms <comment><USER2> Turns out algorithms aren't just for clearing the customs!</comment>


Ask HN: Does having "and" in a domain name not recommended? - <AUTHOR>

any penalty on the branding, SEO or word-of-mouth effect? <comment><USER2> It didn't work out too well for these guys: <http://www.ferrethandjobs.com/>

<comment><AUTHOR> I LOLd for a good 2 minutes for that. nice catch.</comment></comment>


ASP.NET MVC 4 Beta Released - <AUTHOR> http://weblogs.asp.net/jgalloway/archive/2012/02/16/asp-net-4-beta-released.aspx

<comment><AUTHOR> Sorry for the double submission. I fixed the title and URL (deleted the original submission since I couldn't fix the URL)</comment>


Tesla, Apple, and the 'secretly terrible' engineer conundrum - <AUTHOR> http://venturebeat.com/2015/10/17/tesla-apple-and-the-secretly-terrible-engineer-conundrum/ <comment><USER2> Even in my local, not silicon valley part of the world, this is a problem. So many recruiters are looking for a very specific skill set they ignore talented general programmers.

Look at the JS landscape. It's disgusting. Quite literally a new framework created almost every day. Who the hell has time to keep up on that? How many ways can you build a React Website? Dozens. AJS 2.x changes versions what, weekly? I read the other day they're gonna skip to V4 and start upping the major version more quickly.

What recruiters need to learn is they need to start finding good, well rounded programmers. A good programmer doesn't really care what they write in. We all have our preferences for sure, but a good programmer wants a challenge more then comfort. They generally want to learn something new, then to do the same thing day in and day out.

A company that spends or allows me to learn something new over the course of a few weeks or month / 2 months is likely to keep me as a loyal employee longer then one that just wants me to write in the same languages I have been for the last decade.

give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime

Give a man a job they know how to do, and he'll stay for a while. Teach a man a new technology and hell stay for years.

<comment><USER3> I'd bet they are filtering them out not for failing to have the verbatim skill set (though that happens), but because they're be too expensive or (in larger companies) too good.

It's been repeated many times that companies mainly look for those who smile, look the part, comply, bow, and are willing to work endless hours for pennies. It's also been repeated that management typically freaks out when someone talented shows up, then become obsessed with stomping them out, drowning them in BS, etc.</comment></comment>


Why Do the Northern and Southern Lights Differ? - <AUTHOR> https://www.scientificamerican.com/article/why-do-the-northern-and-southern-lights-differ/ <comment><USER2> The northern/southern lights are bucket lists items for me. Hope to mark off the northern lights soon though. Flights to Iceland have gotten pretty affordable with Wow airlines. I finally saw the southern hemisphere night sky a year ago and imaged the Large and Small Magellanic clouds. Crossed that item off the list. Atacama dessert is another item.

<comment><USER3> I would recommend going to Yellowknife in Canada instead. If you stay there 3 nights, you have a 95% chance of seeing it. Plus it's closer to the magnetic north, so the lights _always_ pass over regardless of intensity.

<comment><USER4> I'd be curious to hear more about this. Is there a certain time of year, is there a certain area you need to stay, what time of night, etc. Feel free to shoot me a link if there is already a good resource explaining some of this

The Northern Lights have been on my bucket list as well for quite some time but have always let the seemingly random chance of actually seeing them dissuade me from really planning something.

<comment><USER3> I wasn't able to find a single place that had all of this documented, but here's what I know.

* January to March is the best time of the year because the weather is dry and cloudless. * Pick dates where the moon either sets before it gets dark, or when it's new moon. * Book viewing tours at multiple places. Aurora Village is amazing as it's on a frozen lake and there's heated teepees for you to shake the cold off. There are other tours as well which will take you around so you get the opportunity to photograph the Aurora in various backdrops. Best bet is to probably just DuckDuckGo it. * Rent cold weather clothes at Yellowknife. -40 C/F is typical daytime and nighttime temp during the winter.

</comment></comment></comment></comment><comment><USER5> _> Dazzling green and red light displays regularly dance across the night sky-_

OK I'm going to stop this right here.

The Northern Lights do _not_ dazzle you with the incredible green light displays you see in pictures. In real life, you might not even realize you are looking at a Northern Light unless you know what to look for.

The only way the lights appear so vivid is by taking a photo of them with a camera with a long enough exposure. When I saw the Northern Lights I had mistaken them for bright clouds illuminated by the moonlight. Only when I took a photo with my iPhone did I realize they were the Northern Lights.

The fact that I was not made aware of this until I arrived at my final destination after a journey of thousands of miles across the world has made me bitter about it.

With such an inaccuracy on the very first line of this article, I can't bring myself to continue.

<comment><USER6> As someone who grew up in Montana and North Dakota and has been an amateur astronomer from my early teens, I think you're being overly dismissive. Sure, sometimes and they're more of a faint white glow (which is pretty impressive just by itself), but in over two decades of semi-dedicated skywatching (before moving further south), I saw, with nothing but naked eyes, bright green curtains on multiple occasions, dazzling blue-white pillars all the way up to the zenith once, and red curtains twice. And that's living between 46 and 48 degrees of longitude where they're not that common.

If you're setting out on a trip to specifically view the aurora borealis, you'd want to make sure you're viewing from an area with very low light pollution, make sure you stay dark adapted, and schedule your trip around solar max. Under those circumstances, and given multiple clear nights, I would expect the odds would be excellent for seeing some quite breathtaking auroras.

<comment><USER7> It's amazing that you can see them so far south in the US. I live at +48° latitude too and around here it's completely unheard of to see anything of the Northern lights (even when I used to live in the countryside at ~+50°).

Thanks for the advice.

<comment><USER8> In central Kansas, I have seen faint aurora several times in the last 10 years. Nothing bright or fast moving, but if you paid attention and looked they were definitely aurora. And my Canadian mother also confirmed it.

</comment></comment></comment></comment><comment><USER9> Tangental Prettiness:

ISS Symphony - Timelapse of Earth from International Space Station [https://www.youtube.com/watch?v=wgdbZhnFD5g](https://www.youtube.com/watch?v=wgdbZhnFD5g)</comment>


Research Priorities for Robust and Beneficial Artificial Intelligence (2015) [pdf] - <AUTHOR> http://futureoflife.org/data/documents/research_priorities.pdf <comment><USER2> If one starts to tease out any of the arguments around what the existence of general AI would imply, you notice that within this hypothetical terrain, nearly all the abstraction used in the discussion leak very heavily to the point of being difficult to even use.

For example, _what is_ benevolent? Many people would claim that we have failed to "program" the entirely human intelligence of government leaders and corporates heads to act beneficially to large swathes of humanity.

But this just highlights to me that there is little agreement on what "beneficial to humanity" means.

If Margaret Thatcher is correct that society does not exist, only individuals, perhaps the task of "beneficial AI" is to encase us each in our own virtual reality bubbles where we can specify the world we want to live in.

But if we take a different view, that maintaining some sort of human society is necessary for the growth or maturity of people, then beneficial AI would have to work to help people relate as a society.

And in either way, creating "beneficial AI" implies working out beforehand what is beneficial to human, something that we haven't to do before now simply through lack of the god-like powers needed to impose such a conception.

<comment><USER3> We want to make a utopia. But no one knows or agrees what utopia would look like. And it s very easy to get it wrong and get stuck in a bad or suboptimal system.

I think that (superhuman) AI will help us figure this out. It could model us well and figure out "what we really want". It could propose questions and ideas we wouldn't think of, and help us work out what the answer Is.

However the problem is getting to that point. We don't know how to make an AI that will follow instructions at all. Most possible A Is do everything within their power to take over the world.

<comment><USER2> I would love to make utopia. Maybe some portion of technologists would like to make utopia.

If one is talking about "humanity", it is hard to talk about "what we want".

Edit: The problem I'd see with the "AI can let us reach the utopia whose characteristics we can't agree on" is imo that intelligence doesn't have goals as such baked.

A very, very skilled therapist talking to a feuding family may be able to get them to compromise. But almost all such compromises are going to be predicated on the social context the therapist and family work. If there isn't a social consensus pointing to what utopia for the entirety of humanity could look like, even superhumanly intelligent computer couldn't talk us into such a thing - though it quite possibly could talk us into a position that reflected the social value's its creators implicitly or explicitly programmed in. My hope would be it's creators won't be religious fanatics but naturally said religious fanatic might have the opposite hope.</comment></comment></comment>


Groupon Files for Discounted IPO - <AUTHOR> http://abcnews.go.com/blogs/business/2011/10/groupon-files-for-discounted-ipo/

<comment><USER2> This smells like admitting failure and running for money rather than going through the pain of improving their business model. The novelty is wearing off.</comment>


Survey shows surprise decline in US startups run by young people - <AUTHOR> https://gigaom.com/2015/01/05/survey-shows-surprise-decline-in-us-startups-run-by-young-people/?utm_content=buffer4c161&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer <comment><USER2> I, for one, welcome anything that will get VCs to stop blindly believing "young" equals "better investment", especially when most VCs I come across are only interested in short-term gain.

That and state/corporate-sponsored "youth startup" things. I deal with enough of those to have seen oodles of fickle, immature teams who don't know the first thing about working, let alone building product.

<comment><USER3> I don't believe they think "young" equals "better investment", I think their process expects "young" to equal "controllable".

</comment></comment><comment><USER4> Since when did innovation and skill become a characteristic of being young? What's the new age ceiling for when skill meets opportunity? Does this have anything to do with adults working jobs that would have been occupied by minors just 20 years ago?

<comment><USER5> Youth culture has been responsible for social and cultural innovation (art, music, fashion...) for as long as they've had the extra leisure to devote to it. The growth of the merchant class, the growth of the middle class and the abolition of child labor all resulted in increased leisure for youth segments of society. Your second question must be missing some punctuation because I'm going to suggest you look into Yanni.

<comment><USER5> Why did I get down-voted? I answered the question and added a little humor. "Innovation" is not analogous to "invention". (Here's Alan Kay on the subject: [https://www.youtube.com/watch?v=gTAghAJcO1o](https://www.youtube.com/watch?v=gTAghAJcO1o)) TLDW: Technological adoption follows patterns of consumption like fashion. When we talk about "innovation" we're really talking about technologies that succeed in the marketplace and were successfully adopted by society.

Over the last decade we've seen major tech growth in two areas: mobile computing (computers as fashion accessories) and social networks. Both happen to be areas where youth culture has historically served a role as early adopters. If you're looking for historical perspective to figure out when 20 somethings were given the keys to the tech kingdom, it happened over the last decade when tech companies started using youth culture to drive consumption and "innovation".

Why do we need companies full of 20 year olds?

1) Because 20 year olds develop products for other 20 year olds. Most of them don't have the experience, wisdom or maturity to do anything else and they are fundamentally still trying to please each other and define themselves as a generation.

2) 20 year olds have a ton of energy, health and are willing to work ridiculous hours on bad code for lower pay.

3) Making millionaires out of a few 20 somethings every year helps feed the fire.

</comment></comment></comment><comment><USER6> TLDR 3.6% of households headed by adults younger than 30 owned stakes in private companies, 10.6% in 1989, 6.1% in 2010.

Private company != startup. The article is irrelevant to the startups we talk about here.

<comment><USER7> I agree. It's a worrying statistic for sure, but it has nothing to do with startups.

</comment></comment><comment><USER8> The "decline" is strictly in terms of percentage. I'm sure a sizable amount of this is due to older cohorts making a similar jump. Bad statistics, sheesh.

<comment><USER9> Percentage is what matters. You are less likely to start a startup than your parents were.

<comment><USER10> there are situations where percentage is misleading.

10% of people start a startup today, 5% 5 years ago.

20% of young people start today , 30% 5 years ago.

chance of a young person starting today? 2%, 5 years ago? 1.5% . therefore young people are more likely to start a startup today than 5 years ago

<comment><USER11> To clarify I think what you meant in your example is that 20% of those who start today are young while there were 30% of the same cohort 5 years ago.

</comment></comment></comment></comment><comment><USER12> How is this surprising? Young people are graduating with a crushing amount of debt so now they're faced with a choice between entrepreneurship versus a steady paycheck at zombie corp. It seems most are choosing the job at zombie corp in order to pay a huge pile of bills staring them in the face.

<comment><USER13> I don't think this is a valid criticism. Most of the VC funded startups I'm aware of are founded by graduates of top US schools (Ivy or similar), where almost no one graduates with debt.

An alternative hypothesis that fits the data equally well is this:

As startups become more popular, the low hanging fruit is quickly picked over (social networks, mobile apps that require very little initial capital, etc.). Concurrently, startups have become more mainstream and seen as less risky (whether justified or not). As such, many "older" (by which I mean not in their 20s) people with narrow domain expertise have begun to start companies with the goal of addressing some narrow need that isn't addressable by the typical smart, CS-grad, startup founder with little to no real world experience outside of tech internships.

<comment><USER14> > Most of the VC funded startups I'm aware of are founded by graduates of top > US schools (Ivy or similar), where almost no one graduates with debt.

Not true. Although not bad, students typically graduate with $15k in debt. Families with income just above the aid cutoff point or with high assets but lower income tend to get hit hard; I know some individuals that are graduating with $50-100k in debt from undergrad at Ivies.

[http://www.browndailyherald.com/2014/04/07/brunonians- highes...](http://www.browndailyherald.com/2014/04/07/brunonians-highest-debt- ivy-league/)

<comment><USER12> > Most of the VC funded startups I'm aware of are founded by graduates of top > US schools

I know plenty of folks that graduated with debt from top schools.

> graduating with $50-100k in debt from undergrad at Ivies.

Indeed.

</comment></comment></comment></comment><comment><USER10> households headed by people under 30 is somehow a good indicator of college dropouts? would not be suprised if there were other effects like college students living in their parents garage to bootstrap their startup.

could have measured the growth in accelerator applications by college students as an alternative</comment>


Trade votes with voters from swing states - <AUTHOR> http://www.pbs.org/newshour/bb/apps-trade-votes-clinton-swing-states/ <comment><USER2> This is not as stupid an idea as I thought from the headline, but it still suffers from the problem that (for good reason), there is no way for a position to prove who they voted for. As such, both parties are strictly better off cheating by saying they want to trade and voting for their own preferred candidate instead.

<comment><USER3> The article says they met up and mailed their absentee ballots together. Maybe the trading sites could use something like a video recording of someone checking a box on their ballot and sealing it up or do a recording of them in the booth casting their ballot (state law permitting). There is still some trust involved, both people can't really vote at the same time. You have to trust the other guy will go through with it after you cast your vote first but it's no different than buying and trading used goods on forums. Although on forums, people are usually making exchanges more often than once every 4 years so someone can build a trustworthy reputation.

<comment><USER2> Yeah, mailing in your absentee ballots together could work - I assume that the people on apps aren't doing that because it would be difficult for most people to travel to another state.

That said, anything that allows you to prove who you voted for is a flaw in the design of the voting system anyway, so to the extent that there are loopholes that allow this, one would hope that they will be fixed.

As for reputation - reputation systems rely on the ability to verify that something has already been completed. If you order something online and never receive it, you are able to complain and cause reputational damage to the seller. In the case of vote-swapping, you shouldn't _ever_ be able to know whether or not the other person voted the way they said they did, so you never actually know that the other person is untrustworthy.

</comment></comment></comment><comment><USER4> Fighting 'anti-democracy' with anti-democracy.. Disgusting.

</comment><comment><USER5> We just need to remove the Electoral College and use a straightforward majority wins (or plurality, as the case may be).

</comment><comment><USER6> Wondering how/if this strategy could help voters in safely Republican states who wish to make a dent in that safety.

<comment><USER7> Is this a legal strategy? It seems illegal.

<comment><USER6> On what grounds are you thinking there's anything illegal happening here?

<comment><USER8> [http://www.slate.com/articles/news_and_politics/net_election...](http://www.slate.com/articles/news_and_politics/net_election/2000/11/is_voteswapping_legal.html)

Technically, tampering. Yet-to-be-adjudicated in the same way as someone doing something illegal doesn't always get prosecuted because stare decisis (or political backlash) is so crazy-overused that you rather avoid going forward than lose on technnicalities or by a small margin.

<comment><USER6> Swapping your vote is not technically tampering. And the practice has been adjudicated. See _Porter v. Bowen_. Of course, since that was only an Appeals Court ruling and not SCOTUS, the legality is strictly dependent upon your state and whether it has laws on the books expressly prohibiting swapping. But, those laws could be challenged if one desired to put _Porter v. Bowen_ to the test.

PS: The article you linked dates from 2000. _Porter v. Bowen_ was in 2007. You really ought to check the dates on things you read. A lot has changed in 16 years. Look at _Citizens United_.

</comment></comment></comment></comment></comment><comment><USER9> everything looks like a market when all you have is a hayekian ideology</comment>


Warning from Prince Charles that artisanal cheese could disappear - <AUTHOR> http://www.telegraph.co.uk/news/worldnews/europe/france/12036040/French-traditionalists-praise-warning-from-Prince-Charles-that-artisanal-cheese-could-disappear.html <comment><USER2> This danger was foreshadowed years ago in this period documentary: [https://www.youtube.com/watch?v=B3KBuQHHKx0](https://www.youtube.com/watch?v=B3KBuQHHKx0)</comment>


Turing suicide in doubt - <AUTHOR> http://www.bbc.co.uk/news/science-environment-18562183

<comment><USER2> Previous submission with multiple comments (now on HN front page):

<http://news.ycombinator.com/item?id=4150495></comment>


A tale of an impossible bug: big.LITTLE and caching - <AUTHOR> http://www.mono-project.com/news/2016/09/12/arm64-icache/ <comment><USER2> Properly configured big.LITTLE clusters should be set up so that all CPUs report the same cache line size (which might be smaller than the true cache line size for some of the CPUs), to avoid exactly this kind of problem. The libgcc code assumes the hardware is correctly put together.

There is a Linux kernel patchset currently going through review which provides a workaround for this kind of erratum by trapping the CTR_EL0 accesses to the kernel so they can be emulated with the safe correct value: [http://www.mail- firstname.lastname@example.com/msg...](http://www.mail- firstname.lastname@example.com/msg1227904.html) and it seems to me that that's really the right way to deal with this.

<comment><USER2> Also, if I'm reading the proposed fix in the mono pull request correctly, it doesn't deal with the problem entirely because there's a race condition where the code might start execution on the core with the larger cache line size, and then get context-switched to the core with the smaller cache line size midway through executing its cache-maintenance loop. The chances of things going wrong are much smaller, but they're still there...

(Edit: rereading the blog post, they say they need to figure out the global minimum, but I can't see how their code actually does that, since there's nothing that guarantees that the icache flush code gets run on every cpu before it's needed in anger.)

<comment><USER3> It's fine if core migration happens during the invalidation loop - the core migration itself surely must wipe the non-shared cache levels thoroughly, otherwise nothing would work.

EDIT: Actually if the big and little cores are used together, and not exclusively, then this might still be an issue, yeah.

<comment><USER2> No, in general Linux migrating processes between cores won't nuke the caches. The hardware's cache coherency protocols between CPUs in the cluster ensures that they are all in sync sufficiently that it's not needed.

<comment><USER3> I understood that the configurations currently in use usually only power up either the big or little cores at the same time, and that kind of migration has to wipe the caches, right? But that might be inaccurate, and you are of course right in the general case.

<comment><USER2> The state of the art in Linux scheduler handling of big.LITTLE hardware has moved through several different models, getting steadily better at getting best performance from the hardware (wikipedia has a good brief rundown: [https://en.wikipedia.org/wiki/ARM_big.LITTLE](https://en.wikipedia.org/wiki/ARM_big.LITTLE)). You're thinking about the in-kernel-scheduler approach, but global task scheduling (where you just tell the scheduler about all the cores and let it move processes around to suit) has been the recommended approach for a few years now I think.

</comment></comment></comment></comment></comment></comment><comment><USER4> Seeing bugs like this reminds me of how much nicer things are on x86 where JITs do not need to flush caches. You can actually modify the instruction immediately ahead of the currently executing one, and the CPU will naturally "do the right thing"[1] --- it does slow down execution, as the CPU is essentially automatically detecting and flushing its cache/pipeline, but used sparingly can be a great optimisation. The write can even come from another core ("cross-modifying code") and everything will still work. Someone I know used this to great effect in squeezing out the last bits of performance from an application by eliminating checks on a few flag variables and the associated branching in a tight loop --- it simply "poked" instruction bytes from another core into the loop when it was time for that core to do something else.

[1] With the exception of pre-Pentium CPUs, where modifying at various forward offsets from the locus of execution could give insight into how big the prefetch queue is. With the Pentium it was fully detected and the later multithreaded/multicored react similarly to cross-modifying code as described above, which leads me to believe that Intel is very much supportive of these things as otherwise they could've just told programmers to do as ARM does.

Maybe what ARM needs, short of doing it the Intel way, is a "flush region" instruction which takes both the address _and_ size, so it can automatically flush the appropriate cache lines based on the current hardware's cacheline size.

<comment><USER2> x86 is really the oddity here -- it has its no-explicit-cache-maintenance design because of wanting to maintain backwards-compatibility with self- modifying code that was written for x86 cores that had no caches at all. Almost all other architectures have explicit cache maintenance because it's more efficient (and requires less hardware), at the minor cost of requiring the very few bits of software which do odd things like JITting to explicitly tell the CPU what they're doing.

An instruction for flushing an entire region would potentially have a very long execution time, which is awkward because you would want to be able to interrupt and resume it. So it would need "how far have I got" state stored somewhere. The obvious observation from a RISC-architecture point of view is that you can get the equivalent effect without the pain of making a long- running interruptible instruction, by having an "invalidate one cache line" instruction plus an explicit loop in the code, and that's what most architectures do.

<comment><USER4> Actually, Intel explicitly broke backwards-compatibility starting with the Pentium, by adding the hardware to make SMC work without additional effort. The 486 and below needed an explicit branch to flush the prefetch queue, and this effect has been exploited for various anti-debugging tricks and even this amazing 8088-only optimisation:

[https://news.ycombinator.com/item?id=9340231](https://news.ycombinator.com/item?id=9340231)

 _An instruction for flushing an entire region would potentially have a very long execution time, which is awkward because you would want to be able to interrupt and resume it. So it would need "how far have I got" state stored somewhere._

x86 has the REP prefix for this purpose; used with certain instructions, it decrements a register and if it's nonzero, executes the instruction. The earlier implementations simply didn't update the instruction pointer in this case so the CPU would repeatedly fetch and execute the same instruction, and it's interruptable between each step. The register counts down how many iterations remain. Otherwise, the instruction pointer moves to the next instruction. Modern x86 handles this by generating uops instead in the decoder, but the basic functionality is the same.

</comment></comment></comment><comment><USER5> Different cacheline sizes for the different cores seems like an absurdly bad idea. One because it opens one up to bugs like these, but also because it makes optimization a lot harder. I have a hard time believing the savings due to a larger line size are worth it.

<comment><USER6> ARM's own designs (A53, A57, A72, A73) all have 64-byte cache line sizes and avoid the problem entirely.

The one at fault appears to be Samsung, who designed M1 Mongoose with 128 byte lines and packed it together with A53 cores in their SoC.

<comment><USER7> Perhaps a better (and simpler) workaround, then, could be to clamp the reported cache line size to 64 bytes. So even if the Samsung core reports 128-byte cache lines, the code would simply invalidate each line twice, and if it is migrated in the middle of the invalidation loop, it would still work correctly.

<comment><USER6> This is exactly the fix used in the kernel, yes. The trickiness and complexity comes from the cache line size probe having to be intercepted.

</comment></comment></comment></comment><comment><USER8> wow, just wow. That is a really awesome bug (and like the authors I have issues with trying to sleep when that sort of puzzle is sitting there :-)

Still a bit hazy on _why_ they manually flush the cache for a given block of memory (presumably for protecting disclosure?) but I'm also a bit curious how it works if you get the sequence big fetches a cache line, switches to little which fetches a line (half as long and changing half the bytes in the cache) and then you switch back to big and its thinking it has a full cache line? Presumably there is some mechanism that invalidates cache lines?

<comment><USER2> Handling of the case where big and little both want the same thing in their cache should be dealt with by the usual cache-coherency traffic between the CPUs that ensures they don't disagree about what's in their L1 caches (very handwaved because I don't know the details).

The reason for the manual cache operations is because they're generating JITted code -- on ARM to ensure that what you execute is the same thing you just wrote you have to (1) clean the data cache, so your changes get out to main memory[.] and then (2) invalidate the icache, so that execution will fetch the fresh data from memory rather than using stale info. This clean-and- invalidate operation is usually informally called a flush, though it isn't really one in ARM terminology.

[.] not actually main memory, usually: only has to go out to the "point of unification" where the iside and dside come together, which is probably the L2 cache.

<comment><USER9> I don't see why they have to do this in userspace at all. If they did:

* allocate read/write buffer

* JIT instructions into it

* change mapping to read/execute

* run the JITted code

Then the kernel manages flushing the data caches on the mapping change, and Mono gets to wrap a Somebody Else's Problem field around it. It sounds like they are instead:

* allocate read/write/execute buffer

* JIT instructions into it

* manually flush relevant data caches (with an assumption that the cache line size is constant)

* run the JITted code

<comment><AUTHOR> That approach is harder to use in practice that in sounds. It's not like people have not tried it.

The OS only let you alloc in large granules, like 4k or 16k, and the vast majority of the methods are significantly smaller than that, meaning a JIT must colocate multiple methods in the same allocation block or waste a significant amount of memory.

We could get around that by remapping memory between read/write to read/execute and have the OS solve the problem for us. Except for a couple of small details, modifying a memory mapping is very expensive and we're, well, in the performance business, and that mono is multi-threaded so one thread might be executing code from the exact page we just made non-executable.

This approach, IIRC, was tried by Firefox as it has some security advantages, but discarded due to the measurable performance impact - and they don't have the second problem as JS is single threaded.

Full Disclosure: I'm part of the Mono team.

<comment><USER10> How is this safe in the multithreaded case anyway? If a process has just written a new JITted method and is flushing the i$ on the CPU it's executing on, but then gets scheduled away part-way through the flush, if you were very unlucky then couldn't another thread then get scheduled on that CPU and try to execute the just-written method, which failed to be fully flushed from that CPU's cache?

<comment><AUTHOR> Multi-threaded safety is simply due to JIT controlling the visibility of the newly compiled code. First flush, then make it visible for execution, can't go wrong with that and scheduling won't matter.

Things get a lot more complicated when it comes to code patching, but the principle is similar.

<comment><USER10> I don't think that helps - the point is that the flush might not be effective if the flushing thread gets scheduled away from the core which has the stale I$ before it manages to fully issue the flush.

Or is the flush guaranteed to flush _all_ cores caches? That would be a fairly unusual design.

<comment><USER11> IC IVAU instructions are broadcast to all cores in the same 'inner shareable domain' (all cores running the same OS instance are in the same inner shareable domain)

<comment><USER10> That does seem like a good solution to let you do this kind of invalidation in userspace. Thanks.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I had this problem too with GDB on the Odroid UX4 big.LITTLE SoC.

Since GDB is patching the instrution with ptrace to insert a breakpoint for example.

See my blog post about it: [https://www.kayaksoft.com/blog/2016/05/11/random- sigill-on-a...](https://www.kayaksoft.com/blog/2016/05/11/random-sigill-on- arm-board-odroid-ux4-with-gdbgdbserver/)

Or the post on the GDB mailling list: [https://www.sourceware.org/ml/gdb/2015-11/msg00030.html](https://www.sourceware.org/ml/gdb/2015-11/msg00030.html)

Too bad however that the kernel patchset mentionned in a previous post only covers arm64..

So it's still a problem from arm32.

</comment><comment><USER13> > Worse, not even the ARM ISA is ready for this. An astute reader might > realize that computing the cache line on every invocation is not enough for > user space code: It can happen that a process gets scheduled on a different > CPU while executing the __clear_cache function with a certain cache line > size, where it might not be valid anymore.

I rather see the problem in the fact that there seems to be no possibility to say to the Linux scheduler: Only schedule this process/thread between cores that have the same cache line size. Or add an attribute when some thread is created on the cache line size of the cores it is allowed to run. Or an attribute when some thread is created for "allow arbitrary cache line size but don't let it run on cores with a different size". This way it would suffice to check for the cache size on program or thread start.

<comment><USER14> That's too specific a feature to expose to developers - most people wouldn't even know about the feature, and the ones that did might needlessly enable it just to be conservative.

The intention of the big.LITTLE architecture is to let processes be migrated seamlessly between the small and big core and let the unused core be turned off to save power. The kernel and the hardware should work together to make the core switching transparent and expose a safe way to invalidate the cache independent of the current processor.

<comment><USER13> > The intention of the big.LITTLE architecture is to let processes be migrated > seamlessly between the small and big core and let the unused core be turned > off to save power.

On the other hand flushing specific cache lines is a rather special feature. If the code uses such obscure low-level features (much more than the "typical" application) such as the invariant that the size of a cache-line stays constant over the execution (which most applications really don't care about) one can at least expect from the developer to pass this information to the scheduler so that the scheduler can take that this invariant will indeed be satisfied.

<comment><USER15> Anything that uses a JIT necessarily uses this "obscure low-level feature", including any program implemented in one of many programming languages. Forcing such programs onto the heavy processor makes no sense; there just needs to be a way to properly clear the cache.

<comment><USER13> On [https://news.ycombinator.com/item?id=12483698](https://news.ycombinator.com/item?id=12483698) I wrote a better idea how this might be implemented without causing this problem. Nevertheless even if we use my first, worse interface this should not be a problem: Spawn a thread that does the lowlevel stuff, sync it and after that run the JITted code.

</comment></comment></comment></comment></comment><comment><USER16> Huge props to the team for finding this out. What a nasty issue.

One question about the intro, which states that this is the first mass produced AMP architecture, but isn't the PlayStation 3's Cell CPU one?

<comment><USER17> Cell behaves more like a CPU + GPGPU system, big.LITTLE can schedule an instruction stream on any core of the cluster (depending on the configuration setup), on Cell you'd primary use the general-purpose PPE from which you'd start (and chain) vector-based threads on SPEs.

PPE and SPE don't even share an ISA, SPE have a custom-built SIMD-oriented ISA.

</comment></comment><comment><USER18> _" first mass produced AMP architecture"_

Nope. Remember the Cell? The processor in the Playstation 3? One main CPU with 8 little CPUs and no shared memory, just channels.

The Playstation 4 isn't a AMP machine because programming the Cell was so hard.

<comment><USER19> Cell is essentially distributed memory cluster on single chip, because each SPU has it's own address space and cannot directly access main memory. I'm not sure about what the exact definition of AMP is, but it does not exactly match my feeling of what AMP should be. In this regard Wii seems more like AMP systems with two completely different CPUs (PPC and ARM) sharing what essentially amounts to be same address space (and in WiiU there are 3 PPC cores where one of them is slightly different than other two and cache coherency between them can only be described as broken).

There is no question of hardness of programming for Cell, but I think it's mostly about middleware support (probably because the platform is so different from PC and xbox360).

<comment><USER18> The real problem with the Cell was that each Cell SPE processor only has 256K of local memory. It has bulk DMA access to main memory, but that's more like I/O. 256K is too small for a video frame, a game level, or much else in a modern game. So everything has to be done on an assembly line basis, where data is pumped into a Cell processor, processed, and pumped out. Great for audio, terrible for everything else. In comparison, the main processor had access to 256MB of RAM.

If they'd had, say, 16MB per processor, it might have worked out. One CPU for collision detection and physics, one for NPC management and AI, etc. But giving each SPE processor only 0.1% of the total memory space was too constraining.

</comment></comment></comment><comment><USER20> It appears the that the caching code was added in this patch:

[https://gcc.gnu.org/ml/gcc- patches/2012-09/msg00076.html](https://gcc.gnu.org/ml/gcc- patches/2012-09/msg00076.html)

Prior to that, the call:

      asm volatile ("mrs\t%0, ctr_el0":"=r" (cache_info));

was always made.

<comment><USER21> But the task can be rescheduled on a little core part-way through the execution...

The big cores should report the smaller cache line size always.

<comment><USER20> I was just pointing out where the caching of the size was added. I'm not making a comment on where the fix should be.

</comment></comment></comment><comment><USER22> [https://gcc.gnu.org/ml/gcc- patches/2012-09/msg00076.html](https://gcc.gnu.org/ml/gcc- patches/2012-09/msg00076.html)

<comment><USER23> Well, that is the patch that causes this problem...

At the time this patch was submitted, ARM was running a program that awarded engineers who could improve performance of aarch64.

</comment></comment><comment><USER24> There's no such thing as a simple cache bug. - Rob Pike

Caches are bugs waiting to happen. Rob Pike @rob_pike 21 Mar 2014

<comment><USER25> _" There are only two hard things in Computer Science: cache invalidation and naming things"_ ― Phil Karlton; not sure to what extent this quote is compatible with the second one by Rob. Or does it mean by implication that simply "Computer Science is bugs waiting to happen"?...

<comment><USER26> It's a great quote, but it's wrong. There are actually _two_ hard things in CS: cache invalidation, naming, and off-by-one errors.

<comment><USER27> The parent actually appears to have the quote both correct in content and attribution. Supposedly, someone else added the "off by one"[1][2][3]. That seems to be the extent of the Internet's knowledge on the quote, though the Skeptics link notes that there's nothing direct to the supposed originator.

This is one of those quotes where I feel there's more than one right answer. I like the addition of "off-by-one", and to make it a nice round three things, I usually use this version:

      There are three hard things in computer science:

      1. Naming things
      2. 3. Concurrency
      Cache Invalidation
      4. Off-by-one errors

[1]: [https://twitter.com/timbray/status/506146595650699264](https://twitter.com/timbray/status/506146595650699264)

[2]: [https://skeptics.stackexchange.com/questions/19836/has- phil-...](https://skeptics.stackexchange.com/questions/19836/has-phil-karlton- ever-said-there-are-only-two-hard-things-in-computer-science)

[3]: [http://martinfowler.com/bliki/TwoHardThings.html](http://martinfowler.com/bliki/TwoHardThings.html)

<comment><USER26> The other one is the original, I just like the "off by one" version better.

I like the addition of "concurrency" too, but I'm not quite sure how to make it flow....

</comment></comment></comment></comment></comment><comment><USER28> As an embedded developer also working on big.LITTLE arm64...

I would do nasty, painful things to the designer of such a system.

</comment><comment><USER29> When would a programmer explicitly need to invalidate the CPU cache? Does this not happen automatically on a context switch?

<comment><USER30> One example is when doing a dma operation to memory. That often bypasses the cache. If the new data is to 'stick' the cache needs to be convinced it doesn't know what's in that buffer any more. This is an ARM thing; Intel architectures integrate DMA with the cache.

</comment></comment><comment><USER31> From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?

<comment><USER32> That would create a race condition addressed at the bottom of the article: the process can get switched onto another CPU between the invocation of get_current_cpu_cache_line_size() and the invalidation.

      An astute reader might realize that computing the cache line on every
      invocation is not enough for user space code: It can happen that a process
      gets scheduled on a different CPU while executing the __clear_cache
      function with a certain cache line size, where it might not be valid
      anymore.

<comment><USER33> The follow up doesn't make sense to me

       Therefore, we have to try to figure out a global minimum of the cache line sizes across all CPUs.

Wouldn't this mean they'd always just end up clearing half the cache line for larger core anyway?

<comment><USER14> No, you just sometimes issue twice as many flush requests as necessary. You can't flush or invalidate half a cache line, since the data is stored in units of cache lines.

In theory I think you could just invalidate the addresses byte for byte, ignoring the cache line size, but I assume the performance hit would be noticeable.

</comment></comment></comment></comment><comment><USER34> This is an excellent bug journey, and I'm even more impressed that the resulting discovery has already been used to improve Dolphin. A testament to the quality of both projects.

</comment><comment><USER35> I wonder why no one tried to validate Asymmetric MultiProcessing by first validating all cases with either little or big first. And then bisect further down when both are enabled.

<comment><USER36> Because that's the sort of thing you think of only when you already know the answer.

</comment></comment><comment><USER37> I don't understand that level of coding, what i do understand is the great way of debugging. Its all about deduction mr Watson.

<comment><USER20> Watson was actually Dr. Watson. Which is/was also the name of a debugger in Windows[0].

[0] - [https://en.wikipedia.org/wiki/Dr._Watson_(debugger)](https://en.wikipedia.org/wiki/Dr._Watson_\(debugger\))

<comment><USER37> You are right, although i was referring to a Sherlock Holmes quote, but that is also Dr.

<comment><USER38> Funny thing is that Sherlock Holmes usually used abductive reasoning, instead of strictly deductive reasoning.

</comment></comment></comment></comment><comment><USER39> Can someone explain why cache flush is used for on ARM or in general low level programming?

<comment><USER40> Overwriting executable machine code in memory is a rare case (only JITs need it, only for emitting code). Some CPUs do wire the memory write operation to the instruction cache to make sure the instruction cache knows some code changed, but ARM chose a simpler design where the instruction cache assumes the code never changes in memory and if this assumption is wrong, the programmer has to use a special instruction to tell the instruction cache to throw something out of cache. The cache clearing instruction throws out either 64 bytes aligned to multiple of 64 or 128 bytes aligned to multiple of 128, depending on the core.</comment></comment>


Ask HN: Are cloud storage services safe for legal personal documents? - <AUTHOR> I mean specifically things like tax returns, passport scans, driver license scans, and things of that nature.
I work in software, and I know that no systems are ever 100% secure. As long as something is online, there is a probability of being hacked into.
Having that said, are services like Google Drive, Dropbox, Box, etc. "secure enough" for storing said backup documents?
This comes up as a question I have after the thread about house possibly being burned down. The urban part of Bay Area is currently not completely safe from the fires, even though we're at a lower risk than areas around Santa Rosa. <comment><USER2> I personally see no reason to not encrypt such documents when stored on cloud services. Why risk a compromise if it's relatively easy to mitigate?</comment>


Climate Engineering - <AUTHOR> https://en.wikipedia.org/wiki/Climate_engineering <comment><AUTHOR> Most of the discussion of combatting climate change focuses on reducing atmospheric CO2, but there's an entire second category of approaches: Solar Radiation Management, some of which could be much faster and cheaper than reducing CO2. [https://en.wikipedia.org/wiki/Solar_radiation_management](https://en.wikipedia.org/wiki/Solar_radiation_management)

While reducing CO2 emissions via investment in renewables is still an important long term goal, if we're already heading towards a potentially catastrophic situation, it seems we should be investigating other forms of climate engineering more seriously?</comment>