Offloading Human Memory to the Cloud - <AUTHOR> http://jordancooper.wordpress.com/2010/05/31/offloading-human-memory-to-the-cloud/

<comment><USER2> My Dad, in his late 80s, is redoing the Watson family video. About 1 hour of media. Dad (still a member of the US National Academy of Science) has a quirky sense of humor: he likes to mix in Poser 3D animations with his videos so his favorite virtual character, 'Hank' an acerbic gorilla, keeps chiming in to my family history. Fortunately, Roger Ebert will never see this :-)

</comment><comment><USER3> Though several startups are addressing this area, I like the timeline approach taken by Eternos (<http://beta.eternos.com>)

[Disclaimer: I've blogged enthusiastically about them in the past and we've discussed a possible advisory role.]

</comment><comment><USER4> I'm still too busy being wow'd by the implications of color video and widespread photography. Of my grandparents, I have a few black-and-white's from their late 20's. Of my parents, a couple off-color's from their early 20's.

My decedents (if I have any) will have full, accurate color video and photo from a ton of points in my life.

Forget recording who I was and what I did- it still blows my mind to imagine 'knowing' your parents as they were when they were young. Images are the first step to this; they make the past real.

<comment><AUTHOR> images are a big piece, but what i'm suggesting is that it will get to a point where you can actually interact with your dead grandparents. think about predictive AI on these data sets... it's going to be way more rich than images alone...

<comment><USER4> You're right on the mark there, I just haven't even gotten used to the idea of images yet!</comment></comment></comment>


Improving Supercomputing Accuracy by Sacrificing Precision - <AUTHOR> https://www.top500.org/news/improving-supercomputing-accuracy-by-sacrificing-precision/ <comment><USER2> A large portion of engineering calculations overall requires something like three significant digits of precision; the problem is that these digits need to be accurate, i.e. we need a good error estimate, consistent with that precision. Most numerical algorithms do not provide such estimates, and so we've fallen into a cargo cult of excess precision, hoping that it will save us from the accuracy loss in the calculation. It often works, but occasionally fails, and we have no idea which one happened.

I have a simple example of that: a polynomial rp(x,y):=1335/4 _y^6+x^2_ (11 _x^2_ y^2-y^6+(-121) _y^4-2)+11 /2_y^8+x/(2*y) that evaluates to about 1.1726 in both single and double precision, but whose real value is -0.827....

So, the bottom line is that I'd gladly trade off precision for better accuracy guarantees---but it turns out that's surprisingly hard in general case.

<comment><USER2> Sorry--forgot to specify the values for the polynomial: 77617 and 33096. In Maxima, the effects of precision can be simulated thus:

      for fpprec:10 step 5 thru 50 do print (rp(77617.0b0, 33096.0b0));

1.17260394b0 1.17260394005318b0 1.8014398509481985173b16 1.172603940053178631858835b0 1.17260394005317863185883490452b0 1.1726039400531786318588349045201837b0 -8.27396059946821368141165095479816291999b-1 -8.27396059946821368141165095479816291999033116b-1 -8.2739605994682136814116509547981629199903311578439b-1

google will find you more info about Rump's polynomial..

<comment><USER3> Numbers of that magnitude combined with sixth and eighth powers should get you thinking 'hey, something's fishy here'.

Also, here's a good reference: [http://epubs.siam.org/doi/book/10.1137/1.9780898718027](http://epubs.siam.org/doi/book/10.1137/1.9780898718027)

And I'm too headachey to bother trying, but I wonder what happens if you ignore that x/2y part and put the rest of the thing into Horner's form.

</comment></comment></comment><comment><USER4> TLDR: When using an iterative solver, do the first few calculations in reduced precision, and only then switch to double -- It is more energy efficient.

</comment><comment><USER5> I always thought it would be cool to have a top500 list that does not restrict the algorithm, just accuracy and precision of fft. It would be interesting to see how algorithms would get us from 10 petaflops to "1 exaflop equivalent" \-- the flops equivalent would be time speedup to get the same answer in less time. As it is they have very strict algorithm requirements (which makes sense if you are regularizing on flops). Just curious, but of course execution time on the top supercomputers isn't cheap.

<comment><USER6> What about doing something like the inverse? Top500 for algorithms on a fixed hardware configuration. You have an input dataset and an answer (perhaps something with an analytic solution, for sake of comparison). That would let you compare the algorithm more directly.

</comment></comment><comment><USER3> The idea of changing the method as you advance in the problem is not a new one, here's a recent article in SIAM Review [http://epubs.siam.org/doi/abs/10.1137/130936725](http://epubs.siam.org/doi/abs/10.1137/130936725)

<comment><USER7> Googling yields a not pay walled one Paper title is Composing Scalable Nonlinear Algebraic Solvers</comment></comment>


Large study shows females are equal to males in math skills - <AUTHOR> http://www.news.wisc.edu/18508

<comment><USER2> Unfortunately the article does not mention the standard deviation of the distribution of mathematical ability after controlling for other factors. As mentioned here: <http://www.lagriffedulion.f2s.com/math.htm>, Larry Summers got in a lot of trouble at Harvard for making this assertion:

"It does appear that on many, many different human attributes-height, weight, propensity for criminality, overall IQ, mathematical ability, scientific ability-there is relatively clear evidence that whatever the difference in means-which can be debated-there is a difference in the standard deviation, and variability of a male and a female population. And that is true with respect to attributes that are and are not plausibly, culturally determined."

Is that assertion backed up by evidence? And, if true, is making this distinction useful in informing public policy and shaping our culture? In a fight between naturalistic and moral fallacies, which wins?

<comment><USER3> The article doesn't say a darn thing about methodology. For all we know, they could have done anything from comparing means to measuring the overlap of various confidence intervals to arcane statistical methods that predict the accuracy of classification based on the data.

Darn paywalls around scientific journals. I really wish more would go the route of <http://www.plos.org/>

</comment></comment><comment><USER4> I think it's generally accepted that if you take a sampling of average men and average women, they will both perform at similar levels in regards to mathematics.

However, I believe the differences between men and women in mathematical thinking, should they exist, will only be apparent when it comes to dealing with the levels of math that deal with abstractions upon abstractions and so on.

In my personal experience, I've noticed that 90%+ of (American) women are just not interested in formal systems, be it predicate logic, computer science, pure mathematics, etc. I'm inclined to believe women are just wired to prefer more practical disciplines, but I think a compelling argument can be made for either view. I suspect the answer probably lies somewhere in between.

<comment><USER5> I think it's quite possible there are imbalances in the tails of distributions, but it still means that the middle could plausibly be more balanced. I'd personally not worry that much about gender balance among Fields Medal winners, and more about gender balance among engineering graduates from the local Big State U. The men there are probably above average in math ability, but few are world-class mathematicians; most are probably in something like the 70th-90th percentiles for math ability, a region which doesn't seem (from what data is available) to have large inherent gender imbalances.

<comment><USER6> I agree with the grandparent post, because I think it's misleading to lump all of mathematics together. I'm very bad at mental calculations and basic arithmetic, and I couldn't memorize integration formulas if my life depended on it, so I've never had very high grades in math, at least until I hit university. Yet I have a relatively easy time picking up the theory behind SAT solvers, Fourier transforms, or solving complex integrals.

I think some people are better at concrete things like geometry or arithmetic, some people are better at logic, some people are good with operators and algebra... but one does not imply the others, and engineers may have an above- average capability for abstraction or logic that does not translate to much higher results in a high-school level exam.

The study here is said to have studied people "from grade school to college and beyond", so I suspect the "math ability" they tested really has little to do with the math ability required for science and engineering.

Here's another (slightly caustic) blog post which appears to be about the same study: [http://motls.blogspot.com/2008/07/janet-hyde-boys-girls- in-m...](http://motls.blogspot.com/2008/07/janet-hyde-boys-girls-in-math- not.html)

 _It is enough to read a brief review in Science to see that the headlines don't follow from the paper at all. The review explains the main reason of the results: they have detected no signal because the tests were too simple. They didn't really test "g" or the ability to think mathematically but rather attainment._

<comment><USER7> More anecdotal evidence!

Last night, I jokingly asked my SO what 30/4 was. They said 8.25. Seriously. And they're an academic that does math for a living.

I never got past single variable calculus, but I can do most 2 digit math in my head, accurately.

<comment><USER8> I study maths and if you asked me what 30/4 was, I'd respond, "it is 30/4, obviously".

Math is not about calculating. I rarely perform calculations on numbers greater than 20, and I do not even remember when was the last time I performed a long division. What is this "2 digit math" you are talking about?

<comment><USER7> I'm sorry, I should have typed "thirty divided by four," which is what I actually said, instead of 30/4, but I was lazy. And yes, math is absolutely not about calculating. I was supporting the previous poster's point that they suck at calculating, and that math and calculating are very different things. I hate math. I am completely comfortable with calculating. WTF with the downvotes.

</comment></comment></comment></comment></comment></comment><comment><USER9> Autism rates are 3 times more likely in males than females, which I see as an extreme symptom of a more general trait. This is what I chalk up the math and science gender gap to, though I have absolutely no scientific data to back that up.

<comment><USER10> Has anyone ever seen an articles with a Professor's of Women's Studies or holder of a doctorate in Women's Health arguing that rates of autism in men and women should be the same and claiming that they're only different because of a misogynist agenda?

> _The idea that both genders have equal math abilities is widely accepted > among social scientists_

Really? Seriously? Sounds like "social scientists" prefer spin to statistics.

>If, before a test, you imply that the women should expect to do a little worse than the men, that hurts performance. It's a self-fulfilling prophecy.

This sounds like a specific form of a general psychological result that would hold true for men too. If so then this article is beyond disingenuous it is a flat lie. That aside in order for the result to have a bearing on real world test scores it would require someone to be telling women just before their maths test that they're going to fail. Who's doing that, it would be pretty easy to spot in a school and certainly in my country a teacher doing that would lose their jobs.

> _These changes will encourage women to pursue occupations that require lots > of math._

Why? Why would measuring maths ability change the preferred career of anyone?

<comment><USER11> /* >If, before a test, you imply that the women should expect to do a little worse than the men, that hurts performance. It's a self-fulfilling prophecy.

This sounds like a specific form of a general psychological result that would hold true for men too. If so then this article is beyond disingenuous it is a flat lie. That aside in order for the result to have a bearing on real world test scores it would require someone to be telling women just before their maths test that they're going to fail. Who's doing that, it would be pretty easy to spot in a school and certainly in my country a teacher doing that would lose their jobs. */

It does hold true for men -- specifically, men of African-American and Hispanic descent, when reminded that minorities generally do worse on math exams before the exam is given.

The proposition that nobody is telling women they're going to do worse on math tests, however, is simplistic. It isn't that some mean old guy is telling them they suck (although one of my female physics professors mentioned, out of the classroom, that she had been thrown out of a physics class at Georgia Tech in the early 1970's for "taking a man's spot" -- you're right that this sort of thing is much less common these days, at least in the West). It's that the whole culture is suffused with the attitude that women's mathematical abilities are suspect -- witness threads like these, which appear over and over on the interwebs, discussing whether or not women are as capable as men at math.

Measuring maths ability scientifically and publishing the results, rather than relying on folk-science and anecdote, could help encourage women to trust their own desires and abilities mathematically -- even when faced with a predominantly male culture and continual doubt being cast on their abilities to function at the highest levels.

I know this to be true because I am now a lone female coder in a group of (really great, smart, delightful) guys, loving my work on a complex, challenging system. And I remember that I dropped out of the very first coding class I took after a few weeks because I was completely intimidated by the swaggering guy classmates who threw around terms I wasn't yet familiar with -- I felt out of place and was full of self-doubt. It was only after maturing and understanding the social dynamics that I retook the class and ended up with one of the top 5 grades, out of several hundred students. Yet, at first, I had been certain I was incapable -- not because any guys were mean to me (not one was anything but helpful), but because I doubted myself, and felt alone and weird.

And surely this isn't gender-specific, and surely many geeky coders can relate, and have probably had similar experiences in different areas of life. This isn't a woman-man thing only -- it's a specific expression of a general human tendency to reflect cultural attitudes about their lives in the images they create of themselves.

<comment><USER10> > _It does hold true for men -- specifically, men of African-American and > Hispanic descent, when reminded that minorities generally do worse on math > exams before the exam is given._

I think I've read a general result along the lines of "you have characteristic X, people with that characteristic perform worse" and that this skews the result. You're poor, you're female, you're disabled, but I couldn't really be bothered digging around for the papers.

Simplistic? Yes, but as I recalled the research was for the situation where they were told quite shortly before the test about their expected sub-par performance so I was relating it to the research.

>Yet, at first, I had been certain I was incapable -- not because any guys were mean to me (not one was anything but helpful), but because I doubted myself, and felt alone and weird.

Overcoming self-doubt and social issues is part of being in a particular field though - if jargon rich fields put you (ie "one") off then there are many fields you would struggle in. If you need someone to believe in you before you can do well in a maths test then IMO you're not going to do well when you've only got yourself to rely on to get something done.

>Measuring maths ability scientifically and publishing the results, rather than relying on folk-science and anecdote, could help encourage women to trust their own desires and abilities mathematically

Go on. What do you mean by maths ability - it's a pretty diverse subject after all. I've seen people do excellently via rote learning whilst for me it was my strength because I could pretty much start with a few "axioms" and work on from there when memory failed - clearly very different abilities that appeared (at undergrad level) to be closely equivalent.

</comment></comment></comment></comment><comment><USER12> The journal article as pdf <http://www.apa.org/pubs/journals/releases/bul-136-1-103.pdf>

</comment><comment><USER13> I wish the article said what _kind_ of math they tested for. There's a world of difference between being good at multiplication and being good at proving P≠NP.

</comment><comment><USER14> God, really, I hope this turns out to be right. If our brains are fundamentally different, we (men and women) are doomed to never really understand each other, which I find totally depressing.

</comment><comment><USER15> Like a wise person once said: _Interest is aptitude_

All of the nature/nurture constructs built around males to interest them in the _autistic endeavors_ like math, physics, computer science are very different for females. It's not that woman can't do math, rather it's that they choose not to. I like that researchers are drifting away from attempts at neurological gender difference explanations, and towards a gestalt of social conditioning.

<comment><USER16> By what metrics are STEM fields _autistic endeavors?_ I've never heard that before, so i find it a pretty interesting assertion.

</comment></comment><comment><USER17> I personally do not feel that there is a correlation between sex and maths skills. Its the stereotypes that result in girls not actively pursuing maths related courses. If provided the right education and motivation, anyone can be good at any subject. In India, for example, where the students have no choice but to learn predefined syllabus, girls do pretty well in Maths.

<comment><USER18> _I personally do not feel that there is a correlation_

I'm sorry that I'm about to sound rude, but...

For this kind of question it really doesn't matter what any of us _feel_. Our values, morals, aesthetics, etc. simply have no bearing on the truth of the matter. And none of us has sufficient data points (rigorous, untainted, and unprejudiced) to form a model that has any validity. We really only have two choices: (1) say "I don't care", and ignore the debate; or (2) read the study, then criticize methodological flaws if any, else deal with the results.

<comment><USER17> There is a third choice: "I care, but please bring a better case-study". As stated above, the gist of my argument is that all are born equal, it is the society and the stereotypes that shape them.

<comment><USER18> That's not a third choice. If you can show that the methodology of the study fails to account for this, then you've got a case, which I accounted for. But when it simply doesn't match your "feeling", well, I'm sorry, but that just doesn't matter.</comment></comment></comment></comment>


Ask HN: Does spammy self-promotion drown out real hacker news? - <AUTHOR>

I've noticed that a lot of interesting news submissions never make the homepage.  I think it's because a lot of companies post a blog entry that's really just pumping their own brand and get all the employees to vote it up.  Meanwhile, the interesting news post doesn't have the manufactured upvotes so gets lost in the shuffle.  My former company even figured out how to game the system and get by vote filters (it's fairly simple).
Looking at the HN homepage now, many are what I would consider blatant self-promotion and border on spammy.  Granted, I have no problem with someone wanting feedback on their site or offering a really in-depth look at internal technology (e.g., this is how we handle failover and scale to 1MM uniques/day), but a lot of companies just manufacture reasons to blog solely to boost traffic (company memo: thanks for the blog ideas and upvoting our post, HN gave us 10K hits today!  good leads!).
HN page as of now (50% are self-promotion, several IMO are legit): 1. Neural Networks for Machine Learning (coursera.org) 3. Keccak wins the SHA-3 competition (nist.gov) 5. Show HN: I built a stock market game based on the top 50,000 websites (linkdaq.net) 8. Lockitron: Keyless entry using your phone (lockitron.com) 9. Software on Steam (steampowered.com) 12. Social Login Buttons Aren't Worth It (mailchimp.com) 16. Online Python Tutor: Learn programming by visualizing code execution (pythontutor.com) 18. How We Doubled Our Android Install Rate in One Hour (dwellable.com) 20. Show HN: Meet your cardboard buddy. (foldable.me) 21. Filepicker.io launches Alfresco, WebDAV, & FTP integrations (filepicker.io) 22. Show HN: Summaries of interesting content, written by people (tldr.io) 24. Lead Product Engineering at OrderAhead 27. How to Model Viral Growth: The Hybrid Model (linkedin.com) 29. Meet the 2012 MacArthur Fellows (macfound.org) <comment><USER2> If you ever figure out a way to game HN, then contact PG privately, but unless you know him personally, do not expect a response from him confirming or denying the existence of the problem. HN uses a lot of anti-abuse code that is not publicly known (e.g. secret sauce), and it's best for everyone if the secrets remain secret.

I have privately reported a half dozen or so abuse problems over the last four or five years. Though I live only thirty minutes south of PG, I don't actually know him, so I always give him an "easy out" by telling him no response is necessary when reporting abuse problems. I also refuse to pry regarding the anti-abuse code, even though I have to fight my own excessive curiosity. Ya, I've always wanted to know how it works, but I also understand why I shouldn't be told. It kind of sucks for me, but it's better for everyone if the anti- abuse code continues working.

If you're good with search engines and have a ton of time to kill reading posts, then you'll find mentions of "voting ring detection" on HN, but as noted, the details are not public. Anyone who has studied discussion forum design will tell how hellishly difficult voting ring detection is to implement effectively in an open voting environment with (mostly) anonymous accounts.

As for the increasing self-promotion seen in submissions, well, it's always been a problem, and worse, it's also a very difficult problem to address effectively. If suitably formalized, you might be able to prove both of these problems to be undecidable (unsolvable, insoluble, or whatever pedantic term is currently being used for impossible). The only good news is, even when you can prove a problem can't be completely solved, there is still the potential of finding a more effective _partial_ solution.

<comment><USER3> >it's best for everyone if the secrets remain secret.

Good old security through obscurity.

</comment></comment><comment><USER4> On the one hand, I think we police ourselves pretty well, but on the other hand I'm pretty sure posting to /newest is a complete crap shoot unless you are posting for a certain domain or as a certain user.

</comment><comment><USER5> Well yeah always cheating will drown out the real thing, but still if the community actually see this new real interesting thing it will find it's way to the front page.</comment>


Samsung Pays Apple $1 Billion Sending 30 Trucks Full of 5 Cents Coins - <AUTHOR> http://dhirubhaism.com/samsung-pays-apple-1-billion-sending-30-trucks-full-of-5-cents-coins/

<comment><USER2> [http://www.guardian.co.uk/technology/blog/2012/aug/29/apple-...](http://www.guardian.co.uk/technology/blog/2012/aug/29/apple- samsung-trucks-nickels-fake?newsfeed=true)

</comment><comment><USER3> I won't believe this without a secondary source. A photo would be good too.

</comment><comment><USER4> you know it's a joke, right?

<comment><USER5> You think? Math don't lie: 20B nickels x 7 grams a piece = 140,000 metric tons. I'd hate to see the UPS bill for that.</comment></comment>


Partially Destructive Garbage Collection - <AUTHOR> http://daeken.com/partially-destructive-garbage-collection

<comment><USER2> There are already frameworks that support lazy loading object data (e.g. Hibernate).

The other part of what you described is having the runtime do the actual unloading of the objects and replacing them with stubs to conserve memory. IMO this would be an extremely difficult problem to do well. A primitive version of this already exists at a lower level than the runtime in the form of paging/swapping.

</comment><comment><USER3> This seems like an inverse of what you get when haskell's laziness is an advantage, namely that with haskell you defer computing stuff until its needed in a certain sense, whereas in this case, we're considering eager computations where whenever the code path for generating some data is purely functional in some sense, the runtime heuristically deletes the data and if it comes to be needed again, it gets once again computed.

This woudn't end well unless there was some way for the runtime/compiler to know the complexity of the computation.

On the other hand, this does remind me of work on what's called "self- adjusting computation" by several folks at uchicago and cmu, (<http://ttic.uchicago.edu/~umut/research/>) which winds up being a really slick way of converting certain offline algorithms into online algorithms that in some cases have runtime complexity comparable or the same as the best known explicit online algorithms

<comment><AUTHOR> Thanks for your input -- I just started thinking about it earlier today, so I'm really looking for a lot of feedback.

As for the Haskell reference, I agree. I'm thinking about lazy evaluation in terms of the way objects are recreated.

As for knowing the complexity of the computation, I don't really think that's necessary. If the PDGC is part of the runtime, it can track the cost (in terms of time to create the object).

I'll give the linked research a read over, thanks, and if you can think of any other related research I'd love to see it.

<comment><USER4> PDGC would make a lot of sense as the GC system for Haskell.

The key change you need to make to the RTS is, when you evaluate a thunk, rather than overwriting it with the final value, instead append the evaluated value to the thunk. Then, the GC should be free at any time to erase the "evaluated" part of the thunk if doing so would allow it to free up a lot of memory.

When a thunk is entered it should return the evaluated value if present, otherwise it should use the code pointer and closure data to create the evaluated value again.

The key challenge is designing the algorithm that decides what is beneficial to lose references to. Probably you want to keep some stats around about frequency of access of thunks and so on.

I haven't seen anything like this proposed for Haskell, so I would be interested in reading a paper, should you implement it :-)

<comment><USER5> _The key change you need to make to the RTS is, when you evaluate a thunk, rather than overwriting it with the final value, instead append the evaluated value to the thunk._

This already happens in GHC.

Any thunk that is to get updated has one word reserved as a slot for the update result. This is necessary for proper operation in SMP systems, when two processors race to evaluate the same thunk. The only data that gets overwritten is the thunk's code pointer. There is one standard routine in the binary, known as the indirection code, which just returns the value in the update slot of the currently active thunk. The thunk's pointer is set to point to the well-known address of the indirection code. In all this, the thunk's payload remains untouched.

GHC's GC short circuits these indirection nodes, and ignores pointers in the thunk payload (since the thunk payload has become effectively unreachable). In order to accomplish your goal, we would have to recover the original value of the code pointer. This can be done by storing a copy of this value in the payload (similar to what you are suggesting). A way to do this without increasing the size of any heap object is for the compiler to generate a copy of the indirection code per thunk code block... such that the original thunk code address can be recovered from the particular indirection code address via a lookup (or potentially, via arithmetic). This bloats the emitted code by a small fraction, but may be worth it.

</comment></comment></comment></comment><comment><USER6> You could probably make a lot of headway on something like this using existing technologies. Serialization for example. If an object impelements serializable (in java), then the caching mechanism could hold it in memory, remember how long it has been since the last access and then serialize to disk when it hasn't been accessed in a long time.

In a way, as well, the OS handles a lot of this with virtual memory on a page by page basis. If you have a web app that caches a lot in application state, it'll grow in memory size and then start swapping out that app to disk when it gets stale.

It's a good idea. Programming tends toward laziness so lazy caching is _probably_ almost definitely going to happen. Isn't this kind of what memcache is? Couldn't memcache automatically let go of some of its cached queries?

</comment><comment><USER7> You essentially get something like this with Terracotta <http://www.terracotta.org/>

You can even stop a Terracotta backed JVM and start it back up to continue where you left off without any manual caching code or special weak reference APIs (which is the heart of the question I gather, no intervention).

</comment><comment><USER8> this sounds just like weak references to me

<comment><USER5> Combined with finalizers, yes.

</comment></comment><comment><USER9> This is a rather interesting example of a space-time tradeoff. In this case you are trading time for more space.

</comment><comment><USER10> You can get everything you want with weak references.

<comment><AUTHOR> Weak references can be used for the base of this, but it doesn't handle the actual recreation of the objects, nor is it an automatic thing. My goal with this idea is to reduce the cost of implementing such schemes, and shifting them into the compiler/runtime.

<comment><USER11> A GC wouldn't be able to auto-recreate the object without either an indirection (and thus no better than using a weak reference in a kind of smart pointer), or rather costly logic.

Suppose an instantiated object is 512K in size. You've got a reference to it from 50 places. The GC decides to evict it but still has the construction logic which, say, takes up 128 bytes. All the 50 references now need to be updated, but what will they point to? Presumably some kind of proxy or stub which, when touched, inflates to the size of the original object.

The problem is that this thing can't inflate in-place, otherwise it wouldn't actually be saving any address space. So, the re-instantiation must occur somewhere else, and 50 locations need to be updated. Since a GC isn't currently in flight, _finding_ those 50 locations is going to be very expensive. It's going to have a similar cost to the mark phase of a GC.

The best approach for the GC would be to use an indirection, to treat all references to evictable objects as smart pointers of some kind, with an extra indirection between the reference and the object which may or may not have to be re-instantiated. But this is easily implementable using weak references in runtimes like JVM or CLR.

<comment><AUTHOR> I see this as being very difficult to implement well, but I think it's interesting enough to warrant further research. I really don't know if it'll end up working out, but I'm planning on prototyping it on top of .NET.</comment></comment></comment></comment>


On the Comprehension of Program Comprehension - <AUTHOR> http://neverworkintheory.org/2015/02/13/on-comprehension-of-program-comprehension.html

<comment><USER2> This article cites a study that found programmers don't use 'program comprehension' tools:

>>Our results reveal a gap between research and practice, as we did not observe any use of comprehension tools and developers seem to be unaware of them

Oh I beg to differ. I deal with a gnarly legacy code base every day, tail -f and grep are my 'program comprehension' tools of choice nowadays. Ie exercise the UI, watch the log for something I can grep for, and read.</comment>


Ask HN: Is there a website similar to HN for web/graphics designs? - <AUTHOR>

A place to keep up with the latest designs' exciting stuffs. <comment><USER2> <http://forrst.com> and <http://dribbble.com> would be two places to check out.

</comment><comment><USER3> <http://sidebar.io>

It's also a social news website, but works different than HN. It uses telesc.pe which is kinda cool :)

<comment><USER4> Thanks for the mention! My goal for Sidebar is actually to have it become the "HN of Design". I'm starting small, but I hope to get there eventually!</comment></comment>


My Startup Life Budget - <AUTHOR> https://medium.com/p/eaa28cfa191d

<comment><USER2> Great article - I think lots of first time founders don't spend enough time planning out their budget pre-seed round</comment>


Rich Formula: Quant Trading (2015) - <AUTHOR> https://www.forbes.com/sites/nathanvardi/2015/09/29/rich-formula-math-and-computer-wizards-now-billionaires-thanks-to-quant-trading-secrets/amp/ <comment><USER2> Two interesting inside baseball tidbits here that I thik are worth pointing out:

> From inception Two Sigma's early funds, like Eclipse and Spectrum, focused > on trading stocks globally. Eclipse was faster, changing positions within > weeks, while Spectrum had a longer-term horizon closer to one month.

> The duo eventually used their algorithms to create programs that operate > outside the global stock markets, like the trend-following Compass funds > that bet on futures markets. In 2014 another important fund, Horizon, was > folded into Spectrum, which had diversified its offering beyond stocks. One > of Two Sigma's least visible funds is its Partners Fund, an internal fund of > funds, fueled mostly by capital from the founders.

Most, all?, of the largest hedge funds are actually partnerships that encompass multiple funds. From an internal perspective this allows each fund to focus on a few core competencies, like trend following vs market making vs global macro. Each of these strategies will do well at certain times and do poorly during others.

From a firms's perspective this allows for diversification, which is almost always good. From an employee's perspective it allows them to get paid for their work and be insulated a little bit from the performance of their peers in other funds.

And in some cases like Steve Cohen's SAC it allows the good ideas to retroactively be put into the blessed fudn while the bad ideas are shuffled to the lesser feeder funds. Yes, this is illegal, yes it happens.

From an outsider's perspective its usually means that there is one well performing fund that is closed for new money while there are several lesser performing funds that are open to outside money. Even RenTech has funds that are open to outside money and they don't perform anywhere near as well as their master fund that is for employee's only.

> Two Sigma researchers spend time testing existing models, and each > researcher is expected to come up with two or three new models per year. > These are presented to Overdeck in a white paper that is typically less than > ten pages long. Since Two Sigma's trading models can change its forecast in > seconds, lots of back-testing goes into each model. It's not unlike the way > Amazon exhaustively tests various Web-page changes in real time to ensure > optimal clicks and purchases. At Two Sigma headquarters the model builders, > who need to write code, sit with the engineers and collaborate with them all > the time.

From the strategy development side, often idea's have a half life of anywhere from months in the HFT space to years in the global macro space. For idea's I've since come to believe that idea generation is equal parts people, ie brain power, and platform, ie the ability to iterate.

RenTech is a perfect example of this. They had two people leave who were very high up and go to another fund. They had 2-3 years of poor performance away from the huge backtesting platform that RenTech had built. it's not like these Phd's suddenly forgot everything. Its the ability to iterate quickly on idea's that is the key once the bar has been met for math and intellect.

As they say, its not the algorithm you use but the features that produce your alpha. If you want to make money in the markets focus all your time on feature engineering.

Fun reading:

[https://medium.com/@63ey5f4uw3k42v1exp7/chronology-mercer- me...](https://medium.com/@63ey5f4uw3k42v1exp7/chronology-mercer-medallion- fund-9aa719ceeb4f)

<comment><USER3> > idea generation is equal parts people, ie brain power, and platform, ie the > ability to iterate

To be clear, when you talk about the platform, you mean the research platform used to develop the idea, rather than the execution platform used to apply the idea, right?

What is there to such a platform? What does it actually do? Is it just a question of pumping historical data into a model and measuring its performance?

<comment><USER4> Based on very little more than other journalism and recruiters trying to chat me up, here is my understanding.

The major hedge funds build in-house big data platforms that allow their quantitative traders come up with an idea (what if the price of IBM is 22% predicted by soybean futures in Tokyo on Tuesdays?) and then essentially replay history and see the result.

Say you have the soybean-IBM theory. You feed it into the platform. It applies your proposed strategy across billions or trillions of data points. It probably does a lot more than an SQL query -- possibly checking for interactions with other current strategies in use at the firm, running sensitivity analyses, blasting various parts of it with Monte Carlo simulations and so on. A lot of the same tech is also used to perform forecasts of the coming hour, day, week, month or whathaveyou.

It comes back with the simulated return. If it's good, and if you think you can disguise your strategy while trading, then and only then can you code it up and deploy it to the trading platform. In this article it's claimed that the models are reviewed by a company founder as well.

By now, going on Google's publications about Millwheel and Dataflow, the hedges will already be advanced enough to provide model testing that can converge to a number quickly enough that you can abort the test if it's not shaping up well, thus saving platform time for other uses.

Building your own platforms is a mixed blessing. If you the first to do it you can get an enormous advantage because you are the only company with that capability. Later, as technology continues the march of commoditisation, others gain the capability "for free". Now you are at a slowly increasing disadvantage -- again, because you are the only company with that platform.

</comment></comment></comment><comment><USER5> Could somebody explain why so much effort is being put into quant strategies, when it seems that real-world information gathering would be a much easier way to gain an edge over others? Let's say you pay to place a camera on a building next to a given company's factory, and use analysis software to count the number of trucks coming and going from the factory to predict their order flow and earnings. This kind of thing is harder to scale up, but also gives an edge because not everyone else is doing it.

In an age when all hedge funds have the resources to hire the best and brightest engineers and buy the fastest processing hardware, it seems that none of them will have an edge if they are all starting with the same publicly available data.

<comment><USER6> _> Could somebody explain why so much effort is being put into quant strategies, when it seems that real-world information gathering would be a much easier way to gain an edge over others?_

I used to be part of a research group that sold the so-called "alternative data" you're describing to 30 or so hedge funds in the NYC area, including several of the largest. The example I like to give is that we knew well ahead of time that Tesla would miss on the Model 3 because we knew every vehicle they were selling by model, year, configuration, date and price with <99% accuracy. I still occasionally sell forecasts like this and the methodology is straightforward enough that even a solo investor can consistently beat the market if they know how to source the data. But I've mostly lost faith in this technique as the sole differentiator of a fund's alpha.

Some funds, like Two Sigma, have large divisions with a very sophisticated pipeline for this kind of analysis. They do exactly what you describe. For the most part it works, but there are several obstacles that keep this from being the holy grail of successful trading:

1\. First and foremost, this analysis is fundamentally incomplete. You are not forecasting market movements, you're forecasting singular features of market movements. What I mean by that is that you aren't predicting the future state of a _price_ ; if the price of a security is a vector representing many dimensions of inputs, you're predicting one dimension. As a simple example, if I know precisely how many vehicles Tesla has sold, I don't know how the market will react to this information, which means I have some nontrivial amount of error to account for.

2\. This analysis doesn't generalize well. If I have a bunch of information about the number of cars in Walmart parking lots, the number of vehicles sold by Tesla (with configurations), the number of online orders sold by Chipotle, etc. how should I design a data ingestion and processing pipeline to deal with all of this in a unified way? In other words, my analysis is dependent upon the kind of data I'm looking at, and I'll be doing a lot of different munging to get what I need. Each new hypothesis will require a lot of manual effort. This is fundamentally antagonistic to classification, automation and risk management.

3\. It's slow. Under this paradigm you're coming up with hypotheses and seeking out unique and exclusive data to test those hypotheses. That means you're missing a lot of unknown unknowns and increasing the likelihood of finding things that other funds will also be able to find pretty easily. You are only likely to develop strategies which can have somewhat straightforward and intuitive explanations for their relationship with the data.

This is not to say the system doesn't work - it very clearly works. But it's also easy to hit relatively low capacity constraints, and it's imperfect for the reasons I've outlined. You might _think_ exclusive data gives you an edge, but for the most part it does not (except for relatively short horizons). It's actually extremely difficult to have data which no other market participant has, and information diffusion happens very quickly. Ironically, in one of the very few times my colleagues and I had truly exclusive data (Tesla), the market did not react in a way that could be predicted by our analysis.

The most successful quantitative hedge funds focus on the _math_ , because most data has a relatively short half-life for secrecy. They don't rely on the exclusivity of the data, they rely on superior methods for efficiently classifying and processing truly staggering amounts of it. They hire people who are extraordinarily talented at the _fundamentals_ of mathematics and computer science because they mostly don't need or want people to come up with unique hypotheses for new trading strategies. They look to hire people who can scale up their research infrastructure even more, so that hypothesis testing and generation is automated almost entirely.

This is why I've said before that the easiest way to be hired by RenTech, DE Shaw, etc. is to be on the verge of re-discovering and publishing one of their trade secrets. People like Simons never really cared about how unique or informative any particular dataset is. They cared about how many diverse sets of data they could get and how efficiently they could find useful correlations between them. The more seemingly disconnected and inexplicable, the better.

Now with all of that said, I would still wholeheartedly recommend this paradigm for anyone with technical ability who wants to beat the market on $10 million or less (as a solo investor). A single creative and competent software engineer can reproduce much of this strategy for equities with only one or two revenue streams. You can pour into earnings positions for which your forecast predicts an outcome significantly at odds with the analyst consensus. You can also use your data to forecast volatility on a per-equity basis and sell options on those which do not indicate much volatility in the near term. Both of these are competitive for holding times ranging from days to months and, with the exception of some very real risk management complexity, do not require a large investment in research infrastructure.

<comment><USER7> > The example I like to give is that we knew well ahead of time that Tesla > would miss on the Model 3 because we knew every vehicle they were selling by > model, year, configuration, date and price with <99% accuracy.

Is the way in which you got that information something you can divulge? I mean, was it talking to an employee or was it something exciting and far fetched? By the way, I presume you meant ">99%" or something similar.

> A single creative and competent software engineer can reproduce much of this > strategy

By "this strategy", do you mean prediction based on a source of "alternative data"?

Interesting comment, in any case.

</comment></comment></comment><comment><USER8> I'll just leave this here:

"Our definition of success has become narrow, boring, and limited. If we want young people to be creative and innovative, we need to reward them for it."

from "Skip The Hedge Fund: We Need Young People To Take Risks And Build Inspiring Things" at [https://www.fastcompany.com/3026586/skip-the-hedge-fund- we-n...](https://www.fastcompany.com/3026586/skip-the-hedge-fund-we-need- young-people-to-take-risks-and-build-inspiring-things).

</comment><comment><USER9> Quant trading is harder than people assume - really, really, hard. Nearly everything you can think of has already been done, and is being done within latency limits you are priced out of as a retail guy.

So you have to get creative, and if you want it to keep working, you can't tell anyone about it. Ever.

<comment><USER10> Let me let you in on a little secret of the quant world, it is easier than you assume at least in terms of idea generation. Every fund I've ever been with the vast majority of strategies are sourced from academic papers, you don't need to think up new and novel ideas.

<comment><USER11> If every fund is getting the majority of their strategies from the same academic papers, that must do wonders for the profit :)

<comment><USER10> The edge is found usually in the details of the implementation of a strategy.

</comment></comment></comment></comment><comment><USER12> Quants have worked in finance for ages. What's so unusual about Two Sigma? Headcount?

<comment><USER13> Marketing strategy.

</comment></comment><comment><USER14> Tldr for this 2015 article: 2 sigma uses tons of data and processing power to extract any alpha from the markets. They've done it consistently since 2004. People leave 2 sigma and sometimes email themselves code. Bad idea as the company will come down hard on you with lawsuits.

<comment><USER15> Email themselves code? That's an astonishingly stupid way to try stealing something. What did these people expect?

<comment><USER14> Most financial firms have decent lockdown... Generally firewalls prevent file sharing sites, web mail, connecting to any external port that's not http or https. I've seen some places prevent direct IP connections (need a hostname to connect). USB and SD card slots are disabled... Not to say that there still aren't dozens of ways to get around all this if you really wanted to.

</comment></comment></comment><comment><USER16> It seems possible that mathematical breakthroughs are no longer being published, as they are now trade secrets/matters of national security. I wasn't surprised when Tao was beaten by a hedge funder.

<comment><USER6> This has been the case for a long time in applied mathematics and computer science (not so much pure mathematics). There are hedge funds using work that is not only unpublished, but also unknown to research labs like FAIR and Google Brain. The easiest way to be scouted by one of those funds is to publish research that looks like you're on the verge of re-discovering their work.

<comment><USER17> Do you have any proof of this, or is this or is it just your opinion that the comparatively smaller groups of researchers at hedge funds are well ahead of academia and the rest of industry?

<comment><USER6> 1\. I don't have proof I can share publicly,

2\. It's not just my opinion, and

3\. I didn't say they're "well ahead" unilaterally.

This isn't unique to finance; industry labs in tech also often have novel results in applied mathematics and computer science that are ahead of academia and other industry labs. You don't have to believe me but it's not exactly a controversial topic. Not everything is published or patented.

<comment><USER17> I mean I have little doubt that there are trade secrets that these companies have. Specific algorithms and models. And yeah, industry labs are often ahead here.

But I read your claim as saying that there are broad methods and approaches that they hide. And that's, while possible, more peculiar. Most of the tech industry labs don't keep their theoretical research secret. Practically anything that could be published is.

As for 3, the way you described the "rediscovery" made it sound like those Labs were a number of steps ahead, so I hope you pardon my misunderstanding.

<comment><USER6> At the highest level there _are_ broad approaches which are kept secret in the financial industry, but the reason that's peculiar is because their efficacy is inherently antagonistic to publicity. Tech firms (mostly) don't lose utility of their trade secrets if they're exposed, they just lose first mover advantages on those techniques. But if everyone is aware of your techniques in finance, your techniques cease to have an edge.

Like I said in the original comment: this isn't (to my knowledge at least) pure mathematics that's being kept secret. But there are absolutely families of techniques and algorithms whose applications to finance are nontrivial, non-incremental and very well guarded.

<comment><USER17> I guess my only followup would be are these "techniques" more akin (in broadness) to ResNet, or to Dropout? (to use an area that I believe we're both familiar with)

In other words, techniques that are broadly applicable to the field, or techniques that maybe spawn a family of related techniques, but appear to be useful only in a specific subdomain.

<comment><USER6> That's a good comparison. In general, closer to Dropout.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> These guys averaged a ~9% return over the last few years. The S&P 500 has exceeded that. So what's so special? Seems that passive investing works just as well.

<comment><USER6> _> These guys averaged a ~9% return over the last few years._

Who are "these guys"? The funds discussed in the article have average annual returns well above 9%.

<comment><USER18> Overdeck and Seigel.

"The firm's biggest fund, Spectrum, has earned an annual average return of 9.4% net of fees since 2004."

The S&P 500 has averaged 9% for 80 years. Joe Blow can buy index fund and do just as well as the quant clients. Sure, these guys are averaging 14% before fees, and it's a great way to get rich. But after fees the client might as well just passively invest.

<comment><USER19> You need to consider the following elements: -You compare the S&P and two Six Sigma funds over different timelines. The index yielded much less than 9% on 2004-2015 (you can halve the performance). -What's the common feature between: the S&P, the formal idea of passive indexing, and the emergence of passive investment funds? None of them are 80 years old. -Retail customers (Joe Blow) rarely have access to hedge fund products directly (they might still be exposed through pension funds, sovereign wealth funds, etc. but it makes the allocation change almost out of their hands). -Stating performance figures alone is mostly a Bloomberg-reader thing. In practice, there is usually at least an attempt to incorporate some kind of risk measure when selecting hedge funds.

I am not even anti-passive funds but you are memeing a bit too hard.

</comment></comment></comment></comment><comment><USER20> Quant trading is easy with small accounts < 10 millions, the more money you trade, the harder is to find an edge...

<comment><USER21> Can you elaborate on this? That sounds like the opposite of the way it would work.

<comment><USER22> In general, large companies have a lot more data around them, a lot more coverage from analysts and journalists and the like, and generally have a lot more eyes on them. These companies are a lot more efficiently priced. You can do the legwork yourself- getting on management calls, reading industry news, poring over the financial statements to find strength or weakness that are not widely known.

So lets say you are a 10 billion dollar fund. You do your research and find that there is a small $100 million dollar company that doesn't have much debt, is growing rapidly, and the management seems good and there are good tailwinds for the industry as whole.

You have 100x the deployable capital than the company's market capitalization. Even if the stock price doubles or triples in price, your potential upside for the firm as whole is a relative drop in the bucket. You can't deploy more than a few million into this company without moving the market. For a large fund, it isn't worth wasting time chasing these small opportunities. In fact, "scaling" is a hard problem in funds.

Hence, there is actually a lot of opportunity out there for small scale investors who are willing to look under rocks that bigger guys don't see enough potential opportunity in.

</comment></comment></comment><comment><USER23> Article is from October, 2015.

<comment><USER24> Thanks! We've updated the headline.</comment></comment>


Open Source Color Management Is Broken - <AUTHOR> http://www.lieberbiber.de/2018/02/24/open-source-color-management-is-broken/ <comment><USER3> The complaints in this article appear to be regarding OS-controlled lut adjustments, i.e. via special X11 features. I've used color calibrated monitors on linux in color-sensitive industries with hundreds or even thousands of artist seats (including film, animation, vfx, real-time, etc.) and the monitor has always been calibrated via the physical menus/buttons on the bezel and _not_ a software/OS monitor-specific lut adjustment.

I'm not judging which is better or worse as it would indeed be nice if the calibration could be controlled by the OS. I'm just saying that adjusting the monitor hardware directly is what's being done in the pro linux content creation world. Also, I imagine that certain brightness and color gamut controls could only happen from the bezel controls. Dreamcolors can switch between srgb & P3 for example.

<comment><USER4> i don't know color management, but it sounds like the situation is analogous to complaining that VLC on linux doesn't have volume controls (for example; of course it does in reality...) when the solution is to reach over and twist your hardware volume knob.

it's good to know that you've found the controls sufficient on the systems you have adjusted!

</comment></comment><comment><USER5> The Linux community will attempt to solve the problem by attempting to create an entirely new color management system, which will get at best 80% done with a handful of outstanding Hard bugs, at least one of which will be bad enough to break the entire thing except on one specific distro/gui/hardware combination.

<comment><USER6> It will just be a sub-component of systemd. Named colord.

<comment><USER7> Hackernews will tell us that we should really be using Wayland, because Wayland will solve the color problem once and for all.

When it comes time to implement the color management stuff into Wayland, it's declared out of scope for the core Wayland protocol, but don't worry guys -- someday, somehow, a consortium led by GNOME will implement it on a DBUS interface that all compositors will understand. At least two, mutually incompatible, such protocols emerge, neither of which fix the underlying issues, both of which are tied to particular compositors.

Meanwhile, we are told that X11 is still horribly, irredeemably broken in this regard and if we haven't yet switched to Wayland, we really should by now.

The professional colorist industry goes on using Xorg on Linux.

</comment></comment></comment><comment><USER8> It's unclear to me why there's no color palette/correction table exposed in libdrm/KMS at the kernel. It should just be part of the video mode, maybe with a bit to indicate when the driver/hardware doesn't support changing it. But always be able to read it out, make it linear nonsense when the support isn't there - that won't be worse than what we have now.

Then the layers up the stack like Wayland/X/GNOME/KDE are just messengers to/from the bottom @ drm.

We also need floating point frame buffers to be first-class citizens at the KMS level. I don't want to be forced into OpenGL/Vulkan just to be able to have the hardware apply gamma correction on a software-rendered frame buffer, and if I have the hardware do color correction, it kind of needs the HDR of floats - not uchars, which I don't think libdrm supports with dumb buffers of today. If not floats, at least more bits than 8 per color component.

Programs like Plymouth or other embedded style applications running directly on libdrm should be able to have color-corrected output without needing bespoke software implementations and their own calibration tables. I should be able to tell the kernel the correction table, maybe compile it in, or another payload on the boot parameters cmdline.

Hell there are fairly well-known simple algorithms for generating approximate color tables from a single gamma value. If I only want to make things look "better" in my linux kiosk, and care not about absolute color correctness, let me stick a drm.gamma=.55 field on the kernel commandline to generate the correction table in lieu of a full calibrated table.

<comment><USER9> Have you created it, cause if not then that might explain why what you want in detail doesn't exist.

<comment><USER8> Is your point that I should get off my ass and do it instead of complaining?

Or is your point that it's substantially more complicated than it appears, and I'd know that if I tried doing it myself?

<comment><USER9> Yeah... I think both are applicable, but if you really want something done then you do it yourself or pay someone to help do it, especially for open source. It sounds like have a descent idea of where to start. I'd go find some kernel developers working on the underlying components, and offer to pay them or see if they have recommendations of people to get the work done.

<comment><USER8> In doing some searches on the subject this [1] came up. So there seems to already be movement on this front, and it's possible my knowledge of the current situation is stale.

[1] [https://lists.freedesktop.org/archives/intel- gfx/2015-Septem...](https://lists.freedesktop.org/archives/intel- gfx/2015-September/076001.html)

<comment><USER8> There's definitely some level of gamma/color correction funtionality at the DRM level already in the kernel [1]. So my desires may already be largely fulfilled, and maybe userspace just needs to get its act together.

[1] [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/i915/intel_color.c)

</comment></comment></comment></comment></comment></comment><comment> mark-r If you write the calibration info to the video LUT, why does software also need to know about it? Isn't your monitor displaying perfectly calibrated sRGB at that point?

<comment><USER10> This is assuming all the content is sRGB, which is not a safe assumption especially if you care about color management to begin with.

<comment><USER11> If the content is not in sRGB, don't you need to actually know the contents color profile in order to convert it to sRGB? How does the monitor profile matter at this point?

(not challenging, just asking, I know next to nothing about color correction)

<comment><USER12> You don't want to convert to sRGB, you want to convert to the display device color space (which in professional displays is often larger than sRGB).

Most desktop environments and applications assume that the source is sRGB, unless specifically tagged (in image metadata).

So you have two points: source (image, video) and reproduction device (display, printer etc). And you convert colors from the former to the later. sRGB may not even come into play if the source image is AdobeRGB and the display is a wide-gammut pro LCD display!

<comment> mark-r Does the software know how to bypass the LUT, or do you in effect get a double conversion?

<comment><USER12> which LUT?

</comment></comment></comment></comment></comment></comment><comment><USER11> > Except that they only set it for the primary output. Turns out if you have > multiple displays, the profile for the first one is put into the > _ICC_PROFILE X11 atom, but the profile for the second one in the > _ICC_PROFILE_1 X11 atom, for the third display in the _ICC_PROFILE_2 X11 > atom, and so on. It's just that nobody seems to do this.

Sounds like an easy thing to fix. I'd suggest the author to try and make some patches - don't know about GNOME, but KDE is pretty friendly and easy to contribute to.

<comment><USER13> Not everyone is a developer, and this type of comments is why everyone that cares about UI/UX design professionally, eventually goes back to Windows or macOS.

<comment><USER14> That type of comment is also the underlying reason why Linux improves/fixes things at such an astronomical rate compared to either of the choices you listed.

It's not just saying "Hey you, go fix that." , it's saying "Hey everyone viewing this comment on a public page : this exists and needs to be fixed. Someone grab a wrench."

<comment><USER13> Yet the two most successfully desktop experiences among common users, based on Linux, don't expose any details of its eco-system. Quite telling.

<comment><USER15> Which two are those?

<comment><USER13> ChromeOS and Android.

<comment><USER15> I have yet to see an Android desktop.

<comment><USER13> Easy, get a Chromebook.

[https://sites.google.com/a/chromium.org/dev/chromium- os/chro...](https://sites.google.com/a/chromium.org/dev/chromium-os/chrome-os- systems-supporting-android-apps?visit_id=1-636551452624253216-612273770&rd=1)

Or if you prefer to jump on Apple's bandwagon about how regular computers are outdated, a detachable keyboard and mouse for an Android 10"/12" tablet.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> I've been using the ColorHug 1, and found it worked quite nicely. The included Fedora CD (yes, this was some time ago) let me calibrate just fine. Once I wanted to do it straight from XCFE, I ran into a few challenges, but it wasn't unsurmountable: [https://askubuntu.com/questions/427821/can-i-run-gcm- calibra...](https://askubuntu.com/questions/427821/can-i-run-gcm-calibrate-on- xubuntu-without-installing-all-of-gnome) Basically, you have to run xiccd in the background, since XFCE doesn't have built-in colord support to set the X11 atom[1]. I've never used multiple monitors though, I don't know if that's possible with xiccd+xfce instead of Gnome.

[1] [https://bugzilla.xfce.org/show_bug.cgi?id=8559](https://bugzilla.xfce.org/show_bug.cgi?id=8559)

</comment><comment><USER17> I think niche market software, used by a limited number of highly specialized professionals, is somewhat incompatible with the open source economic model. When a piece of software is used by very many users, and there is a strong overlap with coders or companies capable of coding, say an operating system or a a web server, open sources shines: there is adequate development investment by the power-users, in their regular course of using and adapting the software, that can be redistributed to regular users for free in an open, functional package.

At the other end of the spectrum, when the target audience is comprised of a small number of professionals that don't code, for example advanced graphic or music editors or an engineering toolbox, open source struggles to keep up with proprietary because the economic model is less adequate: each professional would gladly pay, say, $200 each to cover the development costs for a fantastic product they could use forever, but there is a prisoner dilema that your personal $200 donation does not make others pay and does not directly improve your experience. Because the userbase is small and non-software oriented, the occasional contributions from outside are rare, so the project is largely driven by the core authors who lack the resources to compete with proprietary software _that can_ charge $200 per seat. And once the proprietary software becomes entrenched, there is a strong tendency for monopolistic behavior (Adobe) because of the large moat and no opportunity to fork, so people will be asked to pay $1000 per seat every year by the market leader simply because it can.

A solution I'm brainstorming could be a hybrid commercial & open source license with a limited, 5 year period where the software, provided with full source, is commercial and not free to copy (for these markets DRM is not necessary, license terms are enough to dissuade most professionals from making or using a rogue compile with license key verification disabled).

After the 5 year period, the software reverts to an open source hybrid, and anyone can fork it as open source, or publish a commercial derivative with the same time-limited protection. The company developing the software gets a chance to cover it's initial investment and must continue to invest in it to warant the price for the latest non-free release, or somebody else might release another free or cheap derivative starting from a 5-year old release. So the market leader could periodically change and people would only pay to use the most advanced and inovative branch, ensuring that development investment is paid for and then redistributed to everybody else.

</comment><comment><USER18> Is there anyway to display something in Linux without any color management? Can you access the buffer that gets read directly to the monitor?

I've wanted to know whether two colors I'm displaying actually get distinguished by the monitor or if the LIST maps then to the same output value.

</comment><comment><USER19> Many of my (Hollywood) clients need to do color grading and other accurate color work.

What platforms do they all use? Not Macintosh, despite its reputation for being the platform for "graphics professionals" (it was missing 10 bit/channel color until very recently). And not Linux, despite its use in render farms.

They use Windows 10 and HP DreamColor monitors. That's the only platform that works and works well for people who need to care about color.

<comment><USER20> I'm a colorist. Many of us do use windows, many use OSX, many more use Linux. Every major color critical application supports many types of LUTs and color management.

Further, HP Dreamcolors have tons of problems and aren't considered solid for color critical work (but are fine for semi color accurate stuff like intermediate comps etc). Color accurate work is done over SDI with dedicated LUT boxes handling the color transforms and the cheapest monitors being $7500 Flanders Scientific Inc 25" OLED panels.

<comment><USER21> _I 'm a colorist_

This sounds pretty interesting. What do you actually do, in layperson-y terms?

<comment><USER22> Sit in darkened rooms, spin 3 trackballs, and turn a few knobs to make pictures look pretty, mostly. DaVinci Resolve[1] and Baselight[2] are quite popular, take a look at the websites to give yourself an idea of what it looks like.

1\. [https://www.blackmagicdesign.com/products/davinciresolve/](https://www.blackmagicdesign.com/products/davinciresolve/)

2\. [https://www.filmlight.ltd.uk/products/baselight/overview_bl....](https://www.filmlight.ltd.uk/products/baselight/overview_bl.php)

<comment><USER21> Hah, thanks! I should have been a little bit clearer - not an ELI5-layperson, a layperson who has some vague handwavey idea of how video/film is made and once read a popular article about orange/teal contrast.

It's more stuff like 'what is it about this process that makes a dedicated colour specialist necessary?', 'what are the things things they're supposed to accomplish?', 'what are their technical and creative constraints/inputs/deliverables?', etc.

<comment><USER23> Check out this YouTube tutorial to see the kinds of things they're doing with those knobs and panels.

The inputs are roughly "An ordered sequence of footage clips" and the output is roughly "A final projection/broadcast/download-ready movie"

[https://youtu.be/ojjfhCrjDus](https://youtu.be/ojjfhCrjDus)

</comment></comment></comment></comment></comment></comment><comment><USER24> This all sounds interesting but I have 0 clue about what the author is talking about. Can somebody explain what is being talked about, what color calibration is and all? Not a designer, but a systems programmer. I do understand rgb, hsv colors but that's it.

<comment><USER25> The author cries about being not able to boot from the USB stick and that his ColorHug2 might be broken.

<comment><USER26> He wrote a pretty detailed blog article explaining his issues and how he got around them, this is all very useful information to someone who is looking to do something similar.

Spending €100 also isn't an insignificant amount, and for that price I would expect what I'm buying to work properly.

<comment><USER11> I don't understand that attitude, that's a pretty low price for a niche open hardware device.

<comment><USER15> It's not. An Eizo EX3 sells for 85 bucks here, a Spyder 5 for 95. A ColorHug2 amounts to 115. Since the ColorHug2 doesn't include the actual calibration software, it is equal to the EX3. Paying 35% more just for the "Open Hardware" label and then not being able to reap the expected benefits (better support etc.) doesn't sound like a good deal.

<comment><USER11> It is.

Better support? Open Hardware means Open Hardware, and nothing more - you get the access to the schematics, documentation, sometimes also right to produce similar devices by yourself. You can expect greater hackability, definitely, but "Open Hardware" sticker means nothing in terms of support or reliability. It might be better, it might be worse, you can't tell.

The price in such projects is directly related to the production scale. How many EX3s, Spyders and ColorHugs have been produced? Open Hardware projects (especially the equivalents of already available non-free devices) are often costlier because it initially attracts only the people who really care about its hackability, which makes the yields low, which makes the prices high, which further strengthens that relation, and the circle is closed.

With userbase kept small, most users usually keep the firmware/software support just right enough to scratch their own itches.

Please remember that hardware is not software, and open hardware comes with completely different set of challenges than free (open) software and when it comes to hardware, you often really need to pay extra for the freedom - <USER2> just with your time, like we were used to with early FLOSS, but also with your money. If you choose a project because of its "Open Hardware" sticker, it's really more than likely that it will be costlier and it will be rough at edges, because it's usually harder to roll with such projects than with closed competitors and the ROIs are usually way smaller too. That's just how it is and there's nothing surprising about it; if you care about openness, you have to accept it, otherwise it will never get better.

<comment><USER27> This is one of the more important comments here, imo. Open hardware != free software. As someone who's been in the unfortunate position of having to work with locked down chips on many occasions, just simply having access to proper documentation is awesome. It can take a lot of work and dedication to interface with open hardware, but we all benefit when the fruits of that labor are shared. This is a struggle we should all be willing to undertake.</comment></comment></comment></comment></comment></comment></comment>


Indus OS- World's First Regional Operating System - <AUTHOR> http://www.indusos.com/ <comment><USER2> Second, I think; North Korea has their own Linux distro. This, similarly, is an Android distro.

<comment><USER3> Cuba also as a Linux distro: [https://en.wikipedia.org/wiki/Nova_(operating_system)](https://en.wikipedia.org/wiki/Nova_\(operating_system\))

Edit: It is discontinued now, however</comment></comment>


Harrrah's Low Rollers Account for 80% of Revenue & 100% of Profits - <AUTHOR> http://ryanspoon.com/blog/2008/07/03/harrrahs-low-rollers-account-for-80-of-revenue-100-of-profits/ <comment><USER2> I think they've got the cause and effect backward there. Harrah's low rollers account for so much of their overall action because their casinos are so bad that high rollers avoid them. You can't simply assume that because all of their action comes from low rollers, they are pursuing some sort of brilliant strategy that nobody else is by not courting the high rollers (when they could easily go after both).

I'm guessing the 30% of low rollers that return very frequently are largely locals or near-locals (Harrah's has casinos in damn near every gaming jurisdiction in this hemisphere). Harrah's does a great job of catering to them.

Much of the other 70% would probably be mid-rollers, who I'd call value gamblers. They give the casinos just enough action to get some good freebies and nothing more. They end up spending less than if they had just booked the hotel and paid for food like a low roller does.

Harrah's does comp those two groups much better than anyone else. That also makes them the easiest to abuse comp program on the strip.

<comment><USER3> _You can't simply assume that because all of their action comes from low rollers, they are pursuing some sort of brilliant strategy that nobody else is by not courting the high rollers (when they could easily go after both)._

Sure, they could, but then they'd have to spend money to target that group. Is this any different to the high-end/low-end market segmentation of any other business?

I guess one of the reasons that they can open a Harrah's in damn near every gaming jurisdiction (New Orleans and South Lake Tahoe spring to mind as the two Harrah's I've been to) is that they can afford to build a fairly cheap casino without tainting their brand image. Compare that to, say, The Venetian, which can only open a new branch (in, say, Macau) if they're willing to spend the billions it takes to build something comparable to the Venetian in Vegas.

<comment><USER2> Harrah's has locations in every jurisdiction mostly due to merger and acquisition. They're basically a number of small gaming operators that got rolled into one.

Also, most of their casinos were not cheap when they were built. They are just old. The Imperial Palace (now the laughing stock of that part of the strip and soon to be bulldozed) was one of the nicest hotels around 50 years ago.

In fact, I don't think they've actually built a single casino since Loveman took over.

</comment></comment></comment><comment><USER4> Not even a surprise--thats how the casino biz works. High rollers are there for the resort to create buzz and bring people in. Just like Mercedes puts an SL convertible on the showroom to bring in gawkers, and some of them leave with a C class. Amex spreads rumors about a black card---it increases the wannabes signing up for credit cards.

<comment><USER2> That's not true at all. The casinos that cater to them make fortunes from high rollers. The average person on the Vegas strip blows something like $20 a day in gaming and has a total gaming budget of about $600 per trip. High rollers bet big enough that an unusual winning or losing streak can shift a casino's quarterly results by 5% or more.

And Amex does have a black card, it's not a rumor (though there are many untrue rumors about it). I know a holder, who not coincidentally is how I know so much about comp programs and how casinos treat high rollers.</comment></comment>


Cyber Security-Everyone's Responsibility - <AUTHOR> https://medium.com/@cyware/cyber-security-everyones-responsibility-73a498d3e88 <comment><USER2> The Internet of Things will take an ugly turn as hackers start breaking into people's thermostat to steal money from their bank account because the thermostat was connected to their garage door, which was connected to their car's system, which allowed direct payments from their bank account. The threat landscape is changing everyday! We win if we change our e-behavior with it, they win if we don't.

</comment><comment><USER3> While this is true, I just feel like there isn't enough help out there. I know a few people who are security conscious but are not sure what they really need to do to "protect" themselves. Loads of articles out there but most of them require a certain level of familiarity with tech which is not very appealing to the uninitiated.

<comment><AUTHOR> Well true the help needs to be increased. Programs like National Cybersecurity Awareness Month need to be popularised more. Also most people are not updated with what's happening and what news threats are popping up everyday.

</comment></comment><comment><AUTHOR> Yes, we are entering an age where cybersecurity cannot be just a resposibility of the experts. We need all people in the organization to be aware and stay secure on their part. One bad fish spoils the entire pond. Similarly, one loophole is enough for entire systems to go down.

</comment><comment><USER4> Cybersecurity awareness is foundation of digital world. all of us are living in digital world now and now IoT being around, we are further being digitalized. All of us must be aware on Cyber security basics and concepts. Prevention and awareness is better than any cure.

</comment><comment><USER5> Growing Internet activity created demand for cyber security in abundance. The least thing we can do to secure all the information is by increasing awareness among the users and create a user friendly, foolproof environment to secure our personal data.

</comment><comment><USER6> Yes, in this cyber-addicted world, it is the least you can do to protect yourself. So, take charge and be responsible while you do your 'things' in the Internet.</comment>


TED Talk: Elaine Morgan says we evolved from Aquatic Apes. - <AUTHOR> http://www.ted.com/talks/elaine_morgan_says_we_evolved_from_aquatic_apes.html

<comment><USER2> Can I flag this for being stupid? OK I am instead going to use it as a teaching moment.

Remember hacker news disease? The symptom of having a strong opinion on topics you really shouldn't be having an opinion on? OK so if you find yourself even remotely agreeing with the Aquatic Ape theory, do yourself a favor and research the counter evidence, it's easy to find and obvious enough to convince yourself.

It's just a shame that non-stupid ideas have to share TED with this. This and that rambling talk about the dangers of robots (OMG roombas!) are so far the worst things I've seen at TED.

Lets just hope that TED doesn't devolve from here.

<comment><USER3> First of all, this theory is not stupid.

Humans belong to a small number of animal groups that display vocal learning.

Vocal Learning Animals have a special "language" gene ( FoxP2 ), that gives them the ability to learn and imitate sounds they hear.

Among mammals only Dolphins, Whales, Seals, Elephants and Humans have FoxP2 gene. All of them are aquatic animals or have aquatic ancestors.

<comment><USER4> I think the notion of a (single) gene giving the ability to learn is already flawed.

<comment><USER3> People with mutation in this single gene lose the ability for vocal learning and speech. How do you explain that?

<comment><USER4> I would guess it is responsible for one crucial part for the "speech apparatus", but that doesn't make it the speed apparatus. A car without the ignition loses the ability to drive, yet the ignition hardly seems the most important part in the car.

I feel the same uneasyness when I hear "hormone X is the cause of ability Y" - sure, maybe if Hormone X is missing, ability Y is missing. But Hormone X does not manufacture Y, it is just a messenger that makes something else produce Y. Likewise I suspect speech is possible because of the combined effects of several genes. If one is missing, maybe the system fails - but perhaps it could be replaced with something else, given time.

Sorry, it is late, I feel unable to express myself more clearly atm.

<comment><USER3> An ignition indicates that the car has an internal combustion engine.

The speech-enabling gene FOXP2 indicates that there are brain circuits that make vocal learning possible.

The question is why this gene is present in aquatic mammals and is absent in terrestrial apes

<comment><USER4> "The speech-enabling gene FOXP2 indicates that there are brain circuits that make vocal learning possible."

That's kind of what I meant: surely that speech gene is not a gene that describes a whole brain?

"The question is why this gene is present in aquatic mammals and is absent in terrestrial apes"

Doesn't that refute the "speech gene" theory? Surely some terrestrial apes also communicate by "speech"?

I don't know enough about genetics to confidently discuss this more. Presumably if several species share a gene, they are likely to have a common ancestor?

<comment><USER3> "Vocal learning, the substrate of human language, is a very rare trait. It is known to be present in only 6 groups of animals: 3 groups of birds (parrots, songbirds, and hummingbirds) and 3 groups of mammals (bats, cetaceans[whales/dolphins], and humans).

All other groups of animals are thought to produce genetically innate vocalizations.

To understand this concept, it is important to distinguish vocal learning from auditory learning. Auditory learning is the ability to make sound associations, such as a dog learning how to respond to the sound "sit". All vertebrates have auditory learning.

Vocal learning is the ability to imitate sounds that you hear, such as a human or a parrot imitating the sound "sit". Currently only vocal learners have been found to have forebrain regions dedicated to vocal learning and production of these learned vocalizations. Vocal non-learners only have been found to have non-forebrain vocal regions responsible for the production of innate vocalizations." <http://www.jarvislab.net/Evolution.html>

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> It should be noted that the Aquatic Ape theory is controversial at best.

<comment><AUTHOR> Did you watch the talk? That was clear from the theme.

<comment><USER6> I'm in no position to judge the evidence, but I'm glad to see that Elaine Morgan is still at it.

</comment></comment></comment><comment><USER7> This isn't particularly new -- the theory has been around for more than 60 years.

Anthropologists certainly are aware of the theory. It comes up all the time in student papers and discussions. My first encounter with it was when I was a grad student TAing Anthro 1.

There does not seem to be any evidence in the fossil record that supports the theory.

</comment><comment><USER8> If there is is evidence it will be accepted for review, if not it will be rejected..... simple.</comment>


TinyMCE 4.2 released with Image Tools - <AUTHOR> http://blog.tinymce.com/2015/06/25/tinymce-4-2-with-image-tools/

<comment><AUTHOR> Contained in this new release is a feature yet to be seen in any open source WYSIWYG editor. Thanks to our efforts with our new colleagues at Ephox, we are able to provide you with image editing capabilities in our TinyMCE Image Tools feature! You will now be able to perform a wide range of image editing tasks all within the rich text editor.

</comment><comment><USER2> Wonderful new addition!

</comment><comment><USER3> Awesome stuff.</comment>


Ask HN: Review my app: GTimeReport - <AUTHOR>

I've created a web application that generates time sheets from Google Calendar.
It's sprung out of what I needed for my own usage, but I'd like it to be useful for others as well. It's been up for 2 months or so, and I've been updating it during the time, adding features and polishing.
My aim with it is to provide a super easy to use time report creator that makes time reporting less tiresome. Just a few clicks and be done.
What do you think of it?
http://www.gtimereport.com <comment><USER2> It would be nice if you put up a screencast or some screenshots of the workflow. You have a lot of content on the site that seems to describe it, but I'd really like to see what the process looks like without needing to actually log in / link with my Google account.

Overall I really like the idea and this seems like a useful tool. Good luck!

<comment><AUTHOR> Will do that. Thank you! :)

Btw, I thought I was asking at the "ask" page, but the post went to the news page. Did i miss something when I posted?</comment></comment>