The Most Popular Language for Machine Learning - <AUTHOR> https://www.ibm.com/developerworks/community/blogs/jfp/entry/What_Language_Is_Best_For_Machine_Learning_And_Data_Science?lang=en <comment><USER2> Ugh, indeed.com data.

Was not a great place to find data science jobs during my last job hunt, but that might just be personal experience.

Beyond that, wouldn't there be some bias - in that the jobs that don't get good candidates are going to have to be re-posted more often to maintain their visibility, leading to an over representation of undesirable roles in the data?

<comment><USER3> What did you find to be better than indeed? I would have thought it a safe bet since it aggregates from many sources.

<comment><USER4> I guess indeed doesn't represent how many people write ML code in large companies like FB, Google, MSFT, IBM. I think another good metrics would be number of LoC in ML projects on github.

<comment><USER5> Maybe also stackoverflow.

</comment></comment></comment></comment><comment><USER6> Everytime a "what's the best language for Machine Learning/Data Science?" thread pops up, it always devolves into a flame war between "real data analysts use R" and "Python has thousands more libraries!". (most recent example 16 days ago: [https://news.ycombinator.com/item?id=13110230](https://news.ycombinator.com/item?id=13110230) )

My response is always use-the-most-appropriate-tool-for-the-job-dammit and don't pigeon-hole yourself into one language, since each language has their pros and cons. I am very tempted to write an HN autocomment bot at this point. (in Python instead of R, of course, since that's the _most appropriate tool for this job_ )

<comment><USER7> Are R and Python really your only choices? Basic competency in machine learning is on my todo list, and I don't relish the thought of using either language.

<comment><USER8> There is a pro for Python in that it makes machine learning _really easy_ , or at least incredibly accessible. You can write and test classifiers in about 5 lines of Python using scikit-learn. The second point is that virtually all the latest deep learning packages come with Python frontends by default nowadays. For stats you could also use SPSS.

The other advantage of Python is that as a scripting language it's very powerful for data wrangling and pre-processing, without needing all the boilerplate that e.g. C++ would require.

<comment><USER9> I have played around with scikit-learn and love how simple and easy it is to work with, but the story for scaling it doesn't seem super straightforward - is this something anyone here has experience with?

I built a recommendation system in Spark earlier this year that used terabytes of input and would run it on a 40 node EMR cluster so it took less than half an hour. It wasn't trivial to make it run in a clustered environment, but it wasn't very hard either.

<comment><USER10> Out of curiosity, were you using spark-scala or pyspark?

<comment><USER9> I was using scala

</comment></comment></comment></comment></comment></comment><comment><USER11> You seem to get significantly different results when you ask people directly what they're using:

[http://www.kdnuggets.com/2016/06/r-python-top-analytics- data...](http://www.kdnuggets.com/2016/06/r-python-top-analytics-data-mining- data-science-software.html)

I don't really like the idea of looking at search correlations to infer popularity in a given field. People who use R might have a higher level of education, resulting in search results that are are narrower and more focused than Python users, or simply be more likely to call it "AI" or "statistical learning" or something of that nature. Or it may be that people learn a language or tool because it is useful in a field, whereas people who use a more popular tool might tangentially search for a given combination, even though they're not really in that field or doing any real kind of ML work.

Although KDNuggets survey is self-selected, which is inherently an unsound method, but it's not like the google search results are really a random sample, either.

<comment><AUTHOR> I fully agree on using search correlations. Fortunately, this is not the case here. This is not about any search. This is about actual keywords occurring in actual job offers. I thought I explained it clearly, sorry if you missed it.

<comment><USER11> Oh, damn. I wish hacker news let you delete comments.

<comment><USER12> If the comment is still within the time it can be edited, you can, unless it has a response. In that case, if you want the comment effectively gone, you can edit the comment to remove the content. As a courtesy to the reader, you can leave a "[self-deleted]" or some such to indicate what happened.

</comment></comment></comment></comment><comment><USER13> Half of machine learning is data wrangling. Python is so easy to use and elegant, that it _feels good_ when you're using it. When was the last time you enjoyed data manipulation in R/Java/C/etc?

<comment><USER14> Anything for PRODUCTION data processing that doesn't have strong/static typing is a crappy experience.

That being said, for adhoc work, with Scala you get the best of both worlds.

Python is certainly approachable from early programmers, and R from mathemticians/business folks. Sadly there are more early access libs here, but all the popular stuff is aviailable in the languages above.

C++ is really efficient, but it's a bit unforgiving.

<comment><USER15> Why are you doing data wrangling in production?

</comment></comment></comment><comment><USER16> After a year of experiments i realized that machine learning and big data pipelines are inseparable. So at first i was thinking R/Python is the greatest. And it might be until you need to do more that a few isolated models. At that point i reverted to building the pipeline parts with Spring Java + InMemory DataGrids because there is so many options.

<comment><USER17> There's certainly a tension between quickly prototyping something in R/Python w/ limited data vs. making a ML system that scales once proven useful.

I believe the majority of data science jobs today are involved on doing only the first (to gather pontual insights) and dropping the ball on the second since it involves a lot more software engineering, and those jobs are currently being fulfilled by those without this skill.

I foresee this being a source of frustration in the next years for companies that fell for the data science hype, once they figure out it takes significant investment and commitment to build intelligence into their systems, or even curate high-quality data to do it right in the first place.

<comment><USER18> I cannot agree more. Took us a year to figure out the whole process. Especially updating and maintaining the models in production can also be a handful.

</comment></comment></comment><comment><USER19> spoiler: "Python is still the leader, but C++ is now second, then Java, and C at fourth place. R is only at the fifth rank."

<comment><USER20> It's important to distinguish between the API language and the core language used for large-scale computation. Every single deep learning library exhibits that division. The API language(s) indicate the communities the lib seeks to serve. The core languages are lower-level, faster, and help optimize on the hardware. The core languages are always C/C++/Cuda C. The API languages tend to be Python, Java, Scala, R. Conflating the API languages with the computing cores is comparing apples to oranges. [http://imgur.com/a/Z6fGr](http://imgur.com/a/Z6fGr)

<comment><USER21> The higher level languages are also used to munge the data into lib-readable format. Python and R's standard libs are especially good at putting out various tabular and database formats in preparation for ML input

</comment></comment></comment><comment><USER22> python is that nice mix between scripting and programming, as well as having all sorts of ugly hackish modules that make it easy to do otherwise complex tasks with all sorts of data.

Since the hard part about ML is more about manipulating the data into something manageable, this makes python well suited for the task.

If Tensorflow for .NET had come out sooner, I might have jumped on that as C# is a nice balance of performance vs ease of manipulating data, but now all my code snippets are python so I'd need a large probject before the benefits of changing over push me that way...

</comment><comment><USER23> I wonder how much of the job details are actual vs. aspirational. For example, in many of our R&D job listings we list Python and Hadoop as desired skills, when in reality those are emerging needs rather than the day to day work which is mostly SAS.

</comment><comment><USER24> If Rust is a replacement for C/C++, why isn't it being adopted for machine learning?

<comment><USER25> Nobody uses it, period.

IMHO, ML people are pretty pragmatic, possible due to the fact ML itself a mixed paradigm with people from different background, so they don't hold religious belief towards programming languages comparing to some pure CS background folks.

<comment><USER26> For the record, I've used rust for machine learning, and it was great. It isn't for "religious beliefs" \- both myself, an engineer, and a data scientist on my team, had great results using rust.

Rust is entirely practical and well suited for the task. And yes, people use rust.

<comment><USER25> That sounds great.

What I am objecting here, are those annoying cool kids try to sell their flashy green solutions, like 'A new machine learning library written in Rust' to other people, because 'yeah, Rust is better programming language than C++'.

That is what I called 'religion belief', for those people who don't really care what problems they solved, they just naively assume they are better because of the language they are using is different.

<comment><USER26> Maybe they're not annoying cool kids but people doing work in a new domain and trying to share the benefits.

I don't think people are saying that their solutions are better just because they're in rust. Maybe somewhere someone is, but the majority of what I see doesn't really sound like that.

</comment></comment></comment></comment></comment><comment><USER27> We use python for all of our local and back-end ML processing pipeline. It's actually the reason our OS/Architecture intermediary was written in Python as well.

</comment><comment><USER28> Lua is ranked surprisingly low, when I was getting started it seemed easiest to find good code examples in Torch (e.g. neuralstyle, char-rnn).

<comment><USER20> Torch is a great library, and Lua a fine language, but they are competing against ecosystems built on the world's largest languages.

</comment></comment><comment><USER29> For those who use R, how hard is it to search for material on the web? I imagine in some cases the search engine may treat the R as a typo?

<comment><USER30> That isn't an issue, Google's pretty smart. The actual documentation is terrible though, and you do not have the depth of StackOverflow posts to make up for R's inscrutable and sometimes non-existent errors.

</comment></comment><comment><USER25> TL:DR, it is Python. And everyone already knows it.

<comment><USER31> And Python 2.7 to be specific. All the big framework releases are still 2.7 first. You have to wait for ML researchers to be bothered with 3.x.

<comment><AUTHOR> Not sure where you get this from. The most popular machine learning framework, scikit-learn, runs fine in 3.5. I am also using xgboost with Python 3.5 (yes, xgboost is a major open source for machine learning, just look at what framework is used by most Kaggle competition winners). TensorFlow, mxnet also support 3.5. I would have agreed with you a year ago, but there has been a major shift in use from 2.7 to 3.5 in 2016.

edited for typo.

</comment></comment></comment><comment><USER32> Javascript is mentioned in the charts but as of this writing has no hits on the discussion. Is anyone using Javascript or Node for ML?

<comment><USER28> As far as I can tell there are no great options with all the necessary features - extensible API and GPU support especially. My team is Node based but went with Lua for the ML parts because of this.

</comment></comment><comment><USER33> Super easy to call R scripts from python - can use rpy2 to send dataframes from R to pandas or can just run an R script that outputs a csv to a folder and then read that in python...

Legit 3 lines

      import rpy2.robjects as robjects
      #
      r_source = robjects.r['source']
      r_source('myscript.R')
      #
      print 'r script finished running'

</comment><comment><USER34> I would like to see how Octave or Matlab would do in these results.

Before clicking on the link, I thought it would be Python, R, or Octave.

<comment><AUTHOR> You can add them to the queries I used to see the results. Look for pointer in the article. Reason I didn't include matlab was that I didn't include any commercial product. To your point I could have included Octave still. I just did it, and Octave is at 0%: [https://www.indeed.com/jobtrends/q-python- and-%28%22machine-...](https://www.indeed.com/jobtrends/q-python- and-%28%22machine-learning%22-or-%22data-science%22%29-q-R-and-%28%22machine- learning%22-or-%22data-science%22%29-q-Java-and-%28%22machine- learning%22-or-%22data-science%22%29-q-Javascript-and-%28%22machine- learning%22-or-%22data-science%22%29-q-C-and-%28%22machine- learning%22-or-%22data-science%22%29-q-C++-and-%28%22machine- learning%22-or-%22data-science%22%29-q-Julia-and-%28%22machine- learning%22-or-%22data-science%22%29-q-scala-and-%28%22machine- learning%22-or-%22data-science%22%29-q-Octave-and-%28%22machine- learning%22-or-%22data-science%22%29.html)

</comment></comment><comment><USER35> C/C++ users: how do you use it for in ML applications? Exploratory analysis? Converting research to production code? Do you use existing libraries, or do application specifics (or other reasons) require that you write your own?

<comment><USER36> I use a tiny bit of python, a little more LUA, and a TON of C++ in the machine learning work I do. Things like opencv, fbthrift, folly, boost, fblualib, and thpp make writing this sort of code in C++ very time efficient and if you know what you are doing it will end up performing much better than the alternatives. I only use python for some light scripting, data collating and reformatting type tasks, and LUA due to using Torch as my Neural network framework of choice.

</comment></comment><comment><USER37> The positions of the graph legends are terrible - they overlap with parts of the most interesting data points.

If the author or an editor of the article reads this: It would help a lot if you could move the graph legends to the left. Thanks!

<comment><USER6> The source charts are from an interactive explorer on Indeed and not staticly generated: [https://www.indeed.com/jobtrends/q-Python-and-%22deep- learni...](https://www.indeed.com/jobtrends/q-Python-and-%22deep- learning%22-q-R-and-%22deep-learning%22-q-Java-and-%22deep- learning%22-q-Javascript-and-%22deep-learning%22-q-C-and-%22deep- learning%22-q-C++-and-%22deep-learning%22-q-Julia-and-%22deep- learning%22-q-Scala-and-%22deep-learning%22-q-Lua-and-%22deep- learning%22.html)

</comment></comment><comment><USER38> I think it would have been more interesting to constrain the query to must have PHD. There's probably a lot of grunt work that needs to be done in other languages that aren't really specific to machine learning. Eg, python is fantastic for data wrangling, but PHDs with R probably earn 2x what masters/bsc data wranglers do. But you probably need like 4x of the latter in terms of staff.

For the person who downvoted me, I wasn't saying R is better than Python. I was just saying that if you have just R, you're probably not doing data wrangling..

What this post is completely failing to capture is the exceedingly high value work in machine learning versus the typical work any skilled undergrad can do.

<comment><USER39> Having spent last 4 months interviewing at several companies for ML roles, I can assure you that no one doing PhD in Computer Science (Machine Learning, Vision, Data Mining) uses R.

I also dont understand what you mean by "exceedingly high value work". For both production as well as research (ICML/NIPS/CVPR) the languages used are in most cases Python/C++/Lua.

Also Stats PhD (who I believe are the sole users of R) aren't typically hired into Machine Learning roles.

R is okay for wrangling tabular data, applying statical model for hypothesis testing and generating pretty charts for papers. But not suitable for state of art Audio, NLP, Vision or Reinforcement Learning.

<comment><USER6> > Also Stats PhD (who I believe are the sole users of R) aren't typically > hired into Machine Learning roles.

FWIW, I used R for my undergrad, and still use R for personal projects afterwards. (with modern R, dplyr/ggplot2 are an order of magnitude easier to use, in my opinion, than the Python equivalents)

</comment></comment></comment><comment><USER40> In reality, most production code is C++ for machine learning backends.

</comment><comment><USER41> Guys for deep learning we all know it's cuda under the hood.

<comment><USER6> You still need a programming language to interface with CUDA for deep learning.

<comment><USER42> The article confuses users with developers.

<comment><AUTHOR> Agreed. I welcome suggestions on how to distinguish both using keywords in job offers.

<comment><USER42> Carefully? Possibly in some non-automatic fashion?

<comment><AUTHOR> I don't have the data. I use indeed.com trend queries. All we can do is to select which keywords to use.

<comment><USER42> That sounds like your problem and not mine. Right now the article reads like "who is your ISP" and 30% of people responded "Netgear".

<comment><AUTHOR> If you think job offers are written in such a dumb way that netgear could be regarded as an ISP then I would agree with you. What evidence do you have of this?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER43> wonder why there's no mention of golang... perhaps the libraries are not good enough? golang is great for distributed systems.

<comment><USER44> Well, we're getting there. If you're interested in ML with Go, there is Gorgonia: [https://github.com/chewxy/gorgonia](https://github.com/chewxy/gorgonia)

I've been using it for several years to build pipelines</comment></comment>


Flutter or React Native - <AUTHOR> Which one is the best ?? <comment><USER2> This ultimately depends on your situation. I would have said Flutter since it is 1.0 and RN is still in beta. But my main problem here is that you will quickly run into difficulty in attempting to use C/C++ libraries in Dart.

Dart has beta support for native FFI but it is not stable unlike RN. Right now, they instead have 'Platform Channels' which has a significant overhead when passing messages back and forth compared to a native FFI. The issue is still open on Github: [https://github.com/flutter/flutter/issues/7053](https://github.com/flutter/flutter/issues/7053)

I can reuse and port my JavaScript/TypeScript projects into mobile very quickly thanks to React Native. Flutter on the other hand, requires learning Dart which depending on your use-case makes sense if it is a new project, not so if you have invested in JavaScript.

Perhaps once Dart gains a production-level C/C++ FFI, then I can seriously look at both Dart and Flutter again.

</comment><comment><USER3> Depends on what your use case is, what you want to do, what your past experience is.

</comment><comment><USER4> flutter since it has better tooling support. react native requires expo without which the setup is not as good.</comment>


Hacker Monthly Issue #5 - October 2010 - <AUTHOR> http://hackermonthly.com/issue-5.html

<comment><USER2> [http://www.softwarebyrob.com/2010/08/18/why-free-plans- dont-...](http://www.softwarebyrob.com/2010/08/18/why-free-plans-dont-work/)

<http://web.stonehill.edu/compsci/History_Math/math-read.htm>

[http://steve-yegge.blogspot.com/2007/06/rich-programmer- food...](http://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html)

[http://tom.preston-werner.com/2010/08/23/readme-driven- devel...](http://tom.preston-werner.com/2010/08/23/readme-driven- development.html)

<http://www.defmacro.org/ramblings/latex.html>

[http://encosia.com/2010/08/18/dont-let-jquerys-document- read...](http://encosia.com/2010/08/18/dont-let-jquerys-document-ready-slow- you-down/)

<http://symkat.com/35/ssh-tips-and-tricks-you-need/>

[http://pieratt.tumblr.com/post/977179815/in-praise-of- quitti...](http://pieratt.tumblr.com/post/977179815/in-praise-of-quitting- your-job)

[http://www.kadavy.net/blog/posts/design-for-hackers-why- you-...](http://www.kadavy.net/blog/posts/design-for-hackers-why-you-dont-use- garamond-on-the-web/)

[http://alexisohanian.com/keep-calm-carry-on-what-you- didnt-k...](http://alexisohanian.com/keep-calm-carry-on-what-you-didnt-know- about)

<comment><USER3> CSS of visited links here reminds me that i miss half of the good articles...Not sure if i am on higher side..

<comment><USER4> The weekly newsletter I put out might help with that - <http://www.hackernewsletter.com>

</comment></comment></comment><comment><USER5> Our "Why Ugly Design Converts" was included in printing #2.

The publisher, Cheng, reached out to us - asked permission, and gave us the opportunity to make edits.

I assume he does this with everybody, so there's certainly no exploitation of the article writers .

Next, he went through our blog article and made it beautiful with more thought and care than even we had.

I feel indebted to Cheng and his team, we've gotten several mentions from other folks we work with about the article, and being included has been a fantastic marketing tool to show to prospective clients.

Let me sum up:

a) I was left with the impression that Cheng's motivations are built from passion, and his process is incredibly transparent.

b) I support the magazine to give other members of this community the opportunity to be featured when they put together something that the community finds compelling.

c) Maybe Cheng should give the top 100 stories in a given month the opportunity to BUY slots in the magazine. The amount of work he did to make our crummy blog writing look professional, and the fact that it's a brilliant marketing tool would have easily been worth the price of admission. No conflict because he's pulling from top voted stories anyways.

Thanks Cheng for the work you continue to do!

Edit: Our "Why Ugly Design Converts" article - <http://bit.ly/a2w9xO>

<comment><USER6> Sigh... Proper link: [http://conversionvoodoo.com/blog/2010/04/increase-your- conve...](http://conversionvoodoo.com/blog/2010/04/increase-your-conversion- rate-by-making-your-site-uglier/)

<comment><USER5> Out of curiosity why didn't you like the bit.ly link? I put in that format purely for convenience, readability.

<comment><USER6> Because url shorteners break the web[1][2].

[1] [http://en.wikipedia.org/wiki/URL_shortening#Criticism_and_pr...](http://en.wikipedia.org/wiki/URL_shortening#Criticism_and_problems)

[2] <http://news.ycombinator.com/item?id=1536662>

</comment></comment></comment></comment><comment><USER7> It makes me sad and fearful for the future of our industry when someone charging the equivalent of a cup of coffee for something that has a multiple of the value of said coffee is met with dozens of complaints about how it should be free, plus people actively trying to sabotage his efforts (sorry to mmphosis if the list of links wasn't meant in that way).

<comment><USER8> It's just this audience; hackers who bill their clients $75 / hr will spend an hour trying to avoid paying $3 for something online that they want, while sitting in a coffee shop and drinking a latte and a muffin they just paid $8 for. It makes no sense.

<comment><USER9> It's not the money that matters to a hacker, it's the accomplishment of surpassing a tangible value that does.

<comment><USER7> What does "surpassing a tangible value" mean?

</comment></comment></comment></comment><comment><USER10> It's funny - when they are printing a magazine I'm like wow great because I feel they've added value but when they're charging for the digital PDF (and it's a nothing charge let's face it) I feel like they are almost exploiting other's creativity (who I assume has been donated with permission?).

<comment><USER11> I'm a bit torn about that, but I don't feel exploited. Permission was asked and granted, and authors now know the deal. Design is _hard_. Curation and editing are _hard_ work. I bought 5 copies of #1 for my mother but haven't bought other issues.

I think the price of the paper form is too high, but the production values and low run means he's barely clearing a profit on them. It all comes from ads. That's his call, and, frankly, we are a pretty rich demographic. If Cheng Soon can cross HM over from vanity press to a serious hacker journal, this will become a very important magazine on the order of "Old Wired" and "Old DDJ".

<comment><USER12> I agree, but those $3 broke my heart. I am cash strapped down to $.50 and this magazine was amongst the things I look forward to and in fact I used to make people read it despite their too obvious protests. The sad part is that now it can't work that way. I cannot refer them on due to their paywall (yes I know it's online, but the PDF was really pretty and I loved passing it around in a package to my friend) and that's just sad.

Moreover, it was neat convenient and made me loyal to them I can't find a better branding exercise than that.

Sigh. I really wish that it didn't have to be this way, but perhaps they could try an alternate model of revenue (after all they're hackers). I would love to help these guys out in this regard.

<comment><USER13> Just bought you a copy and emailed it to you. Enjoy!

<comment><USER12> I don't know what to say right now.

I'm just touched.

</comment></comment></comment></comment></comment><comment><USER14> Not sure about the decision to charge for the pdf, considering that they're reusing free content.

<comment><USER15> The free content is still on the internet, you're paying for the presentation and the format.

From a business point of view, it may be good or bad. I wouldn't like to guess.

On the one hand, they might sell more copies. Without a free PDF cannibalizing sales, it will be much easier to convince people to buy something. On the other hand, it will decrease their circulation, which will devalue their ads and job listings.

<comment><USER16> Decreasing circulation doesn't necessarily devalue their ads or job listings. Why do you think so?

<comment><USER17> Well, for one, the 'reach' reduces drastically.

</comment></comment></comment></comment><comment><USER18> I did feel better about this project when the pdf was free, the idea being that the hard copy sales (that was half the point right?) and the ads with wider circulation paid for the pdf.

Its pretty much parallel to the problems face by newspapers online, put up a pay wall and lose your readership plus your advertisers, or keep it free and get by on much thinner margins than previously but at least be in business still.

</comment><comment><USER19> I was afraid there'd be no content for this month but industry news and gossip - I was wrong. Good!

</comment><comment><USER20> Just wondering, how do they chose which comments go in this? How many comments per issue, and are they related to the articles or just the best comments that month? If so, how do they put them in context? I'm not $3-worth of curious, but I would be interested...

<comment><USER21> We chose the best comments (voted by HNers) of the selected articles (also voted by HNers). We then asked for permission to reprint the comment (yes, even comments). If we did not hear from the author or got declined (seldom happens), we moved to the 2nd best comment of that article.

</comment></comment><comment><USER22> Great.. definitely worth it. Makes these articles a lot more fun to read.

</comment><comment><USER23> I love reading these on my kindle DX. Thanks again Hacker Monthly! :)

</comment><comment><USER24> Are the previous digital versions no longer going to be free once the next version is released? The digital edition for September is still $3.

<comment><USER21> Only the first 3 issues are free.

</comment></comment><comment><USER25> $3?! you're aving a giraffe!

<comment><USER26> Lol that's cockney rhyming slang for those who are confused. Giraffe rhymes with laugh.

<comment><USER27> Classic cockney is rhyming but two-steps removed. ie.

* cocaine -> charlie -> bob marley -> bob

* telephone -> dog n' bone -> dog

* friend -> mate -> china plate -> china

see: "The construction involves replacing the common word with a phrase of two or three words, and then in almost all cases, omitting the original rhyming word, in a process called hemiteleia"

from: <http://en.wikipedia.org/wiki/Rhyming_slang>

I have heard all three of those examples in common use (especially in South Africa)

<comment><USER28> I like "bunse" from "bunsen burner" from "nice little earner", i.e. money.

But from googling just now, it seems it might not be a proper one.</comment></comment></comment></comment>


First URL active once more - <AUTHOR> http://first-website.web.cern.ch/blog/first-url-active-once-more

<comment><USER2> And it is already back down again because of all the traffic. The archived version is still up at: [http://www.w3.org/History/19921103-hypertext/hypertext/WWW/T...](http://www.w3.org/History/19921103-hypertext/hypertext/WWW/TheProject.html)</comment>


Where Does Affirmative Action Leave Asian-Americans? - <AUTHOR> https://www.nytimes.com/2019/08/28/magazine/affirmative-action-asian-american-harvard.html <comment><USER2> This article is poorly written. Early on the reporter really steps her foot into it by practically insulting the kid. Then he bristles at her and responds as if he's being attacked. It's cringing and I'm learning more about these two people than I needed to.

</comment><comment><USER3> These are clearly driven individuals who will excel anywhere. In fact when you consider grade inflation, they'd be better of going taking a scholarship from a flagship state university. Especially for STEM majors.

Affirmative action is an attempt to break the cycle of educational under- performance for certain groups in America. It's a blunt tool that needs to be improved, not discarded.

<comment><USER4> The problem I see is that higher education never really improved performance of anyone. The way it historically worked was that higher education institutions would admit only the smartest people around and so only the smartest people would come out of them, too. Smartest roughly equates to highest performing.

Employers would then be able to use the certificate from those institutions to filter for only the smartest people since those were already filtered for to begin with to be eligible to get such a certificate.

If you put people into higher education that ain't already the smartest, you will get people coming out of higher education that ain't the smartest and employers will learn the hard way that the certificate in question is worthless and will begin to filter through other means. Now those students sit on debt they have no way to pay off, since high paying employers will find the smartest people around some other way.

</comment></comment><comment><USER5> I'm no fan of judging people by the color of their skin, why not use family income?

This way it's apparent who is getting high scores due to tutoring and who is naturally gifted.

<comment><USER6> They already do that too, but it can't solve the racial distribution problem which colleges care a lot about: [https://upload.wikimedia.org/wikipedia/commons/c/c6/1995-SAT...](https://upload.wikimedia.org/wikipedia/commons/c/c6/1995-SAT- vs-Income-Education.png)

<comment><USER7> Originally I was against the idea of affirmative action but I've came around to it, but I still don't understand the need to solve for racial distribution rather than socioeconomic distribution.

In my mind affirmative action's goal should be to break the cycle of poverty, where smart kids have poor/uneducated parents which holds them back and they grow up to be poor/uneducated themselves and so on.

It's my understanding that this problem disproportionately affects minorities (in particular black people), so solving for socioeconomic would "self select" (I don't know the proper term) for black people and avoid the issue of "Asian kid with poor/uneducated parents passed over for black kid with richer/educated parents" right?

Or is that just not a real issue? The article seems to mention pluses and minuses (e.g. "two college-educated parents (minus) from a major American city (minus) with aspirations to study either computer science (minus)") so it could be that the above hypothetical would never happen.

I don't know, I'm just struggling to understand why colleges would care more about solving racial distribution rather than socioeconomic distribution.

<comment><USER8> I read a paper on this - if you cut the data by social-economic status, the number of 'qualified' black students would be just over 2%, and I think the colleges are targeting 8% of each incoming class (they don't call it a quota but it is, since it stays consistent each year regardless of overall applicants' demographic changes).

</comment></comment></comment></comment><comment><USER9> they got rid of affirmative action in california. the asians were the only minorities to see growth in the number of acceptances to uc. however, it was no where near the rate of growth of their population demographic. most of the new spots went to foreign students. once again the faculty went back to being exclusively white male. also all state government jobs hiring practices were no longer scrutinized for discrimination. minority elected officials in california clearly have difficulties dealing with their own state government.

nobody bothered to check whether eliminating affirmative action actually helped.

<comment><USER10> This is why so many white-centric right wing groups very much want affirmative action abolished, because it is a massive advantage to the politically powerful and connected, who are mostly white in America.

Lets just say its hardly like they are out for justice for all races...

<comment><USER11> "Politically powerful" are obviously mostly white because the population of the US is mostly white.

There is about 10% of black reps in the House which is well in line with the overall population.

And we also just have had a black president.

</comment></comment></comment><comment><USER12> Leave Asian Americans?? Asian Americans have the highest income of all groups of Americans.

<comment><USER13> > _Asian Americans have the highest income of all groups of Americans._

Amazing how people in the United States nowadays are sensitive to stereotyping and racism ... except when it comes to Asians.

<comment><USER14> And caucasians.</comment></comment></comment>


Adam? …is there a reason your laptop is in the fridge? - <AUTHOR> http://www.kempa.com/2006/10/02/adam-is-there-a-reason-your-laptop-is-in-the-fridge/ <comment><USER2> I would recommend just chilling the drive, and using an external enclosure to read the data. Less risk of killing a whole laptop's worth of electronics with condensation, less warmup time wasted in booting, etc.

<comment><USER3> He mentioned that the reason he didn't do that was to avoid voiding the warranty.

<comment><USER4> i'd be shocked if refrigerating it didn't

<comment><USER5> If you manage to get the temperature in your fridge up to 10°C you are good[1].

[1] <http://www.apple.com/macbookpro/specs.html>

<comment><USER6> That's pretty easy really. It's usually around 35-40 F, 10C is 50F, so just leaving your door cracked will probably accomplish that.

In fact I suspect the Macbook's own heat would keep it above that while copying the files.

</comment></comment></comment></comment></comment><comment><USER7> My wife's iPod broke last year. The hard disk was clicking and the device just wouldn't boot up. We took it to the genius bar and they helpfully suggested we buy a new iPod. Tough break.

Then somehow we found out about this cool the hard disk technique. We put the iPod in a ziplock bag and in the freezer overnight. The device was working perfectly fine while backing it up, but as per the post, it would warm up and start stalling. So we put it on an ice block and it lasted long enough to get the backup done.

</comment><comment><USER8> I used to have an old laptop that overheated like crazy, it rested on a 6 pack of soda that had been in the fridge. I had 2 sets of 6 packs so I could rotate them as they warmed up.

Take the six pack, fold it flat:

000

\-----

000

Laptop on top :)

</comment><comment><USER9> This is why you typically leave the hard drive (or in this case, the laptop) in the freezer for atleast a few hours. Doing this allowed me to copy a 60 gig which windows couldn't even mount to work no problem - which took well over a couple hours, all in one shot.

Using this method, the electronics don't need to be kept in your refrigerator or freezer.

<comment><USER10> Sounds like you had removed the hard drive, which does provide an added cooling advantage this guy felt unable to exploit.

</comment></comment><comment><USER11> Can anyone explain the theory behind why this works?

</comment><comment><USER12> My girlfriend is using my ~4 year old Macbook Pro, and it's getting warmer and warmer. It's too hot for direct contact with skin. Also, it's starting to drop the Wifi at high temps. The battery has been replaced once already.

As a result, we're looking at one of those unibody 13" Macbooks, those are reported to be cooler. Temperature is the only reason we're getting rid of this one.

<comment><USER13> Maybe you just have to clean the fans?

<comment><USER12> Thanks for the tip.

</comment></comment></comment><comment><USER14> I had a drive that was clicking and was completely unreadable. I put it in the freezer for 45 min and was able to read the drive for about 10 to recover a few files. Then it died and the trick didn't work again. But when all else fails, try the freezer or fridge!

</comment><comment><USER15> We used to freeze NES cartridges when blowing stopped working. We always chalked its working up to magic/luck, but perhaps there was more to it.

Is it just the heat or is there something else at play here?

</comment><comment><USER16> Does taking out your hard drive seriously void your warranty on a Macbook? That sounds, uh... completely insane relative to the practices of every other PC manufacturer.

<comment><USER17> No, it doesn't. Here's the relevant quote, indicating that the only thing they won't cover is damage you cause while attempting a swap. Swapping itself is fine.

 _WARNING: Apple recommends that you have an Apple-certified technician install replacement drives and memory. Consult the service and support information that came with your computer for information about how to contact Apple for service. If you attempt to install a replacement drive or memory and damage your equipment, such damage is not covered by the limited warranty on your computer._

<comment><USER18> 2nd this - I always remove hard disks before dropping off laptops for service.

</comment></comment></comment><comment><USER19> It used to work with bad floppies too, a long time ago. Nowadays I'm using SSDs in all my development and server machines.

</comment><comment><USER20> I often put my dell D620 in the fridge to cool it down if it overheats and starts running slow.

<comment><USER21> This is a Latitude defect (in my opinion). The fan gets clogged really easily and really quickly. If you open your laptop and clean the vent it will run correctly. Your CPU will go from 60C down to 35C instantly. I'm predicting a class action lawsuit at some point.

[http://vidmar.net/weblog/archive/2008/06/29/my-dell- laptop-i...](http://vidmar.net/weblog/archive/2008/06/29/my-dell-laptop-is- running-loud-fan-is-almost-always.aspx)

</comment></comment><comment><USER22> Cool.

</comment><comment><USER23> And if you put your laptop in the oven you can rip a CD faster. Also I've heard that bubble sort finishes faster when in a microwave set to Popcorn.</comment>


If Your iPhone Is Slow Try a Backup and Restore - <AUTHOR> http://mglenn.com/blog/2017/12/22/apples-bungled-battery-feature <comment><AUTHOR> _" So I went home and immediately did a local backup, wipe and restore. And voila! Performance issues were gone."_</comment>


Cities Find Google Fiber Networks Giving Them Too Much Speed - <AUTHOR> http://www.newsmax.com/Newsfront/google-high-speed-networks/2014/09/06/id/593012/

<comment><USER2> Is it me or is this article stupid?</comment>


Very, Very Well Documented - <AUTHOR> http://thedailywtf.com/Articles/Very,_Very_Well_Documented.aspx

<comment> a-priori Civil aviation projects are not much better.

I once worked for a sub-contractor on a contract to develop the software for a pair of cockpit display units (Honeywell's DU-875/DU-885 units). We had a large amount of documentation -- at least a half metre or so -- before we even left the design phase. And this was just for the application software. The operating system and hardware had a couple times that amount. I wouldn't be surprised if the entire system amounted to at least 4-5 metres of docs upon completion.

This was legitimate documentation; we didn't BS, we were just very very detailed. The standard (DO-178B) specifies how detailed you need to be, not how much you need to produce.

In fact, as I recall, I spent most my time maintaining documentation.

</comment><comment><USER2> That reminds me of when I worked as a consultant with a company that did a lot of military contracts. I had to start on a project right away but couldn't work with sensitive data until my security clearance came through. So I did what any programmer would do, I made up a bunch of data to test the code. The software would produce reports which had to have to have the word SECRET clearly visible on the first page. The funny part was, I wound up producing several boxes full of bogus reports which had to be treated as secret. So they had to be stored and destroyed according to military specs.

<comment><USER3> I had a friend who did admin stuff for a research group at what was once known as ford aerospace. He told me once that his boss always hated anything SECRET because they then had to install more locked cabinets in order to keep it secure for several years.

</comment></comment><comment><USER4> Why humans really need artificial intelligence: To read thru all of that nonsense and produce the 7 page summary that the actual relevant information could be contained in.

If you think engineering produces pointless paper for the sake thereof, you should see law!

</comment><comment><USER5> Your tax dollars at work.</comment>


JITs are un-ergonomic - awinter-py https://abe-winter.github.io/2020/03/28/jitu-brutus.html <comment><AUTHOR> An anecdote from a non-JS JIT, but similar: I once spent a summer working on a game engine with a couple others where the host language was LuaJIT.

It started out great. The iteration cycles were incredibly short, performance was competitive, the code was portable, and we could do wild metaprogramming nonsense that was still fast. If you haven't worked with LuaJIT, its C FFI is also incredible!

As we started scaling though, the wheels fell off the wagon. We'd add a new feature and suddenly the game wouldn't run at interactive framerates. One time, it was the 'unpack' function, which would trigger a JIT trace abort. We would drop from 12ms frames to 100ms frames. I wrote a length-specialized version that didn't abort and moved on.

Another time, it was calling Lua's 'pairs' method (iterator over a map). Okay, so we can't do that, or a few other things that made Lua productive before.

The other problem we hit was GC predictability being impossible. We tried to mitigate it by using native data structures through the C FFI, taking control over the GC cycle to run it once or twice per frame, etc. In the end, like the JIT problem, we weren't writing Lua at the end, we were writing... something else. It wasn't maintainable.

That summer ruined dynamic languages for me. I didn't really want to be writing C or C++ at the time. I ended up picking up Rust, which was predictable and still felt high-level, and the Lua experience ended up getting me my current job.

<comment><USER2> I'm working with Lua right now(gopherlua) as a scripting option for real-time gaming. I've done similar things to your story in the past with trying to make Lua the host for everything and I'm well aware of the downsides, but I have a requirement of maintaining readable, compatible source(as in PICO-8's model) - and Lua is excellent at that, as are other dynamic languages, to the point where it's hard to consider anything else unless I build and maintain the entire implementation. So my mitigation strategy is to do everything possible to make the Lua code remain in the glue code space, which means that I have to add a lot of libaries.

I'm also planning to add support for tl, which should make things easier on the in-the-large engineering side of things - something dynamic languages are also pretty awful at.

<comment><USER3> You might still run into GC problems, but none of the Go-based Luas (built on Go, rather than binding to another Lua library) I am aware of have a JIT built in.

</comment></comment></comment><comment><USER4> The OpenJDK JVM (aka Hotspot) addresses both issues: control [1] and monitoring [2] (there are built-in compilation and deoptimization events emitted to the event stream). You can also compile methods in advance [3], and inspect the generated machine code when benchmarking [4]. You can even compile an entire application ahead-of-time [5] to produce a native binary.

[1]: [https://docs.oracle.com/en/java/javase/14/vm/compiler- contro...](https://docs.oracle.com/en/java/javase/14/vm/compiler- control1.html)

[2]: [https://docs.oracle.com/en/java/javase/14/jfapi/why-use- jfr-...](https://docs.oracle.com/en/java/javase/14/jfapi/why-use-jfr-api.html)

[3]: [https://docs.oracle.com/en/java/javase/14/docs/specs/man/jao...](https://docs.oracle.com/en/java/javase/14/docs/specs/man/jaotc.html)

[4]: [http://psy-lob-saw.blogspot.com/2015/07/jmh-perfasm.html](http://psy- lob-saw.blogspot.com/2015/07/jmh-perfasm.html)

[5]: [https://www.graalvm.org/docs/reference-manual/native- image/](https://www.graalvm.org/docs/reference-manual/native-image/)

<comment><USER5> The GC is still nondeterministic though.

<comment><USER6> So is malloc

<comment><USER7> It is much easier to avoid malloc writing C than to avoid new in Java.

<comment><USER6> Nothing to do with GC though. It's also fairly easy to avoid `new` in Go and C#.

<comment><USER7> To some degree, probably.

Structs have many limitations though and two different argument passing semantics in the same language mean more complexity. Also you may be lucky to eliminate allocations in your own code, but what about libraries? Existence of GC shepherds programmers and library creators into heap allocation. You don't need to write new explicitly to allocate on the heap.

Also high level languages with no GC make manual memory management much more usable and provide way better ergonomics in this area, just because they have to. E.g. automated reference counting, ownership/lifetime control, move semantics etc.

<comment><USER8> Automated reference counting is a garbage collection algorithm, and it is possible to have ownership/lifetime control without giving up on GC, Chapel, ParaSail, Haskell, OCaml, Swift, D are following up exactly this path.

<comment><USER7> How do you avoid heap allocations in Haskell?

Automated reference counting is technically a GC, but a kind of GC that can be enabled only for a subset of objects. In languages which force GC on everything (even reference counted GC) the incentive to provide abstractions that work without GC are much weaker. It is just hard to opt-out from GC once you have it and once all your stdlib relies on it. I thing D learned it the hard way.

<comment><USER8> Stack and register allocation in Haskell is under compiler control via escape analysis, however you can allocate native heap outside GC via mallocBytes/allocaBytes/free, and if they get integrated into mainstream GHC, linear types.

Besides, plenty of GC enabled languages offer the option to stack allocate, static global allocations, or native heap.

Some examples, including languages that for whatever reason failed on the mainstream market.

D, C#, Swift, Oberon, Oberon-2, Active Oberon, Component Pascal, Mesa/Cedar, Sing#, System C#, Nim, Modula-2+, Modula-3, VB, Xojo, C++ (via C++/CLI, C++/CX and Unreal C++), Common Lisp.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> This article has a number of issues. JS with JIT is waaay faster than Python. Not "between python and java" as purported. Second, generalizing jits as "un- ergonomic" seems silly given that what's being specifically looked at is benchmarking. But what makes this claim ridiculous is that _nothing_ is easy to benchmark. Even native code is hard to profile and this is literally my day job. If the JIT makes your code _that_ much faster, this strikes me as a pretty suspect complaint

<comment><USER10> I think that by "between python and java" they meant "faster than python and slower than java". I think Java still beats JS unless you get lucky.

You're totally right that benchmarking and profiling is hard even for native code. I think this post fetishizes whether or not a piece of code got JITed a little too much. Maybe the author had a bad time with microbenchmarks. There's this anti pattern in the JS world to extract a small code sample into a loop and see how fast it goes - something that C perf hackers usually know not to do. That tactic proves especially misleading in a JIT since JITs salivate at the sight of loops.

<comment><USER11> That still doesn't make sense to me.. how can JIT in general be slower than Java when Java is JITed?

<comment><USER12> The Java JIT has static type information to work with, the JS JIT can only infer type information via heroic efforts. Static types do mean something, especially when working with primitives and other unboxed data types.

<comment><USER13> Well, you're right that JS JITs can only infer type information via heroic efforts, but AFAIK the Java compiler throws away any type information from the source code, which means that the JVM JIT needs to inver type information again from the byte code.

Still, the JVM JIT is faster than JS due to reasons explained in the sibling comment of the parent one.

<comment><USER14> > but AFAIK the Java compiler throws away any type information from the source > code

You're thinking about generics. .class files preserve a whole bunch of type information (I'm building a .class decompiler in my free time, and I'm looking at that very same data in my debugger ATM).

<comment><USER15> You probably know this, but it's not obvious to the random reader.

Even generic types are available in the Java .class file and are accessible from the reflection API. Spring for example uses this quite heavily.

<comment><USER16> It depends on where you are at.

Type information is present in fields and class inheritance

For example, a class like this

`class Foo implements Bar<String>`

Retains the fact that the generic type is a String.

That information is completely lost at method invocation. So a method that takes a `Bar<String>` ultimately compiles to a method that takes a `Bar` and knows nothing of the String.

To get that generic information down you have to engage in some fun tricks using either the class or field method I mentioned earlier. (Usually you do this with a second type parameter where it matters).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> As someone who's been writing a lot of JavaScript, Go, and a handful of other languages for a while, I feel this. In Go, I can basically know what's going to happen when I write a function. This operation will read from the stack, these instructions will be run, and I can take a peek at the assembly if I'm not sure (though I've developed a pretty good feel for what Go will do without needing that). I can benchmark it and know that the performance I see on my machine will be the performance when I ship this bit of functionality into production, barring hardware differences.

In JavaScript, it's a black box. I know some constructs might deoptimize functions when run on Wednesdays because I read them on a blog published in 2018 that's _probably_ still accurate. In my benchmark running on Node 12.14.1 on Windows this seems to be true. But then who knows if it'll be the same thing in production, and it might 'silently' change later on.

JavaScript in V8 is incredibly fast these days, but I find it much easier to write optimal code in Go.

<comment><USER18> It's really no different from native compiler optimizations, which are also mysterious and always changing, except for one key aspect: when recording timings to compare against other timings, you can control whether optimizations are turned on or off, to remove that variable from the comparison.

<comment><AUTHOR> Thankfully, with a native compiler you don't have to deal with your JIT warming up, or some other code causing your function to get deoptimized, or subtle JIT trace aborts. :(

<comment><USER19> > or some other code causing your function to get deoptimized

Why doesn't the optimizer generate different compiled implementations of a function for different pieces of code?

<comment><USER20> I would imagine this to be due to the overhead of then having to track where EVERY variant of a JITted function can then be called from, when to deoptimize, etc.

</comment></comment></comment></comment></comment><comment><USER18> > if your economics are such that servers are a bigger cost than payroll

Sorry, and I may be oversimplifying the author's situation, but this really sounds like a case where you need to not be using JS for your server. On the client you don't have much choice, but on the client the pure-JS performance rarely gets tight enough to warrant this degree of micro-optimization work.

Author makes some good points - it would be great if the JIT were more profiler-friendly - but I have to question a little bit how important it actually is, the way the use-cases line up.

<comment><USER21> When someone realized you pay a lot for cycles at the server, but you pay nothing for cycles at the client, that was the moment the world-wide web began to die.

<comment><USER18> This is baseless mudslinging. The bottleneck on the client is virtually never the cycles consumed by the actual running of the actual app's JS code. It's usually, in order starting with the most common:

\- Piles of ads/analytics scripts which have no motivation not to slow down the page

\- Reflow; i.e. needlessly many elements on the page causing the browser to do extra work calculating layout

\- The initial loading and JIT-ing time of a needlessly heavy JS bundle

</comment></comment></comment><comment><USER22> > Interpreters, which read the program line by line as it runs

Byte code interpreters do that? That would be surprising. Programs are represented with 'lines' in byte code?

Things he wants, let's look at SBCL, a Common Lisp implementation (see [http://sbcl.org](http://sbcl.org) ):

> compile short bits to native code quickly at runtime -> that's done by > default

> identify whether a section is getting optimized -> we tell the compiler and > the compiler gives us feedback on the optimizations performed or missed

> know anything about the native code that's being run in a benchmark -> we > can disassemble it, some information can be asked

> statically require that a given section can & does get optimized -> done via > declarations and compilation qualities

> compile likely sections in advance to skip warmup -> by default

> bonus: ship binaries with fully-compiled programs -> dump the data (which > includes the compiled code) to an executable

<comment><USER23> Yes, I thought of SBCL when I was reading this too.

A JS frontend for SBCL could be nice...

<comment><USER8> Or at very least improve browser developer tools to give us what Lisp compilers have been doing the last half century.

Why should we need to hand compile special versions of JavaScript VMs, just to get _(decompile ....)_?

</comment></comment></comment><comment><USER24> A JIT is just another cache, like memory. Yes, it's hard to predict, but not fundamentally that different from caching in any language. It does mean perf tests have to be end-to-end and match real-world loads, but it doesn't mean it's "impossible" at all, it means you need to measure.

Is this a real problem? I've been profiling my JS for years and never actually run into a mysterious problem where some important code I profiled was way way slower in prod than when I was profiling. Has that happened for you? How often does this happen? I take it as an assumption that profiling is something you mostly do on inner loops & hot paths in the first place. I mean, I profile everything to look for bottlenecks, but I don't spend much time optimizing the cold paths.

> Get notified about deopts in hot paths

Studying the reasons for de-opts help you know in advance when it might happen. If you avoid those things, do-opts won't happen, and you don't need notifications.

For example, ensure you don't modify/add/delete keys in any objects, make sure your objects are all the same shape in your hot path, don't change the type of any properties, and you're like 90% there, right?

> statically require that a given section can & does get optimized [...] > compile likely sections in advance to skip warmup

While these don't exist in V8, it's maybe worth mentioning that the Google Closure compiler does help a little bit, it ensures class properties are present and initialized, which can help avoid de-opts.

</comment><comment><USER25> Hey Node/bluebird person here: You want to run Node with --teace-opt and --trace-deopt and --allow-natives-syntax with %OptimizeFunctionOnNextCall before benchmarking.

</comment><comment><USER26> The high level point that is of great importance is that if one wants a computer program to function in a reliable way one needs simple and understandable algorithms/components. Nowadays even the processor is no longer simple and assembly has become a high level language and it got us 'nice' things like spectre. For the practical day to day work the KISS principle that has been the corner stone of effective programming for half a century is now more important than ever. Yes, there is a nice library available to do such and such and maybe you need it, but are you also thinking about the dangers that any additional moving part may increase unpredictability? Let me give one very stupid example that I ran into this very week. The agreed-upon answer according to [https://unix.stackexchange.com/questions/29608/why-is-it- bet...](https://unix.stackexchange.com/questions/29608/why-is-it-better-to- use-usr-bin-env-name-instead-of-path-to-name-as-my) is that it is better to do #!/usr/bin/env bash than #!/usr/bin/bash. I say: absolutely not! You have just increased the number of moving parts involved by one for an immeasurably small benefit. And if you are worrying about bash versions then just stop using any bash features that are less than a decade old. I also say that unless it is necessary for the core problem that you are trying to solve stay away from any features of anything that are less than 5 years old. And if you actually need the new and fancy stuff, and maybe you do, expect to pay a hefty price for it. Every new tool that you introduce has its own peculiarities that you will spend hours of debug time on and one should err on the side of just saying no.

</comment><comment><USER27> LuaJIT, true to form, has a sort of solution for this, in the form of a profiler cheap enough to run in production.

You do have to change how you think about performance analysis, but in return, you get to actually answer the question you're trying to reason about, namely, how does this run in production.

Pacifying the JIT is a bit of a dark art, but the whole thing is pretty transparent with good tooling. I've yet to regret building on LuaJIT.

<comment><USER28> Worth noting that the JVM has this feature as well, under the name of "flight recorder".

</comment></comment><comment><USER29> This article contains a logical error. The premise is JITs are hard to benchmark and keep good performance on. That's true.

But the alternative is bad performance all the time (the JITS fall back to interpretation, after all).

What's the value in having clearly understood bad performance? If you care enough about performance that you need to understand it, surely you care about the absolute level of performance.

<comment><USER30> False dichotomy. No one should be using shitty byte code interpreters like (c)python for real production work. Or at least stop whining about global warming. The real alternative is AOT.

Also in many cases having it's much better to trade off average performance for lower variance.

<comment><USER31> The vast majority of the time it doesn't matter. Most of us don't write performance critical code. Even games use bytecode VMs for scripting. You'd be better off spending the money saved on planting trees.

If you write a web app in Ruby using a similarly light weight frameworks as you would using Go, you'll be lucky to get 3x the throughput out of the Go app.

There are also ways to more efficiently utilize hardware by using bytecode than distributing binaries. Using bytecode verification rather than an opaque binary, you can pack thousands of different web applications into a single process.

This is an application of webassembly which IMHO is much more promising than in-browser use.

<comment><USER30> > The vast majority of the time it doesn't matter. I keep hearing this all the > time and keep seeing all the time that it isn't true. Do you know of a > single comparable Go web app that's not 10-100x less wasteful than goddamn > gitlab? Most of the pathological sources of CO2 cloud emissions are a > combination of terrible architecture and slow as molasses scripting > languages like python or ruby. But these are _not_ orthogonal. In theory > gitlab could probably be 100x less wasteful even when written in Ruby. In > practice many bad architectural decisions are far more likely to occur with > Ruby and Python than a statically typed language (although Java may serve an > interesting counter-example).

> Using bytecode verification rather than an opaque binary, you can pack > thousands of different web applications into a single process

Can you expand on what problem this solves?

Anyway, the problem isn't the bytecode, it's the shitty interpreter. There are plenty of bytecode based languages that are within < 10x of C.

<comment><USER31> Yeah, absolutely. I've worked on some incredibly poorly performing Go services. The idea that a team struggling to deliver on a roadmap and keep up application performance while using a high level language like Ruby will magically be able to do it while using a lower level language is just a complete fallacy. It never happens!

People get bogged down. They don't write low allocation code. Nobody has any time to turn on the compiler warnings for escape analysis. Your benchmarks end up a year out of date and don't even run anymore. Architecture becomes an afterthought. Function/method calls end up becoming even slower gRPC calls as the team struggles to box up complexity and extracts more services.

WRT bytecode verification it's better just to read this: [https://www.fastly.com/blog/announcing-lucet-fastly- native-w...](https://www.fastly.com/blog/announcing-lucet-fastly-native- webassembly-compiler-runtime)

There's nothing shitty about the CRuby bytecode VM. It's all a question of resourcing. You're doing a huge disservice to all the people who worked very hard to make CRuby 2.7 5-15x faster than 1.8 and it detracts from a valid discussion about reducing the CO2 footprint of datacenters around the world.

</comment></comment></comment></comment></comment><comment><USER10> This post is extremely V8-centric. For example it uses terminology like "deopts" which means nothing in JavaScriptCore (we distinguish between exits, invalidations, jettisons, and recompiles). The post also assumes that there is only one JIT (JSC has multiple).

And that's where you've lost me. Not sure how you expose anything about how the JIT is operating without introducing a compat shitshow since JIT means different things in different implementations.

If you really want to know that something gets compiled with the types you want, use a statically typed and ahead of time compiled language.

If you have to use a JIT but you find that it doesn't do what you like then remember that it's meant to be stochastic. The VM is just trying to win in the average. Which functions get compiled and with what type information can vary from run to run.

Probably the best thing that could happen is that developer tools tell you more about what the JIT is doing. But even that's hard.

There are some specifics that I disagree with:

\- I don't think all JIT architects for JS claim that the perf is about competing with C for numerical code. I don't explain it that way. I would say: JITs are about doing the best job you can do under the circumstances. They can make JS run 4x faster than an interpreter if things really go well. "Between Python and Java" is a good way to put it and that's exactly what I would expect. So if that's your experience then great! The JIT worked as expected.

\- It's usually foolish to want your code compiled sooner. Compilation delay is about establishing confidence in profiling. I'm pretty sure we'd JIT much sooner if it wasn't for the fact that it would make the EV of our speculation go negative.

TL;DR. the JIT can't unfuck up JavaScript.

<comment><USER32> > Probably the best thing that could happen is that developer tools tell you > more about what the JIT is doing. But even that's hard.

Why? You have the exact same problem when writing SQL code but there you have lots of powerful introspection tools to make it easier to control performance. You can also use indices and hints to nudge the RDBMS into executing queries in the most optimal way. PyPy for example has a lot of support for introspection.

<comment><USER10> Because JS JITs aren't deterministic. Just starting the profiling or introspection tool could change internal behavior.

The nondeterminism can come from lots of places. In JSC it's that the JIT polls the heap for some of its profiling and it profiles concurrently. So OS scheduling decisions affect what types the JIT sees.

<comment><USER32> Performance in general is non-deterministic in a multi-tasking os because you have multiple processes competing for cpu time. My point is that there is nothing inherent to Javascript VMs that makes them harder to control or analyze. Runtimes for other dynamic languages and database engines shows that proper tooling makes it much easier for developers to optimize for performance.

<comment><USER10> On some level of course that's true but surely you're not suggesting that the nondeterminism of JS is no worse than the nondeterminism of C. C at least runs with the same types every time.

<comment><USER33> C is weakly typed, so that it runs with the same static types means very little.

<comment><USER10> C has strong types for the purpose of optimization. A C compiler never wonders whether + means int addition or string concatenation for example. In JS the optimizing compiler doesn't get such type information except by profiling and profiling is nondeterministic.

</comment></comment></comment></comment></comment></comment></comment><comment><USER34> If you were to get the information you were after it would be specific to a particular implementation, and a particular _version_ of that implementation. The nature of how JIT is done for JavaScript is not defined in the spec and varies a lot (starting from totally nonexistant many years ago).

To have useful performance measurement tools that are going to help your code run across multiple implementations would require a) all of those implementations to work the same, forever, and b) the semantics of this to be specified in the standard.

</comment><comment><USER35> The thing is, javascript performance is mostly "good enough", and honestly that's what really matters.

When you have workloads that are mostly io bound, having syntactic sugar like js async/await to avoid blocking is really a huge strength.

When we write systems, we operate under constraints, and frequently the largest constraint is time to market rather than pure performance.

Dynamic typing can be a huge strength for TTM.

If performance was the only concern, we'd all be in straight C with inline asm.

<comment><USER36> What's TTM?

<comment><USER37> Time to market I'd guess?

</comment></comment></comment><comment><USER38> As a nodejs developer: what is the best compiled server language to learn right now? Is it Java still, or is it better to look at go or rust?

<comment><USER39> Part of me wonders if ASP.NET will make a comeback. A lot of the historical reasons not to use it don't exist anymore (there's an officially-supported Linux port, for example), and a lot of the cool new JavaScript language features like async/await and arrow functions came from C#.

Besides, most JavaScript programmers are already dependent on Microsoft products like TypeScript, NPM, and Visual Studio Code. What's one more?

<comment><USER38> Is it open source?

<comment><USER40> Kind of? Vital parts are still closed (like the debugger), and the tooling still assumes a closed-source world (for example, with OmniSharp, jump-to- definition to dependencies just gives you a synthetic source file that only contains method signatures).

It's also not exactly run like a community project. The issue tracker feels more like a customer support system: "can you try updating to the newest release now?" abound, with no mention of what the actual problem or fix was.

</comment></comment></comment></comment><comment><USER41> I completely agree, it's possible to do some tracing of what the V8 JIT does, but the workflow is awful.

Microbenchmarks don't represent the real world. Instead of running a single microbenchmark, I suggest running a host of them, but all of them "at the same time" (not serially). They'll get in each others ways and total performance will be far worse.

This happens with AOT code as well of course, because caches are being trampled no matter what, but JIT code only exarcerbates the issue because it is larger.

</comment><comment><USER42> Lancet [1] seems like it fixes at least some of the unpredictability. (Haven't used it though, only read about it on another site.)

[1]: [https://github.com/TiarkRompf/lancet](https://github.com/TiarkRompf/lancet)

</comment><comment><USER43> Not a big deal if you're mainly writing client-side JS. You're never gonna know how fast something will run in a browser anyway.

</comment><comment><USER44> I'd love a jit for DosBOX-X, or Qemu. I know, KVM, but think about emulating non-native archs. Or systems without KVM support.

<comment><USER42> Doesn't QEMU already JIT? I guess, depends how much [1] counts.

[1]: [https://wiki.qemu.org/Documentation/TCG](https://wiki.qemu.org/Documentation/TCG)

<comment><USER44> Is not marvellously fast, albeit x86 it's a crawl with these CPU context switches. MIPS emulation seems faster, much faster.

</comment></comment></comment><comment><USER37> Dead on. JITs are high interest credit card technical debt when it comes to performance. It says a lot that all the performance-sensitive parts of widely deployed JITs are themselves implemented with AOT compilation.

<comment><USER36> > It says a lot that all the performance-sensitive parts of widely deployed > JITs are themselves implemented with AOT compilation.

…not really? Managed languages with a substantial runtime tend to make for poor ergonomics when writing JITs.</comment></comment>


The Fatal Shore, Awash in Plastic - <AUTHOR> http://rendezvous.blogs.nytimes.com/2012/08/23/the-fatal-shore-awash-in-plastic/ <comment><USER2> That's really awful. Glad he's going through he effort to make this public.

<comment><USER3> Why is it awful? Species that can't tell plastic garbage from food are obviously unfit for Earth in 2015.

<comment><USER4> To be fair, there's nothing wrong with exctinction, just as there is nothing wrong with "invasive species". These things are as old as the hills. A certain amount of genetic turnover is, as you point out, just the fitness function making its judgements.

The trouble we have is the _rate_ of extinction and invasion. A typical ecology can support a certain rate of species loss... A single species will be replaced by other organisms that can occupy the same niche. They will fill in the gaps and the same level of biodiversity can be maintained.

However these things take time. And if, while the replacement species is adapting to fit the old niche, another several extinctions happens, we start to have problems. Organisms that could've adapted to fill out the niche are themselves under stress. Once you have several key species under stress simultaneously, you run the risk that an entire segment of the food chain... possibly double digit percentages of the metabolic chain, will collapse simultaneously.

With a slow enough rate of extinction, you can just keep filling in the holes, but once you cross a certain threshold, you can trigger a regression to a more primordial state. Like desert. You end up with orders of magnitude less complex of an ecology.

If think we can get by on a planet with an ecology that is an order of magnitude less complex, with an order of magnitude (or two) fewer species, then none of what I'm saying is a problem for you.

I am scared of dramatically decreasing the number of species just because it seems really hard to predict the effects of that. So I'd like to delete the species as slowly as possible to mitigate our risk.

</comment></comment></comment><comment><USER5> From 2012. Submitter or mods, please update the title

</comment><comment><USER6> "How do we get to hope from here?"

<comment><USER7> Starts with you #startswithyou

<comment><USER6> It _really_ doesn't. My cynicism is capable of inflicting permanent damage on innocent bystanders when not kept quietly in check.

</comment></comment></comment><comment><USER8> A problem; deserves attention. Yet I can't help but believe the article's title image was fake; that lighter looks clean and unworn. Was the image staged? The photographer is an artist after all. I know he claimed it was all unmodified. I'm just not buying it.

<comment><USER9> I live by the sea and often walk along the beach and often see things like lighters bits of plastic construction materials, garden chairs, all sorts of things and they appear in all sorts of conditions.

You might be right of course but not because the lighter looks clean and unworn. Plenty of things emerge from the sea clean and it might have been there only a few days or even hours.

<comment><USER8> The dead bird looks very old and weatherworn. I'd expect the stomach contents to be in the same vein. Other stomach contents (a shocking amount in fact) look as expected. So I was just suspicious that it might have been 'curated'.</comment></comment></comment>


Raspberry Pi's smaller, cheaper rival: NanoPi Neo Plus2 weighs in at $25 - <AUTHOR> http://www.zdnet.com/article/raspberry-pis-smaller-cheaper-rival-nanopi-neo-plus2-weighs-in-at-25/ <comment><USER2> It has a H5. Goodbye.</comment>


Big Macs vs. the Naked Chef (2001) - <AUTHOR> https://www.joelonsoftware.com/2001/01/18/big-macs-vs-the-naked-chef/ <comment><USER2> Hmmm, where does agile fit into the story nowadays? :)</comment>


MoMA Adds Video Games to Its Collection - <AUTHOR> http://artsbeat.blogs.nytimes.com/2012/11/29/moma-adds-video-games-to-its-collection/

<comment><USER2> Slate article has a little more info: [http://www.slate.com/blogs/browbeat/2012/11/29/moma_s_new_vi...](http://www.slate.com/blogs/browbeat/2012/11/29/moma_s_new_video_game_collection_museum_of_modern_art_proclaims_video_games.html)

"As with their film collection, MoMA aims to obtain the games in their original format—as well as acquiring the source code."

<comment><USER3> It would be also necessary to collect old hardwares or emulators.

<comment><USER4> And in some cases old televisions. The NES light-gun, for example, only works with CRTs.

<comment><USER5> I know the Super Scope relies on CRT scanlines to operate, but I was fairly sure that the NES' gun should work on any display that can change fast enough to black out all the screen except the targets, which (IIRC) go white with striped lines; is that not correct?

<comment> T-hawk No, the NES Zapper does both. It detects light level changes frame-to-frame as you describe, but it also uses the CRT beam timing to determine _where_ on the screen a target was hit.

<http://en.wikipedia.org/wiki/NES_Zapper>

Also, even if the NES Zapper worked only on luminosity deltas, it could work on other types of displays -- but the response time (lag) of the display would have to be frame-accurate in order to detect the hit in time before the software moves on. (Software could compensate for lag, of course, but actual NES games don't.)

</comment></comment></comment></comment></comment><comment><USER6> Their selections show a lot of thoughtfulness, especially when it comes to negotiating the tricky balance between selecting for historical/cultural significance and selecting for artistic significance.

I never in a million years would have guessed they'd choose Vib Ribbon, but it totally deserves it.

<comment><USER7> I loved Vib Ribbon and I'm very glad to see it there too, it was groundbreaking at the time but never really gained any popularity and has long since been forgotten.

<comment><USER8> We have Vib Ribbon on display now at the MADE <http://www.themade.org> so come by and play it before it's changed out for another exhibit on December 17th.

<comment><USER7> I'd love to but I live in the UK!

</comment></comment></comment></comment><comment><USER9> The creator of one of the exhibited games, Katamari Damacy, went there. You can read about it (in English) on his blog.

[http://www.famitsu.com/cominy/?m=pc&a=page_fh_diary&...](http://www.famitsu.com/cominy/?m=pc&a=page_fh_diary&target_c_diary_id=41433)

<comment><USER10> What's with the "word-break: break-all" in the CSS? It's really weird to wrap English text that way; and removing that doesn't seem to affect how the Japanese text is wrapped.

</comment></comment><comment><USER11> Ahh, about time I think. I'm very excited to see these in MoMA, if only for the discussion that was lingering around finally being brought to the forefront. Videogames (and now computer programs) are an art form. Some certainly deserve getting discussed and becoming part of a dialogue that treats them as such before diving in critically. Maybe it opens up entirely new dialogues for when new-release games come out.

As an artist, and a person who really appreciates the context of Art History as it relates to war/political history, this is good news. Video games are a cultural phenomenon that deserve to be in the art discussion, similar to how it took a while for hip hop to earn merit in the music discussion, even though its proponents knew it was art from the get-go.

</comment><comment><USER8> For those of you in the valley, the MADE is in Oakland <http://www.themade.org>

It is, to my knowledge, the only dedicated videogame museum in America. This MOMA thing was interesting because we've been designing an exhibit about visual artistry in gaming, and we're going to be showing some of the same games as the MOMA, all playable.

It was reassuring to see the MOMA picked some of the same games we did, because it proved to us that we were at least as competent as MOMA. It's been a difficult road to build this museum, and we were excited to see we weren't going in the wrong direction.

Our visual arts exhibit, Games you can Frame will be on display starting December 17, and at GDC next year.

</comment><comment><USER12> How nice it was to see Dwarf Fortress on that list!

<comment><USER13> DF, Another World, and NetHack were three pleasant surprises out of that list (though I might have preferred Rogue or DCSS in place of NetHack, depending on what their focus is).

Then again, in increasing order of bafflement: Canabalt (fun and beautiful, but not sure if it's world-class-museum-quality), Animal Crossing, and Passage (what the hell??).

<comment><USER14> I think both Canabalt and Passage are great exponents of current schools of game making (if you can call the "indie game" and the "art game" a school). They show what people with similar interest to those that made Katamari Damacy are doing on much lower budgets.

Animal Crossing does seem odd though, specially considering The Sims is already there. Seems redundant at best.

<comment><USER13> My gripe about the inclusion of Canabalt isn't the game itself, it's that it's just too recent to determine if it will really withstand the test of time. Another World was a fun and beautiful game that barely got any press or sales at all, yet the fact that we still remember it all these years later is a testament to its merit.

And now that Canabalt's been selected for the MoMA's permanent collection, it will be forever impossible to determine whether the game was remembered because of its own staying power, or just because the MoMA told us that we ought to remember it.

<comment><USER15> _"it will be forever impossible to determine whether the game was remembered because of its own staying power, or just because the MoMA told us that we ought to remember it."_

Isn't that the case with all contemporary art acquisitions, though? I'm reminded of this quote by Dave Hickey from Air Guitar:

"Since there is no absolute authority in the art world, or in the economic world either, we may presume that for every opinion, there is a contrary one. Thus, the social value of a work of art, or an art critic, or a theory, or an institution must be distinguished from its social virtue, since bad reviews, stupid acquisitions, and theoretical attacks, even as they question the social virtue of an object or investment, must necessarily invest it with social value. The raw investment of attention, positive or negative, qualifies certain works of art as "players" in the discourse. So, even though it may appear to you that nearly everyone hates Jeff Koons' work, the critical point is that people take the time and effort to hate it, publicly and at length, and this investment of attention effectively endows Koons' work with more importance than the work of those artists whose work we like, but not enough to get excited about."

I mean, give it time or ignore it, or both. All museums make questionable purchases. Being in the permanent collection is certainly a boon, but it won't change how people fundamentally feel about the art in a few generations. Maybe it'll remain successful, maybe it won't, but it certainly captures _something_ which i guess is reason enough to acquire it.

</comment></comment></comment></comment></comment><comment><USER3> I have a question. I understand Pac-Man and Tetris in the collection. But what does it mean to collect EVE Online? I presume it does not mean MoMA has collected its server code and data?

<comment><USER16> From close to the end of the article:

"some of the games we have acquired (for instance Dwarf Fortress and EVE Online) take years and millions of people to manifest fully. To convey their experience, we will work with players and designers to create guided tours of these alternate worlds, so the visitor can begin to appreciate the extent and possibilities of the complex gameplay."

(Dwarf Fortress is clearly more about the "years" than the "millions of people", mind you.)

</comment></comment><comment><USER17> So is it more difficult for them to get Japanese games in their collection? The lack of Japanese games, save for Katamari Damacy, is pretty striking. Seeing Passage on the list also got a groan out of me.

<comment><USER18> Pac-man, Tetris and Vib-Ribbon are also Japanese.

Their "wish list" includes many more Japanese games, like Donkey Kong, 2 Super Mario Bros. games, The Legend of Zelda, Street Fighter II, Chrono Trigger and Animal Crossing.

<comment><USER19> Tetris is Russian, not Japanese...

<comment><USER18> The Tetris _board game_ is Russian. The Game Boy game is Japanese.

</comment></comment></comment></comment><comment><USER20> Notable that they chose to collect these games as design as opposed to art.

</comment><comment><USER21> I went to their exhibit last year which featured Dwarf Fortress (as a mural) and Passage (playable)...there were a few other games too, including he PS3's Little Big Adventure...I think Katamari is actually on display as part of their Century of the Child exhibit right now

[http://tumblr.eyeheartnewyork.com/post/9525690645/at-the- mom...](http://tumblr.eyeheartnewyork.com/post/9525690645/at-the-moma-indie- videogames-passage-and-dwarf)

I hope the games are playable...what better way to cement the idea that gamess are art in the mind of kids who may see art initially as just paintings and Greek sculpture?

<comment><USER22> I love Dwarf Fortress, but I would not characterize it as "playable."

<comment><USER23> I tried playing Dwarf Fortress over and over. I really want to love the game, but the UI is just miserable.

Different keyboard shortcuts for different types of buildings, that really aren't all that different. No easy way to manage your dwarves, or get vital statistics. I still have no idea how to actually get the little guys to get out there and beat something with a sword. I couldn't figure out how to trade with the damn caravan until I googled for it.

Aren't there any people out there as monomaniacally dedicated to crafting good UIs as whats-his-face is dedicated to building this huge world simulator?

<comment><USER24> You pretty much need to follow the wiki in order to get started: [http://dwarffortresswiki.org/index.php/DF2012:Quickstart_gui...](http://dwarffortresswiki.org/index.php/DF2012:Quickstart_guide)

I agree with you that the default interface is pretty miserable, but it has it's own charm in some way. Dwarves become much more manageable with the DwarfTherapist app. It acts as a usable interface extension for me. <http://www.bay12forums.com/smf/index.php?topic=66525.0>

<comment><USER23> Yeah, I followed the wiki, but I still ended up having to google around and keep a few browser windows open just to list the keyboard shortcuts.

And unfortunately, Therapist isn't available for mac, and I'm not willing to install a VM just to play it :/</comment></comment></comment></comment></comment>


Universal takes on Spotify freemium model - <AUTHOR> http://www.ft.com/cms/s/0/5645cf6c-ce73-11e4-900c-00144feab7de.html

<comment><USER2> bummer .. paywall

pastebin anybody?

<comment><AUTHOR> search the article title on google, read it without paywall</comment></comment>


Apollo 8 Crashed on Global TV: The 50th Anniversary of Apollo 8 - <AUTHOR> https://medium.com/@barryvacker/apollo-8-crashed-on-global-tv-the-50th-anniversary-of-apollo-8-and-what-it-meant-fe2f497805f8 <comment><USER2>
      a creationist in the White House in the person of VP Mike Pence

The VP doesn't live in the White House.</comment>


Building the Google Photos Web UI - <AUTHOR> https://medium.com/google-design/google-photos-45b714dfbed1 <comment><USER2> Section 2 of the article, on Justified Layout, was a joy to read. As it says, it's inspired by (adapted/simplified from) TeX's line-breaking algorithm, described by Knuth and Plass ([http://www.eprg.org/G53DOC/pdfs/knuth-plass- breaking.pdf](http://www.eprg.org/G53DOC/pdfs/knuth-plass-breaking.pdf)). The paper is a _fantastic_ read. Even if you never use TeX, it's worth reading. (I elaborated more on it here: [https://tex.stackexchange.com/a/423578/48](https://tex.stackexchange.com/a/423578/48))

As reading the paper will make clear, it's really incredible how with very simple primitives (boxes, glue and penalties), we can achieve so many different kinds of layout. Sometimes when I'm struggling with CSS layout I yearn for a layout model as simple as TeX's. (Though of course CSS has other hard problems to solve like variable width and "simpler" users, and I guess the most recent additions to CSS, like Flexbox and Grid typesetting, are some small steps in the direction of approaching the power of TeX circa 1978.)

</comment><comment><USER3> As someone who's grumbled at the semi-occasional glitch and lag in Google Photos I feel bad reading this. It's a genuinely hard problem.

<comment><USER4> >As someone who's grumbled at the semi-occasional glitch and lag in Google Photos I feel bad

Don't. They killed a good product, Picasa, which didn't have these problems _because it didn 't need to solve them_.

There's no way Google Photos - or any cloud solution - will be an adequate replacement to handle my 100GB+ library of JPEGs, RAW files, and videos, with instant editing and access to all of it -- if only because it doesn't take a second to pull 100GB of data from the net!

Not everything needs to be cloud-based -- but not according to Google's vision. Picasa still does a fine job for me, but I have to preserve my copy of the installer because Google wouldn't want me to have it any more.

<comment><USER5> It appears you didn't really read the article. There's no need to pull 100gb of data.

And 100gb is comparatively small for a photo and video library. The article mentions testing with 250k photos, which would be well over 100gb (2-3 times that at minimum, significantly more if RAW and high res).

<comment><USER4> >It appears you didn't really read the article. There's no need to pull 100gb of data.

I read the article, paying special attention to section 4, aptly titled "Smoke and Mirrors".

There's always a need to pull in as much data as the user wants to actually interact with.

No matter what tricks you pull, you'd still need about 1s to pull in a 10MB photo over a 100mbps connection at full speed, and most people don't have that. Reading data off a disk is an order (or two, with an SSD) of magnitude faster.

And sometimes, I just want to flip through the library and look at _a lot_ of photos without having to wait seconds for each one to load.

Sure, the layout algorithms are still important, and that's what a lot of the article is about. But all king's horses and all king's men can't make the latency of a javascript UI with a network backend comparable to a compiled program interacting with a disk.

Thank you, but smoke and mirrors aren't for everyone. Google Photos fulfills an obvious need, but it's not a Picasa replacement.

<comment><USER6> I went back to use the filesystem as I was used to do before.

Editing via Paint.NET and exporting to the Web via jAlbum.

Requiring a G+ account didn't help to gain my love.

<comment><USER7> 1) Folders date-labelled when the files were moved from camera to computer 20180712 and files date & sequence labelled 20180712-005.ARW 20180712-005-1.tif as variants are created

2) Edit mostly via RawTherapee (with special help from MS ICE & photoFXlab masks & Silver Efex Pro)

3) Distribute & Share with Google Photos (jpgs down-sized below 16MP) [upload with web browswer interface avoids all the confusion around auto backup and deletes]

</comment></comment></comment></comment></comment></comment><comment><USER8> I'd like to see even more responsiveness in the photo grid. I think these ideas are good, but could be taken further.

For example, while scrubbing fast, I shouldn't ever see blank grey tiles.

My scroll bar has say 1000 pixels of vertical height. That means there are only 1000 positions I can scrub fast to. They should preload all those positions in the form of a video with 1000 frames and then display the correct frame of the video as the initial low res render.

They could alternatively preload them in the form of a CSS/html grid with about 500 bytes per position, or 8 bytes per image. That should at least let them get vague gradients and approximate colours for the content of the thumbnails.

</comment><comment><USER9> I suspect some of this has changed at some point, right around when they switched to material design. Scrolling became laggy on some machines and the page would often become unresponsive. When Google Photos first came out it was wonderfully responsive but I have since switched away.

</comment><comment><USER10> This is by far my favorite Google product. I always wondered how they could do the scroll so fast. But for me the loading time is slow. I can scroll fast but I get into a blank area for a long time. The scroll should also work, on Android at least, when you do a search.

</comment><comment><USER11> Unrelated feature request: use photo resolution that matches the TV resolution on Chromecast.

Currently 26MP photos casted on a 4k TV (with Chromecast built in) look awful.

</comment><comment><USER12> Regarding the scrolling, any decent mobile developer can write a 60fps (and even 120fps with the new iPads) scrolling gallery with billions of photos, but with the web it's still a hard problem 20+ years in. This is pathetic.

<comment><USER13> With multiple aspect ratios and justified layout? Prove it. Neither the Apple Photos app nor Google Photos app can do this. If you're cropping everything into fixed aspect ratios, it's trivial on all platforms with existing libraries.</comment></comment>