Scientists pave the way for large-scale storage at the atomic level - <AUTHOR> http://www.economist.com/news/science-and-technology/21702343-individual-atoms-offer-ultra-dense-information-storage-scientists-pave-way <comment><USER2> From -269ºC to -196ºC is actually a huge improvement. Its the difference between liquid helium (expensive) to liquid nitrogen (cheap). Getting something that cold is hard, but keeping it there is just a matter of insulation. There seems to be a different kind of physics when you get that cold. Stable single atom structures, superconducting magnets, etc. It's all based around the idea that nanometer structures are now stable.

The big issue with bringing these kind of products to the general market is that we operate at a much higher temperature, the range of liquid water. And these technologies may never be able to improved to a point that they work at room temperature. Instead of moving those technologies into our range, I think could focus on moving our technologies to that range and improving/shrinking containment vessels. Self contained units with periodic maintenance/refills are viable (eg vacuum tubes). Having someone to top off a super computer's liquid nitrogen every couple of weeks is not a hard ask. We just have to make everything else work at that temperature so the entire unit can be cooled and contained, instead of just a specific section. Integrated circuit boards are already black boxes to 99.9% of the public, so sealing them in a cooled vessel is viable.

I know MRI/NMR machines already operate at that range, but they are way to big/specialized to be considered the general market. No one has an MRI machine in their medicine cabinet. I was once interning for a chemist, and asked if he ever thought we would have desktop NMR machines. He said there was no way they would ever be small enough. I then pointed at his laptop and remarked that is what people thought about computers 50 years ago.

<comment><USER3> The problem I see is what happens if the temperature doesn't stay in the required range for whatever reason like power failures, earthquakes, sloppy maintenance - you lose the data. Having such an enormous storage that is so fragile seems to have limited usefulness. If they ever get it to room temperature and have it be stabile then it might be more interesting.

<comment><USER4> There is an existing engineering discipline centered around keeping a lot of precious fragile data at liquid nitrogen temperatures for decades without malfunction. Say what you want about cryonics but the storage part is proven technology.

<comment><USER5> As I understand it the tract record isn't actually that good.

<comment><USER6> track

<comment><USER7> If you're going to correct people, at least specify the correction unambiguously. Pedantry is uhpaulleng.

[http://www.merriam-webster.com/dictionary/track%20record](http://www.merriam- webster.com/dictionary/track%20record)

</comment></comment></comment></comment></comment></comment><comment><USER8> This is because it just got published in Nature something, the earlier open arXiv version from April is here: [https://arxiv.org/abs/1604.02265](https://arxiv.org/abs/1604.02265) Figure 3 is absolutely spectacular, and deserves to be admired by the whole world. A comment on the inevitable promises of revolutionized data storage: Yes the areal density is fantastically high (>500Tbit / square inch as opposed to 1Tbit/in^2 in bleeding edge HDDs / NAND flash, if I crunched the numbers correctly). But it requires ultrahigh vacuum, preparation of a clean copper crystal, dosing with copper chloride and then writing/reading with a scanning tunneling microscope, maintaining liquid nitrogen temperature. Also a modern SSD will write about 500MB in a second, while this method would write 500MB in 240 years. I don't mean to slag it off; we should all appreciate it for being an absolutely wonderful and awe inspiring technical feat. Just don't get carried away dreaming of the applications in your laptop/server/phone.

<comment><USER9> Pretty cool, but keep in mind that it's at a massive scalability disadvantage compared to flash because of the requirement for a read/write head. Current 3D flash may only be 1.7Tbpsi, but that's with a layer 4 microns thick. Make it a millimeter thick and you're in the same areal density range.

</comment></comment><comment><USER10> While this is very intesting, the storage mentioned does seem somewhat small to me. The article compares the area covered by the text to be a little smaller than a HIV (Human Immunodeficiency Virus). However, the genome of HIV apparently stores approximately 9.2kb (kilo basepairs, not kilobytes, so ~18KiB) of information. It also doesn't need to be kept at -196C. Are there any viable tactics for atom-based storage that could rival this density?

<comment><USER2> Genetic storage is way more resistant to bit rot than conventional computers. DNA is not stable, has a full time support system fixing it, and still has a relatively high mutation rate. But biological systems on the whole are much more resistant to individual errors.

Yes, single mutations can cause horrible diseases, but these mutations are either in the active site of proteins, or disrupt folding. A large portion of proteins are "bulk." If you look at protein simulations, only a small section of the protein is simulated, and the rest is just approximated as charged mass. Changes there are much less impactful and noticeable. As for it to be a disease, it had to be present at time of concept. If it happens during life (which is does, millions of times a day), the cell either ignores it or dies.

<comment><USER11> DNA kept cool and dry is stable for millenia. Any storage done with DNA would likely be done in a fixed environment outside of the cell.

<comment><USER12> I thought natural radioactive decay was sufficient to mangle DNA.

<comment><USER13> Same for other storage devices.

</comment></comment></comment></comment></comment><comment><USER14> [http://trillian.mit.edu/~jc/humor/Ms_fnd_in_a_Lbry.html](http://trillian.mit.edu/~jc/humor/Ms_fnd_in_a_Lbry.html)

<comment><USER15> Wow! Excellent read!.

It's even more impressive since the footnote says:

"This story first appeared in the Magazine of Fantasy and Science Fiction, December 1961"

 _Fifty-five_ years ago!

It reminds me of a recent hn article (if anyone has a link it would be great!) that I believe it was about how Dropbox tests it's storage system, layer by layer, and the final error graph is always a flat line at "0", since every error is caught and corrected in one of the layers below.

Will it ever reach the case where it will be impossible to really guarantee that we have the right data?

PS: I can't seem to find the article using Algolia or even Google... maybe it was not Dropbox but other storage company? I really would like to find this story but for some reason never marked it as favorite or even upvoted it! (maybe I wasn't logged in).

<comment><USER16> Perhaps it was [https://blogs.dropbox.com/tech/2016/07/pocket- watch/](https://blogs.dropbox.com/tech/2016/07/pocket-watch/) ?

        > we use a variant on Reed-Solomon erasure coding that is similar to Local Reconstruction Codes
        > according to this model, a given block in Magic Pocket is safe with 99.9999999999999999999999999% probability!

<comment><USER15> Awesome! Yes, exactly that.

I tried searching for "petabyte" but I guess I came 1 order of magnitude short...

Thanks a lot! It was driving me crazy..

</comment></comment></comment></comment><comment><USER17> Sure, maintain your chlorine lattice at -196ºC and then use your handy scanning tunneling microscope to manually move atoms around.

I would say "paving the way to large-scale storage" is extremely generous for what they actually achieved, to the point of being incorrect.

<comment><USER18> I cannot imagine what you think "paving the way" is supposed to mean.

<comment><USER19> It means there is now a known route to reach a certain destination.

</comment></comment></comment><comment><USER20> This technology is amazing, but what's even more amazing is that writing at the atomic level is only 100x denser than what we've already achieved.

</comment><comment><USER7> This is pure buzz. Physical storage does not make any sense when light based or magnetic based storage is much faster and denser. It only makes sense if it is very permanent, ie. Etching sapphire discs [1] The position of atoms is just one classical property. Quantum properties like spin and charge are much more numerous and versatile.

[1] [http://www.digitaltrends.com/cool-tech/nanoform-laser- etched...](http://www.digitaltrends.com/cool-tech/nanoform-laser-etched- sapphire-data-storage/)

<comment><USER15> That sapphire disc tech is very interesting.

And while browsing their product sheet [1], I thought this would be great for me to store my most valuable stuff. And then it hit me... if I bought the 500 or even the 2500 document disc... what would I save there?

Probably some family pictures (unfortunately it doesn't seem to support colors, understandably so), a paper I published years ago. Maybe my college thesis. Legal documents perhaps like birth certificates and such.

But then I wished I could store music. Is there any graphical representation of music that would suit this format? Or is the only reasonable option to store the actual 1's and 0's of an MP3 or something like that?

[1] [http://www.fahrenheit2451.com/index.php/shop](http://www.fahrenheit2451.com/index.php/shop)

<comment><USER21> The resolution is too low to store even a single stereo FLAC file of a three minute song.

But this isn't really a viable digital storage system. It's a nice novelty, but it's not designed for serialisation in the way that optical ROM systems are.

It might be possible to increase the resolution with a better laser and optics, and make the disks serialisable. But I suspect you still wouldn't get the density you need for practical media storage.

<comment><USER22> I think you're wrong.

Their smallest listed item (1" medallion) is 110 megapixels. Even storing just 1 bit per pixel (which is conservative) you can store almost 3 minutes of CD- quality FLAC[1]. Seeing as it's grayscale, you can almost certainly store multiple bits per pixel. At even 2bpp, you would have plenty of space for 3 minutes of flac, plus some form of BCH code.

The largest listed item is 30GP, which would probably be enough for over a day of FLAC audio.

1: 110Mbit / (32bits/sample * 44100 samples/s) ~= 78s of uncompressed audio. FLAC is roughly 50% compression ratio, so ~= 2:36 of FLAC.

[edit]

In case people were wondering: the reason this didn't pass my "sniff" test is that War and Peace is shown on an example prototype. That is ~3MB in UTF-8 and a grayscale picture of a page of text at a readable resolution is about 2 orders of magnitude larger than the binary representation, and 300MB is obviously way more than a 3 minute FLAC.

</comment></comment></comment></comment><comment><USER23> Datacenter on the moon? At -200 degrees Celsius in the shade, the temperature on the moon is perfect. Of course, moving the data would be a problem but it would be a great backup solution!

<comment><USER21> You could do a daily backup of the entire Internet to something smaller than a thumb drive. :)

</comment></comment><comment><USER24> > Dr Otte reports read speeds of 1-2 minutes per block. He reckons he could > boost those speeds drastically, to about a megabit per second. That would be > a big improvement, but still thousands of times slower than modern hard > drives.

Try tens of times slower. There would absolutely be applications for nonvolatile memory that reads at 1 Mbps but has this kind of insane memory density. For most applications you'd probably have to make the technology stable at room temperature though.

<comment><USER25> Thousands is correct. A magnetic hard disk has a bandwidth of well over 1 Gbps.

Long-term stability (and this would only be any good for long-term storage, at that rate!) is a pretty tall order. Even current storage methods can't really manage it.

<comment><USER24> Well, I did mix up my units - I should have said hundreds. Megabits is a weird unit to use for hard drives.

But read speeds on a spinning HDD are typically going to be between 50-100MBps, so 400-800 Mbps.

</comment></comment></comment><comment><USER26> I am wondering if the technique described in this Nature publication from yesterday [1] to potentially operate quantum computers at room temperature, could be used for the atomic storage as well. Does anyone know?

[1]: [http://www.nature.com/ncomms/2016/160718/ncomms12232/full/nc...](http://www.nature.com/ncomms/2016/160718/ncomms12232/full/ncomms12232.html)

</comment><comment><USER27> anyone else notice that some of the transcriptions seem wrong, assuming a fixed 8-bit encoding? for instance. the "rrange" doesn't have two columns with identical bits.

<comment><USER28> Look at the rows

</comment></comment><comment><USER29> How vulnerable is this to cosmic rays?

</comment><comment><USER30> Interesting. Hadn't considered the applications of atomic storage before</comment>


Instagram on the Map - <AUTHOR> http://instmap.com <comment><AUTHOR> Do you know how many interesting people around you right now?

Instmap let's you find Instagram photos nearby or anywhere in the world.</comment>


Tacotron 2: Generating Human-Like Speech from Text - <AUTHOR> https://research.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html <comment><USER2> One of the domains where realistic and fully customizable speech synthesis could usher in a renassaince of innovation, would be video games developement.

I've always been curious about how so seemingly little work has gone into speech generation, compared to graphics and physics and other technologies.

Problems with voice actors:

• The costs, often unaffordable by solo developers and small teams.

• All the text and the scripts have to be finalized and set in stone. You can't change much, if at all, after you've finished recorded everything. Being able to re-hire the same actors for future sequels is also not guaranteed.

• Complete lack of real-time customization. For example, in MMORPGs and other online games. So we're stuck with a few set phrases or using our own voice (not desirable all the time for everyone). Imagine if you could fully customize your voice as well as you can do with your character's appearance, and have it realistically speak any text you type, or NPCs actually speaking your name instead of using generic placeholders.

• Players expect voice acting in all "major" titles. Plenty of games with great stories get skipped over if they don't have voice acting.

<comment><USER3> > Being able to re-hire the same actors for future sequels is also not > guaranteed.

Wow, it was not in my radar that AI could replace voice actors' jobs, but your comment has opened me to this idea.

To give an example, in Spain (I'm Spanish), in the Simpsons series after several seasons the much beloved voice actor for Homer Simpson died. They had to replace it ofc with another one. Yet, I know that many people in my circle started stopping watching the Simpsons at that time because they couldn't stand the new voice. Regardless of whether the new voice was "worse" or "better", it was certainly different, and we humans get weirded with that.

In light of this, I could see animated pictures with fully AI-rendered voices. Incredible.

<comment><USER4> I assume you know about [https://lyrebird.ai/](https://lyrebird.ai/)

 _Lyrebird allows you to create a digital voice that sounds like you with only one minute of audio._

<comment><USER3> Yes, just tried to record my voice, but the service didn't finish. Can be a temporal glitch.

The fake voice by Donald Trump is terribly close.

This definitely has a repercussion for fake news and overall trust. We will just not be able to trust what we see (computer-rendered images) or hear (rendered voice).

</comment></comment></comment></comment><comment><USER5> Huge kudos to authors for being upfront about what doesn't work. I am getting pretty tired of people not doing this more consistently and only put out the most attractive results even when they would have observed many not so good results.

 _While our samples sound great, there are still some difficult problems to be tackled. For example, our system has difficulties pronouncing complex words (such as "decorum" and "merlot"), and in extreme cases it can even randomly generate strange noises._

</comment><comment><USER6> A while ago, someone posted this sample from their TTS startup: [https://instaud.io/KlA](https://instaud.io/KlA)

Anyone remember the company?

<comment><USER7> Might it have been Lyrebird[0]? They're (as far as I can tell) they have the best text-to-speech that you can actually use. It's kinda annoying when Google makes these announcements about their advances in TTS only to reveal that it's not actually something you can make use of, and no, the dataset they used to train their model is not available.

0\. [https://lyrebird.ai/demo/](https://lyrebird.ai/demo/)

<comment><USER6> Nope, that is definitely not it. It might have been a smaller, AFAIR french startup. The (english) demos were all weather related.

</comment></comment></comment><comment><USER8> I am constantly surprised by how robust and versatile Mel spectrograms (and Mel frequency cepstrum) are, despite the filterbanks and the transformations being relatively arbitrarily engineered and performed on evenly spaced frames.

These results seem fantastic nonetheless! I anticipate they'll be able to optimize the system to real-time generation or better within the next year, looking at WaveNet.

</comment><comment><USER9> this is the first time I can't tell the difference between the synthesized voice and a human voice.

<comment><USER10> I thought it was pretty easy to pick the audio clips with greater variance in pitch and pronunciation. 2-2-2-1 were the human voices.

<comment><USER11> Haha, try again, the human is 1,2,2,1 according to the filenames (I was fooled too).

I do think the difference would become obvious with a paragraph or more of speech, though. It's difficult to judge what the correct intonation should be on these single sentences without context. Ultimately, correct intonation requires a complete understanding of meaning which is still out of reach. An audiobook read by tacotron 2 would still sound strange.

<comment><USER10> Depends on the audiobook. I think technical docs would be alright, which is what I want this mostly for. Lots of technical docs I'd like to listen while I work out.

</comment></comment></comment></comment><comment><USER12> I've been wondering lately, it seems like audio books might be an amazing training resource for models like these, if you could get the script that the reader was working from!

<comment><USER13> Your wish, granted. [http://www.openslr.org/12/](http://www.openslr.org/12/)

It's 1000 hours of audio book readings, segmented by sentence, with transcripts. All from project Gutenberg, so maybe a little bit heavy on Victorian bodice rippers and such, but certainly a great trove of training data...

<comment><USER14> Project Common Voice [https://news.ycombinator.com/item?id=14794654](https://news.ycombinator.com/item?id=14794654)

<comment><USER15> That data is no good for this purpose, as it's from a lot of different speakers and does not have speaker labels, i.e., you can't tell which sentences were spoken by which speaker.

</comment></comment></comment></comment><comment><USER16> I wonder how hard it is to get this effect in other languages. "The google translate lady" is really helpful for my foreign studies but I wonder how robotic it sounds to a native

</comment><comment><USER17> Arstechnica mentioned this research in an approachable article [1] on the challenges of getting speech recognition and generation working for more languages. I found it fascinating.

Very cool stuff.

[1] [https://arstechnica.com/information- technology/2017/12/teach...](https://arstechnica.com/information- technology/2017/12/teaching-old-virtual-assistants-new-language-tricks/)

</comment><comment><USER18> I suppose an API will come to Google Cloud platform to use this service sooner or later ? Or, is it still on the research side (not production ready) ?

<comment><USER19> It's still a research project and not a production system:

"We manually analyze the error modes of our system on the custom 100-sentence test set from Appendix E of [11]. Within the audio generated from those sentences, 0 contained repeated words, 6 contained mispronunciations, 1 contained skipped words, and 23 were subjectively decided to contain unnatural prosody, such as emphasis on the wrong syllables or words, or unnatural pitch. In one case, the longest sentence, end-point prediction failed."

<comment><USER20> To add: > Also, our system cannot yet generate audio in realtime.

For an production GCP API, I think faster than real-time would be necessary.

For example, WaveNet took a year to go from research to production in Google Assistant: [https://deepmind.com/blog/wavenet-launches-google- assistant/](https://deepmind.com/blog/wavenet-launches-google-assistant/)

</comment></comment></comment><comment><USER21> On the audio samples pages, at the end, which one do you think is the human one?

[https://google.github.io/tacotron/publications/tacotron2/ind...](https://google.github.io/tacotron/publications/tacotron2/index.html)

</comment><comment><USER10> It'd be nice if they could make this a paid service at least.

<comment><USER22> Maybe services like AWS Polly would upgrade over time to be as good as this. (Sadly, Polly is currently worse than the standard macOS TTS, IMHO..)

</comment></comment><comment><USER23> I'm incredibly disappointed that this isn't related to tacos or robots, more importantly robots making tacos.

<comment><USER24> A tacos-making robot is already a level of a project for university students.

</comment></comment><comment><USER25> This is great - but I imagine they are trying to get this to work in Realtime.

<comment><USER26> Ooooh, good point.

</comment></comment><comment><USER27> is this available to use by non-googlers?

<comment><USER28> Apparently not? I imagine it would have to be replicated by someone else. One also assumes that the volume of training data - I don't actually know - is not insignificant!

<comment><USER29> From the paper, the training data is surprisingly small:

"We train all models on an internal US English dataset, which contains 24.6 hours of speech from a single professional female speaker."

<comment><USER28> Uncompressed, that's still quite a lot of data - perhaps not relative to other ML projects, but still.

<comment><USER30> They say they split the audio in "80-dimensional audio spectrogram with frames computed every 12.5 milliseconds". The picture from the post supports that.

For 24 hours that would be 7 mil frames of 80 values ~= 500 mil data points, or 2 GB of raw data (assuming floats)

<comment><USER31> So, something that easily fits into RAM, One might even keep a copy of the whole dataset in GPU memory to avoid copying to and from even if matrix operations are done on smaller batches.

E.g. ImageNet is 50 gb of compressed data, and there are many much larger datasets in practical use.

</comment></comment></comment></comment></comment></comment><comment><USER32> Can the research team find a way to display a blog entry without requiring javascript? Perhaps some sort of mark up language would be ideal. The current version requires all sorts of crap from 30 different domains and at least 2 trackers. By default with uBlock I got just the header.</comment>


The Case Against Sugar - <AUTHOR> https://aeon.co/essays/sugar-is-a-toxic-agent-that-creates-conditions-for-disease <comment><USER2> Salient passage:

 _" Answers began coming with the very first publications out of Yalow and Berson's laboratory and were swiftly confirmed. As it turns out, virtually all hormones work to mobilise fatty acids from fat cells so that they can then be used for fuel. The one dominant exception to this fuel-mobilisation signalling is insulin, which partitions how we use the fuels we consume: in particular, it directs fat cells to store fat, while facilitating the uptake and oxidation of glucose (blood sugar) by muscle and organ cells. In other words, when insulin is secreted – primarily in response to the carbohydrates in our diet – it directs our cells to burn carbohydrate as fuel and store fat. And so, the one biological factor necessary to mobilise fat from storage and have it used for fuel, as Yalow and Berson suggested in 1965, is 'the negative stimulus of insulin deficiency'. Put simply, when insulin levels in circulation are elevated, we store fat and use glucose for fuel; as insulin levels drop, fat is mobilized and we burn it instead."_

 _" Yalow and Berson themselves described insulin as a 'lipogenic', or fat- forming hormone. This lipogenic signal must be turned off, or at least muted significantly, for the fat cells to release their stored fat and for the body to metabolise it for energy. While obesity researchers like to say that the sine qua non of a weight reduction diet is calorie-restriction, this alternative, biologically-based hypothesis would say that the sine qua non is lowering insulin. The more we consume carbohydrates, though, and particularly sugar, the higher insulin our levels will be."_</comment>


Facebook isn't introducing the privacy focus Mark Zuckerberg touted at F8 - <AUTHOR> https://www.fastcompany.com/90342277/facebooks-fb5-is-cleaner-and-groups-centric-but-not-much-more-private <comment><USER2> I wish it was a longer writing</comment>


AngularJS application starter comparison - <AUTHOR> http://www.dancancro.com/comparison-of-angularjs-application-starters/ A side-by-side comparison of things you can use to get started making an app with AngularJS <comment><USER2> Am I the only one around here who prefers starting from scratch in a blank vim session?

<comment><USER3> I do like generators for one reason - setting up a Gruntfile with all the details you want can be tedious (in addition to all the dependencies in your package.json), not to mention setting up stuff like Bower.

That's a bunch of wasted time setting up boilerplate. Generators tend to not be perfect, since we as developers can get opinionated, but I'm fine with tweaking from the boilerplate to get the results I want, which takes less time than setting everything up from scratch.

<comment><USER4> Dude setting up an angular project can be a huge headache. We use a hand built generator just for our needs, but npm in so insanely unstable that we have to get in and end up doing a lot of configuring of the gulp / gruntfile and packages etc.

<comment><USER5> I don't recognize your problems at all. Do you mind elaborating on what headaches setting up an Angular project gives you?

<comment><USER4> I guess I'm complaining about npm more than Angular per se. Most of our issues stem from the myriad package dependencies "in the wild" of npm itself. It's like tilling a field in the wind. Often one has to `npm install` >3 times to install all of your modules. I guess that's what happens when you have ten billion cooks in the kitchen.

<comment><USER6> Is using npm worth it? Would it be simpler to download the dependencies yourself and put them into a vendor directory?

<comment><USER7> The web development industry, and particularly the front-end side, currently seems to be obsessed with (or maybe plagued by) tools that sort of do half the job you sometimes want but feel like you have to always use anyway because everyone else is or appears to be or at least says you should on their TwitBookBlogFeed.

I wanted to do exactly what you describe yesterday -- just download the latest stable version of a one-JS-file library to include on one page of a web app where its functionality is relevant. We used to do this by downloading example-1.0.8.min.js from the obvious Download link on www.example.com.

As it turns out, the project from yesterday lives in a GitHub repository with a dozen files scattered across a root and four subdirectories. I honestly have no idea how to build it; it's written in a compiled-to-JS language, the build instructions are a one-liner that refer to a build tool I've heard of but never used, and there are at least half a dozen other files in the project that seem to be there only for some sort of compatibility with various other tools I've heard of but never used/don't use on this project/don't have installed on this workstation.

I find myself reminded almost daily of this now-infamous XKCD: [http://xkcd.com/927/](http://xkcd.com/927/)

This is not progress. :-(

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> I think Closure Compiler integration is really important for Angular, since it has built-in support for the framework (it allows automatic injection based on parameter names). It also does really good minification and dead code elimination. Sadly, it's still pretty hard to get started with, I think [https://github.com/zemirco/nghellostyle](https://github.com/zemirco/nghellostyle) is the best example I've seen.

<comment><USER9> Isnt the compiler written in java? I think that's the heart of the problem, you aint gona get the javascript community behind that tool if it's not written in js,or at least something that cant be easily packaged with npm.Time to write a Java2Js transpiler,in javascript...

</comment></comment><comment><USER10> Seems odd to make a decision on framework by watching a couple of videos, but then compare dozens of application starters along hundreds of dimensions. I would think the choice of starter would be much less important.

Between the choice of framework, package manager, task runner, test framework, test runner, CI service, seed project, and a million plugins for each of the above, I have no fucking clue how anyone keeps up or writes a line of code anymore. And that's just the front end.

</comment><comment><USER11> This is a pretty good run-down. I'm somewhat in the same boat, so it's nice to see this breakdown.

</comment><comment><USER12> Thanks for your work, interesting information.</comment>


3 more reasons to use node.js - <AUTHOR> http://dailyjs.com/2011/03/30/node-roundup/

<comment><USER2> NowJS looks pretty interesting. From monitoring the IRC room maybe some of the common use cases could be better documented but it seems like it's run by a great group of guys.

<comment><AUTHOR> Thanks for the feedback. In the next two weeks, you'll see a better Demos page (common uses/examples w/ sourcecode) and better Documentation page.</comment></comment>


The Mathmo Test - <AUTHOR> http://www.mathmo.org/test/mathmotest.html

<comment><AUTHOR> I only scored a 29, but I laughed when I read the question about making a model of a dodecahedron, because I did at age 12 or so. That got me to complete the test and to feel that some of the questions were indeed designed with me in mind. :-)

<comment><USER2> 31\. I think I made a dodecahedron, but I can't really remember so I didn't count it. I never had a Rubik's cube and never felt compelled to get one.

</comment></comment><comment><USER3> I scored 49, which is less than when I first took it many years ago as produced by the original authors.

Anyone else want to post their scores?

<comment><USER4> 23 and in my first semester as a math grad student (coming from a non-math background). I can already see myself heading down the path to a higher score. On a related note, I became a member of the AMS today... that should count for something :)

</comment></comment><comment><USER5> I scored 6. It seems I'm not into MathMo that much. What does it measure anyway?

</comment><comment><USER6> 24\. What's a mathmo?

<comment><USER7> It's slang from Cambridge University (and maybe elsewhere, but that's where I encountered it) for a student studying mathematics. Related: 'NatSci' (pronounced natskee) for someone studying Natural Sciences (the course that covers all the traditional physical and biological sciences); 'CompSci' (pronounced compskee) for people studying Computer Science.</comment></comment>


Where in the world it is easiest to get rich (Ted talk) - <AUTHOR> https://youtube.com/watch?feature=youtu.be&v=A9UmdY0E8hU <comment><USER2> First, this is a TEDx. Second this based on defining rich as based on the wealth report, individuals net worth more than 30m US dollars. (Ultra High Net Worth Individual (UHNWI))

So what he did is use the "Per Capita" rate to establish that Norway was #1, even though if you look at total the US comes at top.

So typical TEDx

<comment><USER3> What measure would you use instead of the per capita rate?

How else would you correct for population size?

</comment></comment><comment><USER4> Now can I know which country has the most net-salary millionaires who used to be regular software developers?

</comment><comment><USER5> Summary?

<comment><USER6> In which society is it easiest to get rich? Contrary to common belief, it is not countries like the US or the UK that create the highest number of rich people per capita, but Nordic social democracies like Norway and Sweden. Counter intuitive as it may sound, high taxes, generous welfare states and strong unions makes a better environment for the people who wants to earn huge amounts of money, than free markets, low taxes, and minimal government intervention.

<comment><USER7> First, define "huge amount of money" because obviously Nordic countries have some alternative definition of it :) How many self made billionaires are in Sweden?

Second, the way of becoming rich country haven't changed for the last few millenniums: you have to exploit cheap labor from a large number of slaves (sorry, now we call them immigrants) Therefore becoming rich strongly depends on being born in right race and social class.

Lastly, it's amusing how so many social issues get blamed on capitalism and free markets when both of those things are pretty much nonexistent in most countries - even US have some heavy regulation in place.

<comment><USER8> I was quite surprised that the Nordic countries have more reported billionaires per capital than Canada, despite the Nordic nations being —theoretically— less business friendly.

Example: Canada has about 1.08 billionaires per million people, while Sweden has 3.16. Norway, with its oil, performs more poorly, but still better than Canada: 2.69 billionaires per million people. Denmark is only slightly worse than Canada at 1.05.

Source of my information: [https://en.wikipedia.org/wiki/List_of_countries_by_the_numbe...](https://en.wikipedia.org/wiki/List_of_countries_by_the_number_of_billionaires)

<comment><USER9> > less business friendly

Just hypothetically, which of these two made up countries do you consider more business friendly:

* Country 1 with low taxes and zero labor and environmental laws

* Country 2 with a highly educated work force and low crime but high taxes

<comment> apk-d Crime isn't really a big factor, I think (maybe excluding weird failed states and countries that are at war) and work force is easy to train if you can afford the initial investment.

Low taxes + low bureaucracy go a long way.

<comment><AUTHOR> No fun being rich in a country where public infrastructure are broken e.g crazy congestion</comment></comment></comment></comment></comment></comment></comment>


Funny Motivational Joke (MUST READ =] )  - <AUTHOR>

So I was on my Facebook, and I stumbled on something extremely funny, so I thought I ought to share it with my favorite community.
This guy posted a motivational quote on his feed, he wrote "Success is when you fail over and over again"
One dude commented on the feed "dude, that's called a loser".
I just thought it was really funny =) I guess lesson of the day is that it's okay to fail, but it's essential to learn from your mistakes? Just a food for thought. -KG <comment><USER2> sounds like that belongs on failbook.com

<comment><USER3> haha totally</comment></comment>


Open Reference Architecture for Security and Privacy - <AUTHOR> https://security-and-privacy-reference-architecture.readthedocs.io/en/latest/ <comment><USER2> This is pretty cool. I have been considering collating a bunch of notes I've been collecting on Linux security into a free book which would be aimed at security researchers and system administrators who have the luxury of securing their systems beyond checklists... though I'm nowhere near realizing that.

A few comments on the design of the book here, though. It seems strange to me that privacy and security should be treated together. Security and privacy are frequently mutually exclusive. To get the best privacy you often have to sacrifice some security, and vice versa. An example would be allowing Windows Defender to automatically submit data and files to Microsoft. This increases security but decreases privacy. Another example is how Google requires you to submit a non-VoIP phone number during Google account creation. Obviously, this decreases privacy, but it also prevents spammers from flooding the comments section of YouTube videos with links to sites hosting malware (this used to be a huge problem). Of course there are many controls that increase both privacy and security together. But the relationship is complex, and I think the only way to write a clear book for specialists with actionable guidelines is to place either security or privacy as the priority, not both.

Also, at first glance I am not sure if the book is meant to help administrators and businesspeople design services that are secure and protect end-user privacy, or if it is meant to help end-users themselves protect their privacy/security, or both [Edit: 1]. In the Introduction: "This reference architecture is created to improve security and privacy designs in general." Chapter on security principles seems aimed at the admins too. But there is a whole chapter on OSS Privacy Applications that seems target end-users and show them how they can protect their privacy. I am left wondering: "Is this book for me? Maybe. Idk." Maybe I skimmed too quickly, but it really seems like it's trying to address too many audiences at once.

Maybe the authors can comment on why they made these design decisions.

[1] Edit: add to that developers, with the chapter on secure coding guidelines.

<comment><USER3> Not an author, but there's a lot of overlap. Code bugs lead to breaches which lead to data theft and loss of privacy, namely.

Figuring out various strategies for tightening up that causality chain improves both security and privacy. Formal verification and security proofs of cryptography code. Provably correct software systems. Comprehensive testing. Using application or systems frameworks that have been developed with a security-first mentality, rather than security-bolted-on mentality (too many today use the latter). Etc.

</comment></comment><comment><USER4> This looks like an amazing resource. Security and Privacy is intimidating for engineers and the resources and training are currently hoarded by massive enterprises. Really excited to see stuff like this become widely available.

</comment><comment><AUTHOR> Paperback: [https://www.amazon.com/dp/1540606481/](https://www.amazon.com/dp/1540606481/)</comment>


Analyzing GitHub, how developers change programming languages over time - <AUTHOR> https://blog.sourced.tech/post/language_migrations/ <comment><USER2> Unfortunate they had to exclude Javascript. I understand why they chose to do that, but that's a HUGE chunk of data that's been pretty much randomly ignored. So can this really be considered a fair analysis given that?

<comment><USER3> No one has a choice with JavaScript. Theoretically you could go native or transpile but it's very rare.

Therefore, if you included it, the data wouldn't lead to meaningful conclusions. I feel like this was pretty obvious from the article and that the explanation was enough. For example, substituting node for js seemed to work well enough.

<comment><USER4> No one really has a choice with C, but its tally was very interesting..

I think there is a lot more vertical movement and the horizontal stuff might be a sideshow without considering overlap or experience with C or JavaScript as significant in how one transitions between purely competing languages.

But JavaScript's real problem for the analysis might be that its competition is largely excluded since the walled garden apps rarely have related code on GitHub, even if their language is present.

<comment><USER5> There's a huge choice for C. For a lot of uses, you can use C++, Go, Java, Rust, or a few other systems languages. The only thing that _needs_ C is some very niche embedded stuff, or the Linux kernel (also niche).

<comment><USER6> All the major Unix kernels are in C. If it continues this trend, it seems clear that new kernels in new languages will be needed, or existing kernels will need to be ported to other languages. A memory safe one would be cool...

<comment><USER7> Or maybe C is uniquely suited to the task of writing a kernel, after all that's what it was designed for.

<comment><USER8> Not really, kernels were being written in HLL 10 years before C existed, and at strange places like IBM research.

The only good thing about it is that the language is easier to write a compiler for, as it is basically a portable macro assembler, specially the K&R C variant.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> This looks dubious to me. Not least do to the healthy flow of people moving from other languages to Visual Basic.

Also byte flow makes little sense for programming languages. lower level languages are going to be more verbose than high level languages. They'll be used for different things. Some will have tonnes of boilerplate that travels with the project (e.g java). Forked projects? etc.

Further, to me it seems that this ought to be a more descriptive thing of how something happened in the past and not subject to probability unless the claim is a prediction about next years conversions between languages or that conversions are stationary over time.

I.e a set of metrics that proxy transitions and an order list of from and to, would be just the ticket IMO.

</comment><comment><USER10> Some possible confounding variables: what if certain language users are more likely to squash commits? what if certain language users are more likely to have private repos?

<comment><USER11> I don't think squashed commits matter since they used project byte size as a filter to get rid of "Hello World" sized noise.

Hard to tell how much private repos would sway the results. Maybe a large number of COBOL programmers are stuck behind their organization's private repos and all we see are the languages they play with on the side for fun?

<comment><USER12> I think it could make a pretty large difference, actually. Languages like JavaScript (I know it's not included here, but still), Ruby and Python have large open source communities creating time of libraries in common usage. Compare that with a language like C#, which has an extensive standard library and comparatively less open source third party libraries. You'd expect that even if there was an equivalent number of private repos in both cases, the languages with a larger open source ecosystem would appear more popular in this analysis.

</comment></comment></comment><comment><USER13> Java -> Kotlin is still pretty low. It would be interesting to revisit this in a year.

</comment><comment><USER14> I used to think I understand Linear Programming, and the transportation problem. Is there a relationship between this and the Markov formulation? I'm totally confused now. Posts like this make me feel guilty about not reviewing them once in a while. And I guess while I'm at it with the questions:

>We have to add an artificial source and sink on both sides of our bipartite graph to ensure flow conservation

Wasn't there a hack with the slack/surplus variable in the LP constraints to deal with this or was it a dummy variable? Pretty sure that was able to handle the case where supply was not equal to the demand.

Also, how were cases where the user stopped using GitHub altogether or a new user started programming are handled?

</comment><comment><USER15> I would love to see this language transition thing as a graph - matrices are rarely the most insightful tool to visualizing such data.

</comment><comment><USER16> For reference [https://madnight.github.io/githut/](https://madnight.github.io/githut/) without excluding Javascript

</comment><comment><USER17> I'd like to see this data expressed in a sankey diagram. (I mistyped it as snakey diagram at first... that's a good way to remember it!)

</comment><comment><USER18> People do so much advanced analysis with these outrageously biased datasets (this says nothing about "developers", this has analysis of "developers who put repos on github, skewed towards prolific repo creators")

Yes, it's the only dataset you have. You still sound dumb when you inflate the importance of the population you have data for to make the anlaysis sound more useful.

<comment><USER19> Relax, it's just a blog post from a "machine learning intern", and sounds like just the kind of project you'd give an intern for experience.

Anyway the first paragraph also says: _Thus, it has become engaging to deepen this idea and see how the popularity of languages changes among GitHub users._

I don't get the sense anyone is trying to inflate the importance of anything.

</comment></comment><comment><USER20> Very interesting analysis. It seems to correspond with reality quite a bit more than the Bernhardsson analysis.

</comment><comment><USER21> In 2-3 years, I guarantee you that Elixir will appear strangely absent from this blog post

Source: Consistently steep slope over time of Indeed job interest in Elixir plus the fact that ElixirConf doubles in size every year

<comment><USER22> you're joking, right?

[https://www.indeed.com/jobtrends/q-elixir-q-java-q- go.html](https://www.indeed.com/jobtrends/q-elixir-q-java-q-go.html)

<comment><USER21> YOU'RE joking, right? Because you're looking at the data entirely wrong. The fact that a relatively new language has a CURRENT fraction of the interest of a more established one is an idiotic comparison to make, all that matters are the rates of change, and I challenge you to find another language with the slope of the "Jobseeker Interest" line on this chart:

[https://www.indeed.com/jobtrends/q-elixir.html](https://www.indeed.com/jobtrends/q-elixir.html)

For comparison, Go, stagnant: [https://www.indeed.com/jobtrends/q-Go.html](https://www.indeed.com/jobtrends/q-Go.html)

Java, slight increase: [https://www.indeed.com/jobtrends/q-Java.html](https://www.indeed.com/jobtrends/q-Java.html)

Scala is the closest competitor I've found (and it's still not close): [https://www.indeed.com/jobtrends/q-Scala.html](https://www.indeed.com/jobtrends/q-Scala.html)

F#, stagnant: [https://www.indeed.com/jobtrends/q-F%23.html](https://www.indeed.com/jobtrends/q-F%23.html)

Haskell, stagnant: [https://www.indeed.com/jobtrends/q-Haskell.html](https://www.indeed.com/jobtrends/q-Haskell.html)

Clojure, stagnant: [https://www.indeed.com/jobtrends/q-Clojure.html](https://www.indeed.com/jobtrends/q-Clojure.html)

Erlang, stagnant: [https://www.indeed.com/jobtrends/q-Erlang.html](https://www.indeed.com/jobtrends/q-Erlang.html)

Rust, almost stagnant, extremely gentle slope: [https://www.indeed.com/jobtrends/q-Rust.html](https://www.indeed.com/jobtrends/q-Rust.html)

Elm is also rising fairly fast, but not as much as Elixir (almost, though): [https://www.indeed.com/jobtrends/q-Elm.html](https://www.indeed.com/jobtrends/q-Elm.html)

So, my point, WITH data: _One of these things is not like the other_

Here's the kicker: [https://www.indeed.com/jobtrends/q-Clojure-q-Haskell-q- Elixi...](https://www.indeed.com/jobtrends/q-Clojure-q-Haskell-q-Elixir-q- Rust.html)

Elixir is about to pass _Clojure AND Haskell AND Rust_ in developer interest, and shows no signs of abating</comment></comment></comment>


SQLite 3.8.7 is 50% faster than 3.7.17 - <AUTHOR> http://permalink.gmane.org/gmane.comp.db.sqlite.general/90549 <comment><USER2> I'm glad to see something like this. People often take the typical optimization mantra too far. The minute you mention optimization on most internet forums (especially StackOverflow) you get slammed for promoting "premature optimization."

That's not to say premature optimization doesn't exist. Sometimes writing faster code is the difference between using a hash instead of a list. Sometimes it's spending an entire day optimizing a function that's called once. The former is performance aware, the later is premature. It seems as though most would have you believe that they both fall into the "premature" bucket.

It's as though people are using it as an excuse for their personal ignorance and would rather continue to revel in this ignorance than expand what they know. As far as I am concerned these "unimportant tidbits" about performance are gifts that keep giving.

/somewhat offtopic rant

<comment><USER3> The argument behind premature optimization only really stands when first developing a product. Once you have an established product, it's no longer premature, it's just optimization.

<comment><USER4> Well, only if you never add any new features to it.

<comment><USER5> As long as you are not optimizing those new features as you are adding them, it's not premature optimization.

</comment></comment></comment></comment><comment><USER6> I think people under-estimate incremental improvements a lot. My tai-chi teacher told me that eastern culture is more about "putting one sheet of paper on another", there is slow and steady improvement. Western is much more about "big bang approach", make it or break it.

SQLite has been around for 15 years, they never shifted their purpose and were constantly improving. You get almost perfect product at this point.

I work on in-memory db which was evolving similar way. Now after more than decade of small improvements, it is faster than java collections, despite doing serialization and its own memory management.

<comment><USER7> Not only eastern cultures have the concept. In German there is the phrase 'a steady water drop hollows a stone' and it actually traces back to the Roman author Ovid 'gutta cavat lapidem'. Perseverance is just so much harder than instant gratification.

<comment><USER8> Interesting. The same phrase 'a steady water drop hollows a stone' also exists in Chinese: 水滴石穿, and it also traces back to 2000 years old classics.

<comment><USER9> I don't want to disappoint you, but the same phrase exists in so many languages that I'm afraid it isn't something culture-specific.

</comment></comment></comment></comment><comment><USER10> Ah, the good old 100/100 rule: 100% of the execution time is somewhere in 100% of the code.

Sometimes when you profile code you're dismayed to find that there are no clumps where the time is spent. It's everywhere. This function here takes 3%, this one 5%, that one another 3%, ... you either have to make some radical change, or else make all of these a little faster.

<comment><USER11> And if you make the radical change, you'll see that it will also have these 3% there and 5% somewhere else.

<comment><USER12> And it will probably break something along the way :-)

</comment></comment></comment><comment><USER13> Deep within another thread, I had a conversation with 'nnethercote' regarding Richard's use of Cachegrind to optimize performance. My position is that it is no longer a useful approach, as the actual behaviour of modern processors has diverged so much from Cachegrind's simple simulation. Instead, I think it is much more productive to use tools like 'perf' that use the CPU's built in performance counters to measure the actual performance.

My conviction inspired me to repeat Richard's test to see how the numbers compared when run on a current Haswell i7-4770 processor. Here's what I got for the versions he compared when compiled with GCC 4.8.1 and using 'perf stat speedtest1 --size 5'[1] (sizes are for the sqlite.o lib only):

      Cachegrind per Richard:
      3.8.7a -Os: 953,861,485 cycles
      3.7.17 -Os: 1,432,835,574 cycles

      Haswell:
      3.8.7 -O3: 469,956,519 cycles (870,767,673 instructions, 1,069,032 bytes)
      3.8.7 -Os: 542,228,740 cycles (884,541,860 instructions, 648,360 bytes)

      3.17.7 -O3: 651,680,545 cycles (1,320,867,830 instructions, 1,002,192 bytes)
      3.17.7 -Os: 771,304,471 cycles (1,406,527,795 instructions, 605,976 bytes)

One can make of the numbers what you will, but here are some conclusions I drew:

While the improvement is about the same magnitude that Richard sees, the actual number of cycles is off by about a factor of 2.

Despite having a larger binary, and thus theoretically worse instruction cache behavior, -O3 beats -Os by about 20% in speed in both cases, at a cost of 40% (400K/1MB) in size.

'perf record -F100000 speedtest1 --size 5; perf report' is a really slick and easy way to figure out where the program is spending it's time. If I were optimizing this, I remain fairly certain it would be more a more effective approach than using Cachegrind.

[1] I had to comment out the "sqlite3_rtree_geometry_callback" line in speedtest1.c to get it to compile, which might affect my numbers slightly, but from the source comments I don't think it was actually being used.

<comment><USER14> The cycle-counts returned by cachegrind are repeatable, to 7 or 8 significant figures. That means that I can make a small change and rerun the test and know whether or not the change helped or hurt even if the difference is only 0.01%. I don't think perf is quite so repeatable, is it?

Also, the cg_annotate utility gives me a complete program listing showing me the cycle counts spent on each line of code, which is invaluable in tracking down hotspots in need of work. If perf provides such a tool, I am unaware of it.

Remember that I'm not trying to optimize for a specific CPU. SQLite is cross- platform. I want to do optimizations that help on all CPUs using all compilers. I'm _measuring_ the performance on the "cachegrind virtual CPU" of a binary prepared using GCC and -Os because that combination gives repeatable measurements that are easy to map into specific lines of source code. But the optimizations themselves should usually apply across all CPUs and all compilers and all compiler optimization settings.

Nkruz is, of course, welcomed to use any tool he likes to optimize his projects. But, at least for the moment, I'm finding cachegrind to be a better tool to help with implementing micro-optimizations.

<comment><USER13> > The cycle-counts returned by cachegrind are repeatable, to 7 or 8 > significant figures. ... I don't think perf is quite so repeatable

It can come close. I just tried, and for 'speedtest1' seemed to be getting 3 significant digits for the cycle counts, and 4 for the instruction count. You'd probably gain another one or two if you were to measure computation only and remove the printf() and other I/O statements. The underlying performance counters are pretty much cycle accurate.

> cg_annotate utility gives me a complete program listing showing me the cycle > counts spent on each line of code... If perf provides such a tool, I am > unaware of it.

Yes, that record/report combination I quoted above does just this, with insignificant runtime overhead. Unlike the total counts, this one is sampled, so you might get a little more variation. It's definitely good enough for quickly finding hotspots, and there are other (harder to use) tools that can use the precise "PEBS" events you need complete counts. These even allow you to do nifty things like track the number of times each branch statement is mispredicted.

> I'm measuring the performance on the "cachegrind virtual CPU" of a binary > prepared using GCC and -Os because that combination gives repeatable > measurements that are easy to map into specific lines of source code.

Absolutely, this is the right way to view cachegrind. My question would be whether it is the right generic CPU to be using, and whether optimizations made on it translate well to other modern CPUs. Many of them will, but I think you'd have faster turnaround time even better success with an approach that uses a real CPU and its performance counters.

> I'm finding cachegrind to be a better tool to help with implementing micro- > optimizations.

Please realize I have the utmost respect for your work on SQLite. It's my most frequent answer when asked for an example of C code to study, learn from, and pattern after. I'm certain you will manage to optimize it with any tool you choose, but having spent many hours with GProf and cachegrind myself, I (will the zeal of a recent convert) think you'll be amazed with some of the things that are now possible with performance counters.

</comment></comment></comment><comment><USER15> How is this work funded? Are these developers working full time on SQLite, or is this work done in developers' spare time?

<comment><USER16> not going to say it is 100% of their time, but they are paid to work on it.

You can see five consortium members on the front page of sqlite, the cost of a consortium membership appears to $75,000/year[1]. That's enough to pay developers.

[1] [http://sqlite.org/consortium_agreement-20071201.html](http://sqlite.org/consortium_agreement-20071201.html)

<comment><USER17> And that doesn't even include Apple, who I have to believe has a big investment in sqlite, and probably has some form of support arrangement with them.

</comment></comment></comment><comment><USER18> I wonder what most of those optimisations _were_? In particular, are there any that are general enough to be worth knowing about, or even worth including at the compiler level?

<comment><USER19> Everthing detailed:

[http://www.sqlite.org/changes.html](http://www.sqlite.org/changes.html)

</comment></comment><comment><USER20> > We have achieved this by incorporating hundreds of micro-optimizations. Each > micro-optimization might improve the performance by as little as 0.05%. If > we get one that improves performance by 0.25%, that is considered a huge > win. Each of these optimizations is unmeasurable on a real-world system (we > have to use cachegrind to get repeatable run-times) but if you do enough of > them, they add up.

Interesting! That reminds me of my own experience in a different context (from [https://blog.mozilla.org/nnethercote/2011/07/01/faster- javas...](https://blog.mozilla.org/nnethercote/2011/07/01/faster-javascript- parsing/)):

> Cachegrind [does] does event-based profiling, i.e. it counts instructions, > memory accesses, etc, rather than time. When making a lot of very small > improvements, noise variations often swamp the effects of the improvements, > so being able to see that instruction counts are going down by 0.2% here, > 0.3% there, is very helpful.

<comment><USER21> Of course since processors are out of order and superscalar reducing instructions isn't always a worthy goal either. You might end up with less instructions but less things happening in parallel. And then of course you have to balance this with the cache efficiency of having to look at less instructions to complete a task.

Basically, optimization at this level is really hard.

<comment><USER20> And yet, in practice, I have found that optimizing for instruction counts works really well. I've gotten _way_ more mileage out of Cachegrind's instruction counts than I ever have out of its cache or branch prediction simulations.

<comment><USER13> I was tempted to respond after your first comment, but this followup provoked me to action. When you wrote your post several years ago, modelling with Cachegrind may still have been a defensible approach. But the divergence between its generic processor simulation and actual real world performance continues to diverge. As you note, this divergence first became very apparent with the cache and branch predictions. Currently, I'd argue that your time will almost always be better spent checking the CPU's built-in performance monitors rather than using Cachegrind. For the cases where Cachegrind used to be useful, 'perf' is a joy!

<comment><USER20> Well, the SQLite folks are clearly still getting good usage from Cachegrind.

<comment><USER13> I'm not sure that's a strong argument for your case. Similarly to the way that Usain Bolt could probably beat me in the 100m even if he was in a wheelchair with flat tires, I'm sure Richard Hipp could probably do a decent job of optimizing SQLite with just a stub of pencil and a scrap of napkin. I just think he'd be more effective with better tools.

But you inspired me to test out some real numbers, which I posted in a new thread: [https://news.ycombinator.com/item?id=8426302](https://news.ycombinator.com/item?id=8426302)

</comment></comment></comment></comment></comment></comment><comment><USER22> How long will we have to wait to get this version usable for our CoreData apps in ios ? Is it possible to include it manually into our app or are we dependent on ios releases ?

<comment><USER23> If you want to use it with CoreData, you might have to wait. But you can always use sqlite directly and link with the latest library. It might well be possible to specify a different library version you can use with CoreData, but I've never tried this.

</comment></comment><comment><USER24> By "50% faster" what they actually mean is this new version requires 50% less CPU cycles than the former for doing the same task. In real world I think you will not get your program to run "50% faster": disk io is often the bottleneck in the database area.

<comment><USER6> SQLIte mostly runs on slow CPU devices with relative fast SSDs. Reads are parallel, but writes are single threaded, so there is no advantage from multi core ARM CPUs.

<comment><USER25> Sqlite mostly runs everywhere, especially on normal desktop PCs, including all those old ones with slow HDDs and relatively powerful CPUs.

But I don't get the ARM CPU remark, did parent change his comment?

</comment></comment></comment><comment><USER26> I really like SQLite and posts like this always makes me wonder: Could I run a webshop on SQLite, if I did it right and how many users could I potentially support?

Of cause it's not that hard to deploy a MySQL or Postgresql server, but it's still pretty hard to beat SQLite in ease of deployment and management. Running a simply webshop could be a easy as having a database for inventory and one for orders. Even very large stores aren't going to have thousands or even hundreds of orders a minute, so I don't think that writes would be an issue and reads certainly isn't an issue.

<comment><USER27> You certainly could, but it's not really built with concurrent reading and writing. [http://www.sqlite.org/faq.html#q5](http://www.sqlite.org/faq.html#q5)

<comment><USER28> That FAQ entry is out-of-date as it doesn't mention the write-ahead logging support.

[https://www.sqlite.org/wal.html](https://www.sqlite.org/wal.html)

<comment><USER27> Neat, I didn't realize that had landed. Thanks for the link.

</comment></comment></comment></comment><comment><USER29> Is it possible to compile this version of sqlite for ARM and replace it on an Android device to improve performance? Android mainly uses SQLite for internal stuff and allmost all Apps use it...

As stated here: [http://forum.xda-developers.com/android/software- hacking/sql...](http://forum.xda-developers.com/android/software- hacking/sqlite-50-faster-performance-boost-t2898180)

<comment><USER30> Most definitely. Download AOSP, overwrite ./external/sqlite/dist with the new version, and build libsqlite.so and deploy on a rooted device (note that Chromium, and the Chrome browser, use their own copy of sqlite).

Having said that, while Android uses sqlite copiously, I doubt this would make a perceptible difference in usage, performance or battery life. It is a 50% improvement by itself, but I doubt sqlite3 usage is even measurable compared to everything else going on in the system.

<comment><USER29> Yeah, thanks. That was the thing I was looking for.

About your doubts, I'd like to quote the sqlite improvement comment: "Each of these optimizations is unmeasurable on a real-world system (we have to use cachegrind to get repeatable run-times) but if you do enough of them, they add up."

Therefore, I will build this library for my ROM, even though it maybe only gives a 0.02% performance boost, but constantly improving my ROM with those micro-optimizations will probably lead to a 50% boost one day :P

</comment></comment></comment><comment><USER31> Anyone using newer versions of SQLite out of homebrew on OSX? If so are you having any trouble with conflicts of any sort with the system version? OSX heavily relies on SQLite in many places and I'm curious if it causes trouble to try and use both (and have it in your path).

</comment><comment><USER19> I'm on 3.8.6. So will have most of the benefits. The temptation with sqlite is that it is so well maintained to only do cursory testing with it, so might drop this in.

Interestingly:

> A full 10% of the performance gain has come since the previous release. > >There have been a lot of changes.

</comment><comment><USER32> This is lovely. I believe the most important thing that we learn about this is how to measure your effort correctly. If you do micro optimization, but you don't track it, people will say that " DON'T PREMATURE OPTIMIZATION "

</comment><comment><USER33> can we expect this version to be included in android L ??

<comment><USER34> Just ship it in your own binary.

</comment></comment><comment><USER35> very good news for android developer

</comment><comment><USER36> > Small. Fast. Reliable. Choose any three.

Nice tag line.

<comment><USER37> Cute, but inaccurate. Small? Not quite.

[http://symas.com/mdb/microbench/](http://symas.com/mdb/microbench/)

Fast? No again.

[http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html](http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html)

Note that LevelDB itself isn't exactly the speed king here. It's just the clearest example. To its credit, SQLite _does_ fare well when it comes to reliability.

[https://www.usenix.org/system/files/conference/osdi14/osdi14...](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper- pillai.pdf)

So ... choose one?

(Downvotes from those who despise facts in ... oops, already happened.)

<comment><USER38> Within the space of sql databases they get all three.

Writing to /dev/null beats leveldb anyday, only problem is I can't get the data back out. A leveldb to sqlite comparison doesn't is unfair to both leveldb and sqlite.

<comment><USER37> Ah, "no true Scotsman" rears its head again. Why limit the space to SQL databases? Many applications that need an embeddable database don't specifically need SQL. You're right that comparing LevelDB to SQLite isn't quite fair to either, but _they invited the comparison_ with a slogan that makes comparative claims. It's not unfair to provide citations relevant to those claims, which you don't seem to have done BTW.

<comment><USER39> FWIW: I had never heard of MDB, and saw it performed really well on the benchmark, so I decided to look it up. I found this page:

[http://symas.com/mdb/](http://symas.com/mdb/)

They don't even mention sqlite in the comparison section on that page because they compare themselves to other embeddable "key value stores" which sqlite isn't.

If you look closely you'll even see that there is even an MDB backend for sqlite.

It's not a "no true Scotsman" when the products are so clearly different.

<comment><USER37> Yes, if you compare it only to things that are _exactly like it_ then it will be in the top percentile. Also the bottom. That doesn't seem very useful. What other _embeddable SQL_ databases should it be compared to? If you don't think this comparison is the correct one, tell me which comparison is. I'll go get the numbers, and I bet I'll still find that SQLite is still neither small nor fast by comparison. Otherwise all this goalpost-moving makes it impossible for anyone to prove anything (not that the pro-SQLite contingent is even trying).

Balancing full SQL semantics with being small _enough_ , fast _enough_ , and commendably reliable is an impressive achievement. I respect the SQLite authors for that. If they had picked any three of "small, fast, reliable, SQL" I would never have objected. BUT. THEY. DIDN'T. They left "SQL" out, opening the comparison to others. For many people who need an embeddable database, a key/value or generically table-oriented (but not fully relational/SQL) database is quite sufficient. For them, a claim based on an unstated and irrelevant fourth criterion, against equally unnamed alternatives, is either useless or misleading. But that's apparently the only kind of comparison the herd will accept.

<comment><USER39> I was comparing it to other relational stores, not other embeddable stores.

It works both ways too though, it's very small fast and reliable compared to Chrome or Outlook, which both can store data...

</comment></comment></comment></comment></comment></comment></comment><comment><USER40> And this is one of the problems with "optimize later".

If you have a large project in Python, for example, far too often there isn't any one spot that has massive gains if you translate it to C. It's the death of a thousand cuts.

<comment><USER41> Python might not be the fastest player on the block... but the increase in coding productivity will more than pay for extra equiptment to handle and speed-shortcomings.

<comment><USER42> Except when it doesn't. I'm facing this problem with an embedded-ish project I'm working on where Python initially seemed like a good choice, but now we're regretting it for both performance and power usage reasons. If we'd gone with C or C++ initially we'd be having a much easier time optimizing.

Not that I'm implying Python is bad, but if your target use case doesn't allow for a desktop or server, think carefully about what tech you use.</comment></comment></comment>


Color CEO explains how/why they raised $41M  - <AUTHOR> http://www.youtube.com/watch?v=_WGdwY6h5JI&list=SL#

<comment><USER2> After being prepared to hate this company, it's actually hard not to love this guy. The idea is really interesting, too. "The implied social network will lead to better behavior" -- interesting thought.</comment>


Productivity tips for internet startups - <AUTHOR> http://www.launchbit.com/blog/4-productivity-tips-for-internet-startups/

<comment><USER2> I liked the original title better.

That aside, I really like, and adhere, to the sleeping tip. In college (taking design and development) there were always kids who would pull all-nighter and then sleep during class. I never really understood the point.

Just go to bed at a decent time and if necessary, wake up earlier.

</comment><comment><USER3> Pretty great article.I'm always so leery of outsourcing work, no one can do it as good as I can! Maybe I need to change that.</comment>


Is your city a startup ecosystem? - <AUTHOR> http://blog.press42.com/index.php/2012/11/20/is-your-city-a-startup-ecosystem/

<comment><USER2> test</comment>


Data Mining Reveals Factors That Lead People To Make Blunders in Chess (2016) - <AUTHOR> https://www.technologyreview.com/s/601774/data-mining-reveals-the-crucial-factors-that-determine-when-people-make-blunders/ <comment><USER2> > The bottom line is that the difficulty of the decision is the most important > factor in determining whether a player makes a mistake. In other words, > examining the complexity of the board position is a much better predictor of > whether a player is likely to blunder than his or her skill level or the > amount of time left in the game.

I wonder if this also works the other way around with _good moves_ instead of just avoiding blunders.

A parallel in business: we spend so much time thinking about hiring "A players" and "10x-ers". But could refocusing on better processes, environment, and goal-setting (reducing the complexity of the position) be just as effective for increasing performance?

<comment><USER3> On point. I think that also is applicable to your personal life btw. I had a short-lived existensial crisis a few days back, realizing that my life is getting too complex for me to properly manage it. Things were falling out of hand, I wasn't making enough progress, etc. I even had a suicidal thought, my first in a very long while. But then reflecting on it I realized that this complexity is driven by my mounting desires. To improve my balance and regain the sense of control over my life, I should examine these desires and get rid of secondary ones, which would make my life simpler and I would be less prone to error (blunder in the article's terminology). Instantly made me feel better :)

<comment><USER4> Our lives are frittered away by detail. Simplify, simplify, simplify. -hdt

</comment></comment></comment><comment><USER5> It's poorly written press coverage of [https://arxiv.org/abs/1606.04956](https://arxiv.org/abs/1606.04956)

tl;dr We find strong evidence that in our domain, features describing the inherent difficulty of an instance are significantly more powerful than features based on skill or time.

<comment><USER6> Does it mean the more complex a problem the more likely an error? (while skill and the time available to make a decision matter less)

<comment><USER7> More like if the problem doesn't fit our mental models we're likely to fall into a cognitive tunnel where we depend on a familiar process for an unfamiliar problem.

</comment></comment></comment><comment><USER8> 'Anderson and co have found evidence of an entirely counterintuitive phenomenon in which skill levels play the opposite role, so that skillful players are more likely to make an error than their lower-ranked counterparts.'

I would to see more how the study was done because i highly doubt this could be possible. It can be the case that in high ranked games the oponent will put you in harder spots, and errors will comes. But by definition, better players take better decisions Ev average.

An interesting study will be a list of leaks the population usually have.

For exemple in pkker before solvers, people with a very strong hand on flop, lets say straight draw+ flush draw, when the board turn bad for them, they are way more likely to call a bet on river, because they hand was so strong on flop. Solver will show is a bad call because you block possible bluffs opponent could have (flush and straight draws) so his value range is bigger.

<comment><USER9> The finding that better players make worse decisions is only under very specific conditions where the board complexity is high.

One theory could be that stronger players are going for a win which requires more risk while weaker players are happy to play a non-aggressive move looking for a draw.

Unfortunately the article didn't state if most blunders come from being ahead or behind in material.

<comment><USER10> Sometimes you need to go for the win because of external conditions (last round of a tournament, or your chess team is losing and the captain told you to press on). Last world championship qualifiers for example saw Kramnik tilting because he needed to go all in in order to still make a chance to qualify.

</comment></comment></comment><comment><USER11> 10 seconds for what kind of chess? For classical chess it has to be much higher; probably minutes. For bullet chess on LiChess we have less than 2 seconds per move including the network lag

Also important point to note is that most chess masters avoid reaching this complex position and that itself is an important skill. The best way to solve a problem is to avoid it from happening.

GMs like Garry Kasparov would deliberately go into complex positions to confuse their opponents.

<comment><USER12> re : intentionally creating complications : Yes indeed -- Petrosian was also [in]famous for doing this. It's also a great way to win on time in blitz / bullet.

Silman has spoken / written about the little-discussed but non-negligible role that chance / stochastic processes have in the outcome of chess games. In one of his books [I think the recent edition of 'Reassess You Chess'], he ballparks chance at something like 10% of the total factors influencing outcome.

</comment></comment><comment><USER13> Offtopic: is to me or the Technology Review writing quality has plunged for the worse? It is difficult to get through this article. What is the plot doing in the article without any labels?

<comment><USER14> I don't think I've ever seen a good article from there.

</comment></comment><comment><USER15> One thing I haven't seen mentioned yet on this page is the huge role of _overconfidence_ in chess. It serves as a devilish kind of levelling factor:

If I play someone much lower-rated than me, it's hard to care much about the game. I will play careless aggressive moves, make reckless sacrifices, thinking anything will be good enough.

They will be playing a much higher-rated person. Winning will mean a lot to them, they'll try hard, focus intently, make the most of resources etc.

Because of this, I've often had the impression that higher-rated people are easier to beat than lower!

Then when you get in a very easily won position, it's really hard to keep focus, you just think it's already won, are already celebrating in your mind. But in chess, one slip is often fatal, no matter how winning you are. Or there are stalemate tricks, and the win is thrown away. And the more you're winning, the more paths to victory to select from, and the less you care to think about it. You impatiently wonder why the other guy hasn't resigned already. Thus the saying "There's nothing harder to win than a won game".

When you are losing, you focus, use every resource... Survival mode kicks in.

</comment><comment><USER16> I enjoyed the paper but one thing they didn't seem to consider was the human- difficulty of the move e.g. sometimes long horizontal moves are harder to see (sorry, no reference, just anecdotal based on experience), so one could see later works that fold that in.

Also, they used endgames with IIRC < 7 pieces, so again, some derivative work might use middlegame positions and use engine analysis as an oracle, similar to Regan's work ([http://www.buffalo.edu/news/experts/ken-regan-faculty- expert...](http://www.buffalo.edu/news/experts/ken-regan-faculty-expert- chess.html)).

<comment><USER17> Ivanchuck (I think) said the hardest move to find is a knight retreat.

<comment><USER18> [https://www.reddit.com/r/chess/comments/8015h5/vasilly_ivanc...](https://www.reddit.com/r/chess/comments/8015h5/vasilly_ivanchuk_gives_a_passionate_description/)

"According to my experience in chess, the most difficult in chess? To see moves with knight back." @5:34 --Ivanchuk

</comment></comment></comment><comment><USER19> Setting aside the poor writing of this article, the reasoning seems to miss a key element.

The higher the skill, the higher are chances of blundering probably because: 1\. Higher complexity 2\. Very high likelihood of being punished for it. At lower elo, your opponent probably didn't realize the blunder either.

</comment><comment><USER9> tldr;

> Quick decisions are more likely to lead to a blunder, but after about 10 > seconds or so the likelihood of a blunder flattens out

> More difficult positions are more likely to lead to a blunder. And skill > levels have a big impact in reducing the likelihood of a blunder. In > general, better players make better decisions.

> The bottom line is that the difficulty of the decision is the most important > factor in determining whether a player makes a mistake. In other words, > examining the complexity of the board position is a much better predictor of > whether a player is likely to blunder than his or her skill level or the > amount of time left in the game.

</comment><comment><USER15> I looked at the paper.. They used only positions analyzable by 6 piece tablebases, which was a surprise, i.e. only positions with a maximum of 6 pieces, e.g. each side has king and 2 pawns. The game starts with 32 pieces, so that means eliminating all the complex board positions I imagined a study of chess complexity would involve! So they're not so much studying chess proper, as endgames, and very minimal ones at that.

They have an new definition of _blunder_ \- "a player has committed an error if their move worsens the mini-max value from their perspective. That is, the player had a forced win before making their move but now they don't; or the player had a forced draw before making their move but now they don't. ...we will refer to such an instance as a blunder." .. "Since we are interested in studying errors, we exclude all instances in which the player to move is in a theoretically losing position — where the opponent has a direct path to checkmate — because there are no blunders in losing positions (the minimax value of the position is already as bad as possible for the player to move)."

"Our data comes from two large databases of recorded chess games. The first is a corpus of approximately 200 million games from the Free Internet Chess Server (FICS), where amateurs play each other on-line. The second is a corpus of approximately 1 million games played in international tournaments by the strongest players in the world. ... we focus on this large subset of the FICS data consisting exclusively of games with 3 minutes allocated to each side."

I've played 10,000+ games on FICS, so I have some idea how it works. Players have a standard, blitz and lightning rating, all separate. 15 0 and slower counts as 'standard', anything faster down to 3 0 is blitz, anything faster than 3 0 is lightning. [0] 3 0 is a different world to 14 0, so treating those ratings as the same is already misleading - I often played 5 14 (i.e. 5 minutes on your clock to start, plus 14 seconds added on after each move), a bit faster than 5 15, which is counted for rating purposes as the same as 15 0. (It assumes a game goes for 40 moves, so 5 minutes + 15 seconds x 40 moves = 15 minutes.) So 5 14 is the slowest blitz speed. If I played someone rated the same as me whose rating mainly came from 3 0 games, they would be _much_ stronger than me at 5 14. (Usually, on FICS, the faster the time control, the higher the rating. And obviously the more time you have, the better moves you make.) The point being - someone's blitz rating didnt mean they got it playing at 3 0. It's not their 3 0 rating.

Also, FICS _doesn 't use Elo_. (It uses the Glicko rating system[1]) The paper's authors seem unaware of that, or maybe I missed the part where they explained how they converted between FICS and Elo. As far as I know, it's not possible. There's not a linear relationship, or any simple conversion formula. From memory, lower rated FICS players have higher Elo, higher rated FICS players (over 2200) have lower Elo etc. But chess websites have a variety of rating systems, none of them compatible, commensurable or equivalent.

Sadly, I kind of lost interest there. I meant to study the paper more thoroughly. But already that rating confusion seems a huge 'blunder'.

[0] [https://www.freechess.org/Help/HelpFiles/blitz.html](https://www.freechess.org/Help/HelpFiles/blitz.html) [https://www.freechess.org/Help/HelpFiles/standard.html](https://www.freechess.org/Help/HelpFiles/standard.html)

[1][https://en.wikipedia.org/wiki/Glicko_rating_system](https://en.wikipedia.org/wiki/Glicko_rating_system)</comment>


Block inappropriate words with Javascript Validation. - <AUTHOR> http://9lessons.blogspot.com/2009/08/inappropriate-words-validation-with.html

<comment><USER2> OK, but now write one that doesn't block words like "sextant" or "bombast." I remember using a BBS in the '80s that had a filter like this; it was really hard discussing the Kennedy ass-ass-ination.

Also, if it is only Javascript validation, a malicious user can just turn off Javascript to get around it, or maybe just tamper with the HTTP after the browser is done with it.

</comment><comment><USER3> Validation should be done both at the entry level as well as the server. Banning inappropriate words is not as easy as it sounds and should also include things like viagra etc.. for spammers. Most routines out there will not capture this:

a s s :)</comment>


Geoparsing, Batch geocoding and geocoding Europe - <AUTHOR> http://geocode.xyz <comment><AUTHOR> Geoparsing means there need not be a specific format for the data to be in, for geocoding to work.

For instance On maps.google Department of Food Safety and Zoonoses (FOS) World Health Organization Avenue Appia 20 Geneva Doen't work

On Geocode.XYZ it works: [http://geocode.xyz/Department%20of%20Food%20Safety%20and%20Z...](http://geocode.xyz/Department%20of%20Food%20Safety%20and%20Zoonoses%20\(FOS\)%20World%20Health%20Organization%20Avenue%20Appia%2020%20Geneva%20CH)

46.23384,6.13204 » Geneva, CH » 20 Avenue Appia, Geneva, CH » \+ \- Maps © Thunderforest, Data © OpenStreetMap contributors 46.23384,6.13204

<comment><USER2> It's a challenging project. Here are some problems I spotted in a few of the addresses I tried out.

It doesn't understand diacritics correctly.

With the address "Första Långgatan 10, Göteborg" it shows the name with messed up characters and capitalization, as in "Example: F�rsta L�nggatan 10, GöTeborg, SE".

The locations also appear to be off by a few tens of meters. "The Old School | Temple Road | Oxford OX4 2EP" gives an address which is across the street from where it should be (which is where Google says it is). "Avenyen, Göteborg" is also shifted. And "Storgatan 1, Stockholm" is in the middle of the street named "Storgatan", and not at the address, which is at the end of the stret.

Less acceptable, the Louvre, at "Musée du Louvre, 75001 Paris, France", is off by more than two blocks.

Oh, I tried "Aveyn, Göteborg", which is a typo. It pointed me to somewhere in the middle of a block, with no street named "Avenyn", or anything like that, close by. I don't know how I ended up there.

<comment><AUTHOR> Första Långgatan 10, Göteborg is mainly a utf8 encoding problem. (I'm working on it, it's challenge especially with Russian, Hebrew or Armenian). Should work with ascii values too: eg, [http://geocode.xyz/Forsta%20Langgatan%2010,%20Goteborg](http://geocode.xyz/Forsta%20Langgatan%2010,%20Goteborg) )

As to the typo, I have not implemented fuzzy search yet.

<comment><USER2> My apologies, I see the ambiguity now in what I wrote.

Regarding the typo, I expected to get a "not found" answer, as "not found" is a better answer than being pointed to somewhere meaningless. I did not expect typo correction, though that would certainly be nice for some cases.

I see now that the search gives the same location as "Göteborg", so may be the default locations of the city. If so, that's odd because it's not near the center of the city.

Nor is it the default address for "Göteborg", as "dalkegatan, Göteborg" goes to some place near Göteborgsvägen on the east side of town. For that matter, "dalkegatan Göteborg" (without the comma) goes to some place on the west side of town.

Given these variations, it's hard for me to trust when an answer is correct, other than by double-checking the results manually.

<comment><AUTHOR> In this case it is a city match: [http://geocode.xyz/Aveyn,%20G%C3%B6teborg?geoit=xml](http://geocode.xyz/Aveyn,%20G%C3%B6teborg?geoit=xml) <geodata> <latt>57.67100</latt> <longt>11.97980</longt> <standard> <addresst/> <prov>SE</prov> <city>GöTeborg</city> <postal/> <confidence>0.70</confidence> </standard> </geodata>

The geoparser also looks for city names, and by checking the response you can see that only the city name was matched.

Having said that, it is also true that we have less data coverage for Sweden (compared to some other countries that provide open address data). Our responses will be about as good as the data we have, and openstreetmap does not have 100% coverage in Sweden either.

<comment><USER2> How is it that "dalkegatan, Göteborg" and "dalkegatan Göteborg" give very different locations, neither of which is the city match location?

<comment><AUTHOR> Thanks for pointing this out. It was a bug which is now fixed.</comment></comment></comment></comment></comment></comment></comment>


What's in Store for 2010? A Few Predictions - <AUTHOR> http://redmonk.com/sogrady/2009/11/12/2010-predictions/

<comment><USER2> A lot of the comments are bashing predictions as fiction, linkbait, etc, but I like predictions. They are the only way to really measure the vision of an individual. Perhaps not the only way, but a good way. I takes understanding, imagination, realism. Science fiction writers are not only lauded by the imagination of their stories, but the plausibility.

I personally like making predictions. That's what our industry is all about. That's what science is all about. Predicting the future.

That said, I don't see anything particularly insightful in the predictions made in the linked article.

</comment><comment><USER3> > a.) might lend themselves well to cloud and cloud-like environments and b.) > are receiving disproportionately more attention relative to their erstwhile > competition are Clojure and Go.

This one is easy. Lisp is going absolutely nowhere beyond appealing to curious mathematically-minded undergrads and researchers. Pure functional ideas and Lisp have been supposed to make a big comeback for what, 20-30 years? The generation that learned CS through SICP in their first year of university have utterly rejected Lisp as a practical production language. Now the generation that is learning with Python and Java is going to think it's the great new thing?

Go is "going" nowhere soon, since:

1\. The performance isn't nearly good enough yet. It's going to need to be better than languages like Java and C# to even have a plausible niche.

2\. No exceptions and generics right now is an absolute show-stopper for essentially all potential users, purely and simply.

3\. For the spaces that C currently occupies, C works just fine.

<comment><USER4> You could be right on both counts. Obviously.

Still, I do think it's at least plausible that significant changes in environmental context might lead to different adoption and uptake patterns. Languages not excepted.

Witness the revitalization - relatively speaking - of Erlang as the market fetishized scale-out at the expense of scale up.

The specific criticisms, I think we mostly agree on. I might nitpick on the performance question or the idea that C works fine with respect to Go or the outright dismissal of Lisp dialects, but they're arguable at worst.

That said, I think the trend, and history, both point to the addition of new languages to the toolkit. These seemed like pretty reasonable guesses based on the current intelligence, but maybe your objections are - as you argue - simply insurmountable.

I'll be interested to see what happens next year.

<comment><USER4> Because I'm told it's not obvious, I am the author of the originally linked piece. Just to ensure full disclosure.

</comment></comment></comment><comment><USER4> I'm frankly surprised that it got linked to at all, actually.

</comment><comment><USER5> Prediction is just fiction without the artistry.

</comment><comment><USER6> my random prediction Crunch pad will come out but will soon be put out of biz by the APPLE TABLET

</comment><comment><USER7> The one thing I know about the future is that the predictions are wrong.

</comment><comment><USER8> Ahh the year is almost over. Here come the predictions for 2010 posts that are pure linkbait.</comment>


Show HN: Mystery box adventures every weekend - <AUTHOR> https://www.weekendadventure.fun/ <comment><AUTHOR> Hey all,

I've always been frustrated by the amount of hassle involved in researching and planning interesting things to do on weekends. So much so, that most weekends I end up staying inside.

It's also become harder to make new friends since graduating from university.

I figured other people might have the same experience, and this is the lowest- friction solution we could come up with: Just a 1-character response to an email and we take care of all the rest!

I'd really appreciate any feedback you have.

</comment><comment><USER2> I think it's a cool idea.

Here are a few thoughts:

I think it might work better in urban areas if you stuck to food/wine/microbrewery and art/museum/theater and city park type experiences. If you're going to do sports events, go as a spectator.

I think you limit your audience if you focus too much on the outdoorsy/physical stuff.

I think (at least initially) you should keep it to one adventure, as opposed to a choice of two and I think you might want to consider limiting it to twice a month until you have enough people attending to justify more than that.

I think you should try to offer a similar experience in each of your locations and a way for those who attended to share their experiences online so can learn about those other locations. A FB page could be used for that.

I think you should offer an option to pay per adventure.

<comment><AUTHOR> Thanks so much for the feedback!

I think, based on other feedback we received, at least a subset of our target audience (recent graduates who are too busy to plan stuff) wants a way to be able to stay healthy without the boredom that comes with going to the gym.

I think the near-term plan is to give users the option to pick their desired intensity level. My personal bias also leans towards doing more intense activities, and I'd like the service to be one that I'd use.

The choice of two adventures is to prevent the "what if I don't like that week's adventure" argument. Currently, if too few people sign up for an adventure, we pad the rest of the group with personal friends :)

Almost every person we talked to gave us the feedback that they want to see the experiences of others. Will be adding that shortly.

The option to pay for a single adventure is a good idea.

Thanks again!

</comment></comment><comment><USER3> Can you provide some examples of what kinds of things you organize?

<comment><AUTHOR> Our aim is to provide a variety of events.

They vary from high intensity: climbing, biking, trapeze, archery, hiking, various sports, etc.

To mild intensity: yoga, horseback riding, walks, camping, scavenger hunts, etc.

To low intensity: board/card games, pottery, art classes, concerts, galleries, museums, bird-watching, etc.

We hope to get feedback from the first batch of customers on how to improve these.

I'll be adding these examples to the site shortly.</comment></comment>


Sketchfab adds 3D sound to its its 3D model sharing platform - <AUTHOR> https://techcrunch.com/2017/07/15/sketchfab-adds-3d-sound-to-its-its-3d-model-sharing-platform <comment><USER2> Related fun fact: if you're using THREE.js, you might not know that basic positional audio is supported right in the library. It works with regular <audio> elements via WebAudio.

[https://threejs.org/docs/#api/audio/PositionalAudio](https://threejs.org/docs/#api/audio/PositionalAudio)

I figure if someone told me this it would have saved me a couple of hours.

<comment><USER3> FYI WebAudio[1] has a PannerNode[2] that does exactly this (I only mention in case you thought you might need to manually calculate positional audio without THREE.js)

[1] [https://developer.mozilla.org/en- US/docs/Web/API/Web_Audio_A...](https://developer.mozilla.org/en- US/docs/Web/API/Web_Audio_API)

[2] [https://developer.mozilla.org/en- US/docs/Web/API/PannerNode](https://developer.mozilla.org/en- US/docs/Web/API/PannerNode)

<comment><USER2> Today I learned! If you're working with a scene graph you still need to glue your matrices to WebAudio, but I'm consistently amazed at how far you can get without a library in modern browsers.

</comment></comment></comment><comment><USER4> Artists are doing amazing things on the Sketchfab platform and (as a browser implementer) I'm continually impressed by how hard their team is working to make their viewers work well in all the browsers! It's particularly hard with the spec only just now becoming compatible and getting tests working across them. There are still a lot of quirks to sort out (link navigation, rAF interactions, implementation-specific performance issues, etc.). They're doing great work!

</comment><comment><USER5> Sound definitely adds a lot to the viewing experience. Take this awesome biomechanical whale for instance that someone showed me the other day:

> > [https://sketchfab.com/models/daaaf76c7d9e45618b5da7bf8f08e03...](https://sketchfab.com/models/daaaf76c7d9e45618b5da7bf8f08e034)</comment>


Ask HN: Contracting reputable freelancer for one off gig - <AUTHOR> So, i found an opensource project but it doesn't have all the things i need. It uses react, postgres, nodejs
I am looking for a freelancer who can implement the features. I've found some guys on upwork but they gave up after a few days.
Is there any site where i can hire real professionals who won't give up? <comment><USER2> You can search for Fiverr Pro freelancers, for example: [https://www.fiverr.com/gborunov?source=gig- cards](https://www.fiverr.com/gborunov?source=gig-cards)

</comment><comment><USER3> You can use upwork.com for general requested project and toptal.com for unique requested project</comment>