Atom text editor 1.4 released - <AUTHOR> https://github.com/atom/atom/releases/tag/v1.4.0 <comment><USER2> I continue to want to like Atom, and I try it out several times a year. Just updated, and see that it still has big performance problems on my system. I currently have BBEdit and SublimeText open for real work, and these are apps the have been running for days.

Here 100% would represent full usage of a CPU core:

BBEdit:

        15 files open (various types/sizes)
        76MB RAM
        < 2% CPU
        1 process
        25MB app disk space

SublimeText:

        10 files open (various types/sizes)
        169 RAM
        < 2% CPU
        1 process
        27MB app disk space

Atom

        1 small Markdown file open
        > 1.2GB RAM !!!!!!
        > 85% CPU
        7 processes
        205MB app disk space

Just sitting in the background I am seeing Atom's CPU and RAM usages fluctuate wildly. The numbers above are the LOWEST I observed.

I will spend a little time looking into why this is happening. I certainly could be related to some add-on package that I have installed, but I don't every recall seeing BBEdit or SublimeText behave so poorly with all of the customizations I have thrown at them.

Update: added disk space for base app. Even with a 500GB SSD, Atom would make me think about its value vs space usage.

Update 2: the system specs. 2011 MacBook Pro 13-inch, 2.3GHz i5, 500GB SSD, 16GB RAM, OS X 10.11.2

Update 3: performed a clean reinstall of Atom, removed all third party packages, removed preference files, disabled Markdown preview. Atom is still using over 80% cpu and greater than 1GB RAM with one short markdown file open

<comment><USER2> To summarize my above findings.

For me the biggest problem here is the high CPU usage when the app should be _idling_. That KILLS battery life. I would not accept this behavior from high end video software, I certainly will not accept it from a text editor.

Following that the high RAM use can lead to swapping which will impact performance and battery live. As I said above after a clean install, and with just one short markdown file open, Atom is now using over 2GB RAM and Climbing. Fortunately, I have 16GB RAM on this system, otherwise I would already be swapping.

BBEdit and SublimeText never tax the CPU unless I am working with extremely large files, or doing search & replace on a huge number of files. I often will have 50 to 100 files open for days in these editors.

I like the evolving ecosystem of Atom, it now looks to be about on par with Sublime, and I can see that it will soon over take Sublime on that front. But it really need to get the resource utilization under control.

To be fair to Atom, I also tried installing it on a Mac Mini with an almost clean Mac OS X 10.11 installation. I had similar problems.

<comment><USER3> > BBEdit and SublimeText never tax the CPU unless I am working with extremely > large files […] it really need to get the resource utilization under control

Neither does Atom for most users. It's not just n problem of «resource utilization», you're clearly facing a bug … Maybe you should consider submitting an issue ([https://github.com/atom/atom/issues](https://github.com/atom/atom/issues))

<comment><USER2> I see a number of current CPU related issues for Atom. I will look into using the performance diagnostic tools and see if I can contribute any new insights.

Again, I am see the issues on admittedly rather heavily customized workhorse MacBook (lots of Unix level tweets), AND on my stock Mac Mini. So I don't think this is a case isolated to just me.

<comment><USER2> As noted in this thread, I solved my problem, and filed an issue: [https://github.com/atom/atom/issues/10421](https://github.com/atom/atom/issues/10421)

</comment></comment></comment></comment></comment><comment><USER4> The decision to base Atom on web technologies remains controversial and it's still not as snappy as Sublime, for example, for similar tasks. But I think that decision has been vindicated by the speed of development it's made possible. Since the bar to entry for contributors is so low the ecosystem that's developed around Atom is second to none. You can find a package for just about anything and the UI polish of a lot of these packages is much higher than I've seen for any other editor, thanks to the ability to leverage modern HTML, CSS & JS.

Atom may never be as fast as a truly native client but it's perfectly usable now and is increasingly leaving the competition in the dust when it comes to features.

<comment><USER5> For all the extensibility and speed of development, the Atom ecosystem is still missing features I can find in any other major editor. Off the top of my head, there's no way to automatically (hard) word-wrap text. The vim plugins only implement the basics, and they all seem to be missing the command bar entirely.

<comment><USER6> Here's a nice plugin to add the vim command bar: [https://atom.io/packages/atom-vim-colon-command-on- command-p...](https://atom.io/packages/atom-vim-colon-command-on-command- pallete)

<comment><USER5> Sadly that only supports four simple commands.

</comment></comment></comment></comment><comment><USER7> The development of Atom shows a common anti pattern of software development: choosing the technology first. The goal was to develop an editor with web technologies, and not to develop the best editor in the world. The result is as sad as foreseeable: a buggy and slow memory hog named Atom. When I tried it last time, a couple of days ago, it was impossible to enter an @ sign into the text area.

<comment><USER8> > not to develop the best editor in the world.

The problem is that "best" is subjective. I think one could more accurately say that they're trying to develop an editor that is easily extensible with technologies familiar to many developers.

i.e. your perception of what they want Atom to be and the reality of what they want Atom to be are probably not the same.

My take? Revisit this in five years. I bet it'll be one of the richest ecosystems in software development. As one who has written Atom plugins (live unit testing w/ real-time feedback), I've never encountered such a developer experience until Atom.

</comment></comment><comment><USER9> Please note: [https://github.com/atom/atom/issues/4966](https://github.com/atom/atom/issues/4966) remains open and packages `metrics` and `exception-reporting` are still enabled by default.

* metrics: sends personal data to Google Analytics

* exception-reporting: sends personal data to bugsnag.com

<comment><USER10> The good part about using an open source editor is that you can disable (and know about them) these packages if you don't want to be tracked.

<comment><USER11> Yes, but you first have to know about it.

</comment></comment></comment><comment><USER12> I'm really enjoying the innovation that is happening in text editing. I prefer Visual Studio Code for light weight editing. Both Atom and VS Code are built on the Electron [1] engine but "VSCode uses Monaco for the user interface, not Atom. It is the web editor which Microsoft developed for Visual Studio Online. Electron is just the common core between the two applications. You can think of it like different programs using the same .NET framework, or two games built on top of the same engine." [2]

Both are fine editors and I use both [Ubuntu versions] in different contexts. It's fantastic that both are open source. That's were Sublime Text 3 loses me, because it is proprietary. I agree with some of the concerns raised in this thread, but realize some see closed source with one benevolent dictator for life as a benefit [3]. Other editors I'm keeping and eye on are Adobe Brackets and Facebook's Nuclide. LightTable is interesting. I have given vim and emacs a spin, but I am not a keyboard jockey. Aint we got fun!

[1] [https://discuss.atom.io/t/visual-studio-code-and- atom/16479/...](https://discuss.atom.io/t/visual-studio-code-and- atom/16479/12) [2] [https://en.wikipedia.org/wiki/Electron_(software_framework)](https://en.wikipedia.org/wiki/Electron_\(software_framework\)) [3] [https://forum.sublimetext.com/t/sublimes-future-and-open- sou...](https://forum.sublimetext.com/t/sublimes-future-and-open-source/7424)

<comment><USER13> Brackets was quite nice for what it can do out of the box, but I found the community support very poor, so there's really not much _more_ than that it can do.

LightTable was a clever idea that never quite fit my workflow, and which as far as I can tell is basically dead. The creators have abandoned it, and the surviving maintainer support is of the "pull requests welcome" variety. There are glaring bugs in basic features like inline eval, and maintainer response was a combination of "eh, we're cutting that feature anyway" and "fix it yourself."

Sure as hell doesn't give me any hope that this "Eve" thing their hyping is anything but smoke and mirrors.

<comment><USER14> I'm still using and prefer Brackets. Starting using it because it's performance was better than any other prior to VS Code.

However, Brackets new instant search is awesome and no other editor has it. For large projects I find it a great feature to be able to search all files in realtime as you type.

<comment><USER15> Unless I'm totally missing the point of the instant search feature, this is definitely something you can do in Emacs[1], and something I believe I've seen co-workers do in Sublime.

[1] [https://github.com/syohex/emacs-helm-ag](https://github.com/syohex/emacs- helm-ag)

<comment><USER14> At a cursory glance, I'm not sure it is the same. Brackets keeps some type of index. So there is zero delay whenever you perform a project wide search.

Looks like Emacs helm uses Silver Searcher which is real time, but impressively fast.

</comment></comment></comment></comment></comment><comment><USER16> Haven't tried it since launch, and while it's still insanely HUGE for a "text editor", I find it much faster. It was unusable back then, now it seems to work. Startup time used to be longer than the OS itself (well, not really, but you get the point), now it's somewhat reasonable.

Do you use "in production"? Is it OK?

By the way, Find seems to be only working on three or more characters? How do I find two-letter substring? :)

edit: Regexp find is a bit slow overall (one 5-line file). Like it is waiting for a while before commiting to updating the UI. And that's on an i7 desktop, can't imagine it working on my Celeron laptop (but will give it a try later).

<comment><USER17> The install size doesn't bother me personally (although my colleague's with newer Macbook Pro and smaller SSD may have an other opinion) but the speed of the editor is important for me.

It is fine for smaller simple files but I have had occasions that with some text files it can be really slow to a point that it is unworkable.

Last I was working on a HTML5 game and I copied the base64 encoded version of a font into my file that handled the assets and boy was that a bad idea. The preload.js was just 49KB but one big base64 encoded line made atom choke... Textwrangler or sublime didn't crimp on the same file and opened it in an instant.

Granted my Macbook Pro is old but a i7 with 8Gb RAM should be able to deal with these kind of situations.

<comment><USER18> Yeah atom has always had issues with files over a few kb that are all on the same line.

If the file doesn't have any crazy long lines, i've had it handle 5mb+ files with no issues.

<comment><USER19> _" i've had it handle 5mb+ files with no issues."_

Wow! It's so exciting living in the 1990s!

<comment><USER18> Okay, if you want to be a dick about it.

I've had it handle 5GB+ files with no issues.

I've got a 6.3gb log file open in it right now.

<comment><USER19> Try doing a find/replace/syntax highlight/multiple cursor/anything productive. Actually don't, your computer will explode!

<comment><USER18> find/replace works fine, and it's an XML file so syntax highlighting is working great (although it does take a few hundred ms to actually apply the syntax highlighting after a lot of scrolling)

It did take about 2 seconds to load the file though, so you can make fun of that!

<comment><USER20> On a more constructive note, I've found that it does have significant problems with certain text structures, namely really long single lines.

Just the other day I had to debug a json serializer, so my q&d solution was to copy the text out of the eclipse debugger, paste it into Atom (as it was already open), then find my way to the section I was looking for. After significant delays getting to the spot I needed with 'find', any use of the left and right arrows to navigate the text further was accompanied by a ~5 second delay per character. The same operation in Notepad++ went smooth as butter.

<comment><USER18> I agree 100%

I'm not sure specifically what it is, but it seems longer lines just wreck the performance.

I've seen a few commits talking about fixing specific problems that have caused this in the past, leaving me to think that it's most likely either one or more of my plugins causing the issue, but i just haven't cared enough to look into it yet.

Just like you I keep Notepad++ around for my "quick" needs (open a file for less than a minute) and for big files that aren't "code-ish" (they have long lines), and then atom is used for everything else.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> I find the momentum which Atom has gathered to be impressive. It seems to be building up an ecosystem at quite a rapid speed.

Myself, I prefer Emacs, because I like it's base philosophy ( _everything_ is a text-buffer, _everything_ is hackable LISP) and the endless possibilities that provides in form of customization, extensions (and extension on extensions, and customization of those, etc etc).

I don't think Atom is quite there, or ever will be. It will be interesting to see if if has the staying power of Emacs, or if it will yet another TextMate, Sublime Text or whatever hip text-editor of the month there has been the last decade.

<comment><USER22> Yes, many nice features are missing.

I mean, you have to code your themes, therefore you have to know much stuff about the editor. In many IDEs you can just load a them and get a config UI that lets you fine-tune it.

<comment><USER23> You can style it using CSS. How much easier could it be?

<comment><USER21> I've heard that Atom doesn't really reuse CSS or (SASS/LESS sheet variables) in any meaningful way, meaning having consistent theming is hard, at least if you want to use extra modules and plugins.

He may be thinking about that?

</comment></comment></comment></comment><comment><USER24> Atom: Now one step closer to being emacs. Now all you need is a proper text buffer model, instead of the clumsy hack you've got now, and lisp as an extlang. Wait, you already have that because clojurescript. So now you just need better APIs, and a proper buffer model.

While I am a diehard emacs user, I considered using atom for light editing (short enough that I'd want proper highlighting/indent support, but not long enough to get emacs out), but the lack of uniform buffer treatment was a dealbreaker for me. If you can't switch between settings and terminal screens using the same mechanism you switch between text screens with, and it all of the above don't have the same underlying mechanism, than you're doing it wrong.

<comment><USER25> Do you know of any examples out there on how to write plugins using ClojureScript? I got into Emacs a few months ago, but have been considering switching over to Atom so I can write plugins in CLJS

<comment><USER24> [https://github.com/MarcoPolo/atom-paredit](https://github.com/MarcoPolo/atom- paredit)

I'd also recommend looking at the atom api docs, if you really want to switch.

<comment><USER25> Thank you, this looks like a great minimal yet complete plugin. I'll have to try and hack my own together this weekend.

<comment><USER24> You're welcome :-D.

I'd recommend sticking with emacs though. CLJS is a better lisp than elisp, but emacs is a better platform than atom.

</comment></comment></comment></comment></comment><comment><USER26> One thing I noticed about Atom is that it doesn't handle (or preserve) Byte- Order-Mark and that wrecked havok on one of Unity project I was editing before.

I liked the design of Atom, but for this reason, I guess it's not the time for me to give up Emacs yet

</comment><comment><USER27> Been using sublime, code and atom. Atom seems to freak out in unexpected ways and has the downside of spewing tose node.js errors.

Code and sublime just seem more solid. I don't need explosions in my editor when I'm focused.

</comment><comment><USER28> not calling anyone a liar, consider it an anecdote, but i've used Atom exclusively for over a year and it performs great.

<comment><USER22> Well, I'm coming from WebStorm and in contrast to that it works just great ;)

<comment><USER29> With the notable lack of "click through to definition"!

<comment><USER14> Brackets can do this. Ctrl/Cmd E will open definition inline.

</comment></comment></comment></comment><comment><USER30> If you like vim and spacemacs, give proton-mode a shot: [https://atom.io/packages/proton-mode](https://atom.io/packages/proton-mode)

</comment><comment><USER31> If you're thinking on taking the plunge from Sublime Text, here are some setup notes aimed at web developers: [http://www.developingandstuff.com/2015/04/setting-up-atom- fo...](http://www.developingandstuff.com/2015/04/setting-up-atom-for-rails- development.html)

<comment><USER32> Haven't tried Atom for about one and half years; at that time it was a performance trainwreck. Is it better?

<comment><USER33> My only frame of reference is visual studio... Atom is darn fast.

<comment><USER16> To start maybe?

</comment></comment></comment></comment><comment><USER34> > The Linux version does not currently automatically update so you will need > to repeat these steps to upgrade to future releases.

Bit of a shame, I wouldn't have known about this update if I hadn't seen it here. While I appreciate auto-updating isn't for everyone, an in-app notification would have been useful.

</comment><comment><USER35> The runtime architecture is a bit bloated IMHO.

</comment><comment><USER36> Still can't use it for reviewing a commit (git commit -va) kills my MBPr.

But glad things are moving forward. Still haven't decided between Atom and Sublime though..

<comment><USER37> Me too, but I think I will settle with Atom eventually. Price of the sublime itself is acceptable, but then also sftp plugin is another $30... Or is there some other worthwhile alternative?

<comment><USER38> Vim? Takes some setting up/learning but everything I ever wanted is 'out there'. Amazing plugins for git, linting, code completion, whatever you can think of some one has done/is doing. I used to use Textmate a lot, tried sublime, Atom etc. I'll never go back. (Sorry, but someone had to be 'that guy'.)

<comment><USER39> I've been a happy vim user for 10+ years. I took Atom for a spin last week and I absolutely love it. There's a vim mode package for Atom that implements some vim features (not all, obviously) here [https://github.com/atom/vim- mode](https://github.com/atom/vim-mode)

</comment></comment></comment></comment><comment><USER40> I try it again every few versions and would really like to try Atom for longer but it really bothers me that I can't really configure hot exit the way I want it to work.

It seems like I'm a minority in that I don't want Atom to open the whole folder in the tree view when I open a file. What annoys me even more is that I want hot exit completely disabled, so that when I open Atom, no previous files/tabs or folders are open. I could never get this to work. The previously opened folder will always be present in the tree view when I open Atom again and when I open a file it will always open the folder in the tree view.

Does someone know if there is a workaround to this?

</comment><comment><USER41> The features do not outweigh the performance loss in my experience.

</comment><comment><USER42> Still eats lots of RAM and hangs on large files.

<comment><USER43> I like it, but there was that joke doing the rounds that Atom was an acronym for 'A Terabyte Of Memory'.

<comment><USER44> More funny with the context that EMACS was known as "Eight Megs And Constantly Swapping".

</comment></comment></comment><comment><USER45> I've been using atom for a few weeks. On my low-power 12" MacBook, it is only slightly slower than sublime text. The only two tasks that are annoyingly slow are opening the editor or a new editor window.

The barely noticeable lack of snappiness during editing is more than compensated for by the quality of the editor and its plugin ecosystem. I've also started using a ligature font for programming, which sublime doesn't support.

</comment><comment><USER46> When I tried it last time, maybe a year ago, I was quickly turned off by how uncustomisable it was, despite the claims.

All I wanted was a non-blinking block cursor ...

Now I just found this [https://github.com/olmokramer/atom-block- cursor](https://github.com/olmokramer/atom-block-cursor)

But more than 200 commits to support non-blinking block cursors? I think Atom is not for me.

<comment><USER47> So many of those commits are "updated readme", "updated changelog" and stuff like that. Scrolling through the log it's not uncomming to find 5 or 6 "updated readme" commits right after each other in that repository.

It's not a good metric. Instead look at the 278 lines of code necessary to achieve it.

</comment></comment><comment><USER48> Disappointed to see so much FUD over here. I switched to atom from sublime 2 months back, mainly because its tooling for ES7(emmet, babel, eslint etc) just works. And the editor is much more accessible to me than sublime.

Btw, I develop on an 14'' i3 3rd gen, 8gig, no-ssd laptop, and I am able to do react-native dev, with the emulator running just fine

<comment><USER49> Are you sure its not just other peoples experience that doesn't match yours rather than FUD?

<comment><USER48> Are you sure its not FUD?

I setup the dev team of a startup(in Delhi, India) I was consulting with with Atom for react and node dev few months back. These guys were provided pretty shitty, and pretty old, 4 gig ram laptops. And they used to screw up things which could've easily been resolved with in-text-editor linting. I got them running atom with emmet, babel and ESlint, and they are still using it without any issues.

I have seen atom run reasonably well on _really_ _shitty_ machines. Its very hard for me to accept that all this complaining about its performance is not FUD.

<comment><USER50> I suspect those who are experiencing high cpu usage while idling are running into weird bugs. It's only been 6 months since they released 1.0.

It runs well enough on my laptop with a 4th gen i7, 8gb ram, ssd, running linux but cpu utilization is rather high. Scrolling inside a moderately sized file puts a 20-30% load on all 4 of my threads. If I enable on the fly linting, autocomplete features, minimap, and etc, it starts to add up.

I've used vim, emacs, and sublime text and I've found that with my uses cases (mainly python development), atom ranks at the bottom of efficiency and performance (but highest in productivity). I consider myself a proponent of atom and it's currently my favorite editor, especially with vim-mode but it has it's drawbacks. It's just not a fair fight; comparing the performance/efficiency of atom to editors written in c/c++ doesn't make sense to me.

</comment></comment></comment></comment><comment><USER51> I love Atom and I've been using it for a long time now, but have they fixed the horrible stability issues in Windows yet? I run it on 3 distinct Windows machines and they all experience the same issues (most notably, I cannot have two editors open without the most recent crashing).

</comment><comment><USER52> I love Atom, but it along with Chrome it eats all my memory with just a few open files/tabs! Why would an editor and a thin (!) client use gigabytes of RAM?! Remember when computers had just 48KB of RAM and did great things?

</comment><comment><USER53> Posting to say I love Atom and I am excited for its future. I've added Facebook's Nuclide package and it has some really great features. Would love to continue to see Atom as a full featured IDE

</comment><comment><USER54> Happy to see highlighted JSCAD [0]

Does anyone find JSCAD modeling inside Atom useful?

Tips/suggestions? I've been working on this project [1]

[0] [https://github.com/github/linguist/pull/2712](https://github.com/github/linguist/pull/2712)

[1] [https://atom.io/packages/atom-scad- preview](https://atom.io/packages/atom-scad-preview)

</comment><comment><USER55> Atom seems pretty nice these days, but why would a competent emacs user switch to atom?

Why would a spacemacs user switch to atom? Does it have any advantages?

<comment><USER56> As a front-end web developer that has used Emacs for several years, I often feel the pull to Atom. It draws very clear, direct inspiration from Emacs.

Spacemacs is an improvement, however I think it the biggest advantage it (and any other modern text editor) has over Emacs is the on-boarding process and general beginner friendliness.

Beyond that, Atom's modes for front-end web development are pretty top notch. We have js2-mode, and web-mode in Emacs, but the overall experience in Atom is often superior. For example, it's extremely easy to quickly install a color- picker, pull open a CSS file, and use a color wheel to make a hex value slightly darker. More, the color-picker UI feels tightly integrated into the editor.

On the surface, this feels trite, however the slow build-up of user friendly developer conveniences adds up.

The general command over the entire environment continues to draw me back to Emacs. Still, I continue to watch for when Atom can achieve this.

</comment></comment><comment><USER57> Does anyone know of a good Python completion plugin for Atom similar to what PyCharm has (i.e. can complete models stuff etc)?

<comment><USER58> I use this: [https://atom.io/packages/autocomplete- python](https://atom.io/packages/autocomplete-python)

</comment></comment><comment><USER59> Does anybody know if it's now possible to use Atom as a sane $GIT_EDITOR (or $SUDO_EDITOR...) such that a new tab is opened in an existing window and the invoking process (git, sudo,...) is notified when the tab is closed (similar to Sublime Text)?

<comment><USER60> [http://blog.atom.io/2014/03/13/git- integration.html](http://blog.atom.io/2014/03/13/git-integration.html)

<comment><USER59> I can't test it right now, but I am pretty sure I tried that before and it opened a new Atom window instead of opening a new tab.

<comment><USER60> Ah I see. You're right: It opens a new Window :(

</comment></comment></comment></comment><comment><USER61> Atom 1.3 was hang up when opening same file name. 1.4 seems to fix this issue. [https://github.com/atom/atom/issues/10060](https://github.com/atom/atom/issues/10060)

</comment><comment><USER62> I submitted a PR for homebrew's cask once I read about the update.

[https://github.com/caskroom/homebrew- cask/pull/17022](https://github.com/caskroom/homebrew-cask/pull/17022)

<comment><USER62> Tight. They merged it in already. Homebrew is ready to go!

<comment><USER63> Just curious, what's the advantage to installing this with homebrew? Does apm still work from the terminal?

<comment><USER62> Just makes it super easy. No difference other than how you do it. And yep, apm works

</comment></comment></comment></comment><comment><USER64> I've stopped using atom due to horrendous latency issues. Pretty disappointed. Would really love a modern open source text editor, but noticeable lag between pressing a key and having it appear on the screen is a no go.

</comment><comment><USER65> With all the people complaining about Atom performance, I cannot confirm this. Switched from Sublime Text about two years ago and never had speed/snappiness issues. If anything Atom does quite well with gigantic files.

<comment><USER66> On a 500 line file when I start adding multiple cursors, after the 5th or so It starts taking multiple seconds just to select the next. This was in December, so a recent version. I can't work like this.

</comment></comment><comment><USER67> Best thing about some of the modern tools like Atom is that, by the time you know about the release, you are already updated!

</comment><comment><USER68> But it's still so slow.</comment>


Træfɪk-modern HTTP reverse proxy and load balancer made to deploy microservices - <AUTHOR> https://github.com/containous/traefik/blob/master/README.md <comment><USER2> What's "modern" in this?

Also, "no dependency hell, single binary made with go", but 54 different modules imported. ROTFL.</comment>


MIT's holiday greeting - <AUTHOR> http://alum.mit.edu/holiday2011/

<comment><USER2> Serious question: Is anything MIT-related worth voting up? Because I don't think the content of this video was particularly worth submitting to HN.

</comment><comment><AUTHOR> just for kicks I decoded the QR code in the middle of the video, hoping for an easter(or rather xmas) egg, but alas only the rather boring <http://web.mit.edu/newsoffice/>

<comment><USER3> How fast can average QR decoders using a phone camera work? Would they decode the QR code from the playing video, or would you have to pause it? Just curious.

<comment><USER4> I had to screencap the video (their play button covers the video when paused). The QRReader app on iPhone wasn't fast enough to scan it during the video.</comment></comment></comment>


Reasoning on Web Data: Algorithms and Performance [pdf] - <AUTHOR> https://hal.inria.fr/hal-01148500/file/tutorial.pdf <comment><AUTHOR> I found the submitted paper on the home page of one of its authors, Ioana Manolescu. [1] According to her page, the paper is designed to be more of a tutorial. Having a rough understanding of the Resource Description Framework (RDF) [2], RDF Schema (RDFS) [3], and RDF Query Language (RDFQL) [4] certainly helps, but is not required. If you think about the paper in terms of how a major search engine like Google works, and you remember the occasional "index updates" they used to do a few times a year, you'll see the "saturation" or "closure" approach mentioned in the paper. i.e.

> _" compile the knowledge into data, as follows. Infer all possible facts > derived from the base data and by the semantic constraints prior to querying > the database; add these facts to the database; then, process queries on the > (enriched) database ignoring the constraints which have lead to it."_

Since Google no longer does occasional "index updates" and now updates continuously, it seems they may have moved to something similar to the alternate "reformulation" approach mentioned in the paper:

> _" The alternative technique is called reformulation. Here, the database is > left unchanged, while queries are modified (reformulated) to take into > account all the known semantic constraints."_

[1] [http://pages.saclay.inria.fr/ioana.manolescu/](http://pages.saclay.inria.fr/ioana.manolescu/)

[2] [https://en.wikipedia.org/wiki/Resource_Description_Framework](https://en.wikipedia.org/wiki/Resource_Description_Framework)

[3] [https://en.wikipedia.org/wiki/RDFS](https://en.wikipedia.org/wiki/RDFS)

[4] [https://en.wikipedia.org/wiki/RDF_query_language](https://en.wikipedia.org/wiki/RDF_query_language)</comment>


The Limits to Growth (1972) [pdf] - <AUTHOR> http://www.donellameadows.org/wp-content/userfiles/Limits-to-Growth-digital-scan-version.pdf <comment><USER2> I remember the first time reading this book, or rather the 30 year update -- [https://www.amazon.com/Limits-Growth-Donella-H- Meadows/dp/19...](https://www.amazon.com/Limits-Growth-Donella-H- Meadows/dp/193149858X) \-- thinking, " _this_ is the approach to take to understand how the economy, ecology, pollution, and so on interact."

Everything else was just looking at elements. Technology is important, for example, but exists within a system. They looked at the system. They had to simplify and assume a lot, which the media didn't understand (probably benign ignorance) and critics blew out of proportion (probably maliciously), but I found their approach the most meaningful.

Sadly, I know many people who care about the environment but don't understand the (relatively simple) math in their approach, and many people who understand the math but don't care about the environment, but almost no one who cares and understands. So in about a decade since reading it, I haven't found anyone I can talk to about it meaningfully.

A great companion by one of the authors is Thinking in Systems by Donella Meadows -- [https://www.amazon.com/Thinking-Systems-Donella-H- Meadows/dp...](https://www.amazon.com/Thinking-Systems-Donella-H- Meadows/dp/1603580557).

Both changed my views more than almost any other books.

<comment><USER3> With all respect to the authors, the thinking at that point in time and the impact it made on the world and you: Much of their predictions haven't stood the test of time.

-No exhaustible has been depleted yet and the price mechanism combined with technological progress seems pretty able to set incentives straight. Look at what the oil price bubble of the '00s did for electric cars and batteries in the now. -The world has seen massively more population growth than they modelled, but not the accompanying food shortages they predicted. I'd argue the shortages of the '80s and '90s are over, although local situations obviously have tragic effects (Syria, Venezuela). -Material goods production (plastics) is at an all time high. -Many more humans alive at lower poverty levels than ever before and ever imagined by the Club of Rome

They got the depletion of fish stocks right though. Although arguably, that was the simplest prediction since the 'tragedy of the commons' is so profound in international fishery.

The gloom of the club of Rome just didn't work out. I guess outside of way we are treating our environment, humanity has a lot to be proud of. (I feel the environment is going bonkers and we are a plastic-addicted throw away society.)

<comment><USER2> You've mischaracterized their work.

They didn't predict what you said they did. They showed a range of possible outcomes based on assumptions, among many. You seem to have picked one outcome as their only one and called it their prediction.

They created a model based on a systems approach whose output depended on assumptions on physical properties of the planet and future human choices. Given the large uncertainties, they ran the model under many sets of assumptions and presented the outcomes.

The point of the book is to illustrate and promote a systems perspective, not just a linear, event-based approach, which you seem to prefer.

<comment><USER3> Say I agree (at least I get your point ;) - I'm curious how does one test (in the empirical science sense) a systems perspective with regards to predictions / scenario's? Because at least in popular opinion the Club of Rome is known for a pretty grim worldview and political advocacy.

Perhaps I'm too much schooled in traditional economics but Robert Nobel wasn't too kind [1] and even in my environmental economics classes we spent quite some time deconstructing the Club of Rome.

[1] [https://en.wikipedia.org/wiki/Club_of_Rome#Critics](https://en.wikipedia.org/wiki/Club_of_Rome#Critics)

<comment><USER4> At some point, there are limits to our ability to empirically test large-scale predictions because we have no ability to run history more than once.

This certainly applies to macroeconomics -- how does one empirically 'test' any prediction from a macroeconomic model based on real-life inputs, other than to see what happens? But this shouldn't preclude us from asking the really difficult questions that we care about, rather than restricting ourselves to the realm of physics, where we can get 5 sigmas in classical statistical tests.

With regards to Robert Nobel's criticism, the whole point of the Club of Rome's approach is to model a simplified, 'low resolution' view of the world economy in order to embed it in a larger network of information, goods, and energy flows constrained by the environment -- because _the boundary conditions matter a lot_ to how the macroeconomy functions.

Sometimes you can get away with a much simpler modeling approach if you actually get all the inputs right. The biggest criticism the Club of Rome has on most macroeconomic models is that it doesn't model the environmental inputs in sufficient detail, and these things really matter a lot in the long run.

It is _not_ simply sufficient to take current 'wage levels' or 'capital stock' or even 'level of technology' as a direct input to your DSGE RBC model -- our macroeconomies perform a quasi-ecological function of converting flows of matter and energy, largely taken from Nature and the environment, and converting them further for human use. Understanding the Nature-human interface, then explicitly modeling the stock quantities of the inputs into the macroeconomy _must_ on some level be important, because they are key to how our entire technological society functions.

I you're inclined to learn more about their modeling approach at a high level, I highly recommend _Thinking in Systems_ [1]. Read it a week ago and it blew my mind, as its intuitions match up with a lot of patterns that we see over and over again in dealing with any complex network, but presented in a systematic and convincing manner.

[1] [http://wtf.tw/ref/meadows.pdf](http://wtf.tw/ref/meadows.pdf)

</comment></comment></comment></comment></comment><comment><USER5> Here's a paper from 2014 that compares historical data from the 40 years since the limits to growth study was released:

[http://sustainable.unimelb.edu.au/sites/default/files/docs/M...](http://sustainable.unimelb.edu.au/sites/default/files/docs/MSSI- ResearchPaper-4_Turner_2014.pdf)

</comment><comment> ultra-jeremyx The point of system dynamics is not predicting the numbers--it's understanding the system. If you look at historical data over time and see values increasing, decreasing, and remaining steady, it would be nicer to have an endogenous understanding of what causes those changes instead of extrapolating based on the historical data. When you understand _how_ the system works, then you can reason about where the leverage points are.

I always loved that the biggest leverage point in any system is to change the system itself!

</comment><comment><USER6> From a purely logic point of view, I have some problems accepting this:

> "When the facts change, I change my mind," I included because of its implied > inverse: "When the facts don't change, I don't change my mind."

It is my understanding that "A implies B" does not imply "not A implies not B".

Maybe I am just hairsplitting, but when I read/hear something I see these kind of sentences as bad indicators.

<comment><USER7> The trouble with applying formal logic to natural language, is in the translation.

You are interpreting "when" as indicating an implication, it could also be an 'if and only if'.

<comment><USER6> I know what you mean. It took me some effort to accept that, in natural language, an "exponential growth" was not really meant to be exponential. One has to live with it. :)

<comment><USER7> It sounds like you displeasure with incorrect use of 'exponential growth' may declining exponentially with your exposure to it.

</comment></comment></comment></comment><comment><USER8> I'm still looking for a video or audio of that conference.

<comment><USER9> Same. Please let me know if you're able to find anything!

<comment><USER8> Just got a partial answer from a team member of donella meadows archives saying I'll have to wait a week for the person who knows best about this to come back.

In the mean time, here are the reading suggestions I got:

\- [http://donellameadows.org/donella-meadows-legacy/danas- writi...](http://donellameadows.org/donella-meadows-legacy/danas- writing/articles-donella-meadows/)

\- [http://donellameadows.org/archives/the-club-of-rome-and- sust...](http://donellameadows.org/archives/the-club-of-rome-and-sustainable- development/)

\- [http://donellameadows.org/archives/forty-years-later-the- rec...](http://donellameadows.org/archives/forty-years-later-the-reception-of- the-limits-to-growth-in-italy-1971-1974/)

I'll ping you when the news come</comment></comment></comment>


Deoptimizing a C++ program - <AUTHOR> http://stackoverflow.com/q/37361145/563941 <comment><USER2> Inevitably the stackoverflow mods have moved into shut this down. They should be honest and introduce a "too interesting" classification, complete with a wordy, humorless justification phrase.

<comment><USER3> I should note that this wasn't closed by moderators, but instead just normal StackOverflow users. In fact, one of the mods commented on the question before it was closed (and did not close the question).

<comment><USER4> I don't follow what you mean, normal users are moderators, just they're just users with sufficient karma.

<comment><USER5> Stackoverflow does have a separate concept of "Moderator", which is a voted-on position and denoted by a blue diamond next to their name.

</comment></comment></comment></comment><comment><USER6> For caching issues, you could replace the loops by more functional code: write one loop to generate an array of values, and sum them to _payoff_sum_ in a separate loop.

      for (int i=0; i<num_sims; i++) {
        gauss_bm[i] = gaussian_box_muller();
      }
      for (int i=0; i<num_sims; i++) {
        S_cur[i] = S_adjust * exp(sqrt(v*v*T)*gauss_bm[i]);
      }
      for (int i=0; i<num_sims; i++) {
        payoff[i] += std::max(S_cur[i] - K, 0.0);
      }
      for (int i=0; i<num_sims; i++) {
        payoff_sum += payoff[i];
      }

To make this i7-specific, you have to make sure the arrays are mapped to just the right cache lines, so that writing to an array evicts the dat from the array being read from from the cache (hm, maybe that requires combining some of those loops; what does the level 1 cache of these CPUs look like?)

Next step: DRY. Move the _for (int i=0; i <num_sims; i++)_ loop into a function taking a lambda for the loop body. That hides the ugliness of the above code. Here, it probably is fairly easy to accidentally do some extra copying of data.

Next: parallelism. You shouldn't have individual threads process contiguous sections of the arrays; do

       for (int i=threadNo; i<num_sims; i+= numThreads) {

instead. And of course, you can multi-thread by having the 4 for loops run in parallel. That, I think, would be the big gain, as it would introduce quite a few data dependencies between threads. It also is in the spirit of the question, I think.

It has nothing to do with the i7, but I would also work on worrying about the value of RAND_MAX. It can be as low as 32767. So, if you want to pick a double from the full range of values in the [-1,1] interval you have to make at least 4 calls to _rand()_. Making the distribution uniform from there will take quite a bit of work, too (there are as many double values between 1/4 and 1/2 as between 1/2 and 1, but you want to pick the former half as often as the latter)

</comment><comment><USER7> The context is missing, what's allowed and what's not.

If it's allowed to make own classes, the best thing it can be done is to introduce MyNumber abstract class, then inherit it to make MyDouble, then make NumberFactory and access every double through the virtual operators. Then let every temporary variable be created (with the new operator!) on every loop pass, of course using the factories. Define the operations in a way to need the new temporary MyNumbers too. That way the creations would be hidden in the classes, the calculation would still look relatively innocent.

And additionally, make classes so that they have to do runtime casts for every operation. Make the classes to have multiple inheritance, to make sure figuring out in the runtime which cast works as nontrivial as possible.

I know one commercial program where the programmers actually implemented all this, even more convoluted, and even believed that they "improved" the code with all these insanities.

For style points, use all possible boost classes that you can to achieve the above behavior.

<comment><USER8> But would that cause cache fails? Wouldn't that just increase the time needed to access data?

<comment><USER9> Seems like virtual functions leave you more vulnerable to instruction cache misses. Large enough blocks of code combined with bad branch prediction can leave you all sorts of bad performance here.

</comment></comment></comment><comment><USER10> I'm finding this more difficult than I expected, at least if I interpret the problem as finding "deoptimizations" that are specific to modern Intel processors. Finding ways to slow things down through changing the algorithm, increasing precision, or adding unnecessary work are easy, but I haven't yet found anything that would be a plausible optimization made by a beginning programmer that hurts performance more on modern Intel than would be expected.

In the answers, I think Peter covers most of the likely attacks. One that isn't mentioned is "choosing the wrong compiler". Not just compiler options, but compiler. While there isn't a "best" compiler for everything, for a given program often one will perform better than the others. Especially if one is targetting recent Intel CPUs, Intel's compiler can occasionally beat GCC and Clang by surprisingly large amounts. This appears to be one of those times.

Here are the times I see for for this example on a 3.4 GHz Skylake running 64-bit Linux when compiled with different modern compilers:

      icpc 16.03:  0.87 s
      clang++ 3.8: 1.76 s
      g++ 5.3.0:   1.73 s

All were compiled with "-march=native -g -Ofast", which I think causes all of the compilers to use the equivalent of "-ffast-math". At least, I'm pretty sure icpc uses it by default, and I didn't see any difference in performance when I added "-ffast-math" or "-funsafe-math-optimizations" to clang++ and g++. So in this case, I (provocatively) would suggest that using clang++ or g++ would be a subtle 2x deoptimization. There are definitely reasons to prefer them, but for this example that choice looks to come at a large cost in performance.

If you want to do your own comparison, Version 17 of the Intel compiler is currently available as a free beta for Windows, Mac, and Linux: [https://software.intel.com/en-us/articles/intel-parallel- stu...](https://software.intel.com/en-us/articles/intel-parallel-studio- xe-2017-beta). I don't know how they are defining the license duration, but when I downloaded yesterday it gave me a license valid until October 2016. Separately, I'd be interested to see what MSVC does with this on a comparable machine.

<comment><USER11> Without digging into it too much, I would bet that most of the improvement from ICC comes from their awesome implementations of transcendental functions used in this program (exp, sqrt, etc.).

Still, the general point is good. last time I tried ICC on a heavily vectorized (with intrinsics) program, ICC was a 30% boost over clang or gcc.

<comment><USER10> This was my first guess, but as I dig a bit deeper I'm less sure. It looks like the difference is the implementation of log and exp, but both of them are linked to the same libm from glibc. The difference is that Intel is inlining an AVX2 optimized version that uses FMA, but Clang and GCC are using the compiled function from libm.

The Ubuntu packages I'm using are compiled appear to have been compiled to support AVX, but not AVX2, and hence no FMA. So while GCC and Clang are much slower, this is really an issue of having suboptimal system libraries. The speedup is real, but it's not clear that either GCC or Clang are actually to blame.

<comment><USER12> Not an expert on this, but I think the compiler has to inline the function for that to work well. No matter how good the system library implementation is, the function signature is defined one scalar float, which limits vectorization.

<comment><USER13> That's correct; I'm sure icc has vectorized versions of those functions to inline. Each monte-carlo iteration is independent, so the code vectorizes very easily if you have a vectorized RNG and vectorized exp() and log(). (vector sqrt() is available in hardware, so even gcc and clang will vectorize that, but not exp / log even with -ffast-math)

<comment><USER10> OK, I was able to test this more rigorously. Yes, the vast majority of the difference between Intel and GCC is just the better implementations of exp() and log(). These are not vector implementations that calculate multiple results simulataneously, just scalar implementations that make better use of the available instruction set.

When I can get g++ to use Intel's libimf instead of glibc's libm, it's only ~5% slower than icpc. I'd guess much of the remaining difference is function call overhead vs inlining. Clang still lags by another 10%, but I think that's probably because I haven't figured out quite the right incantation to get it to use libimf without also disabling some other useful optimization.

Here are the commands that I ended up with:

clang++ -fno-finite-math-only -march=native -Wall -Wextra -g -O3 option.cc -o option -Wl,-rpath=/opt/intel/compilers_and_libraries/linux/lib/intel64 -L/opt/intel/compilers_and_libraries/linux/lib/intel64 -limf -lintlc

g++ -fno-finite-math-only -march=native -Wall -Wextra -g -Ofast option.cc -o option -Wl,-rpath=/opt/intel/compilers_and_libraries/linux/lib/intel64 -L/opt/intel/compilers_and_libraries/linux/lib/intel64 -limf -lintlc

icpc -march=native -Wall -Wextra -g -Ofast option.cc -o option

The "-fno-finite-math-only" disables an otherwise good clang++ and g++ optimization that requires a __finite_exp() function that libimf does not have. clang++ seems to also need to switch from -Ofast to -O3 to make this stick.

What I don't know yet is whether recompiling glibc (or upgrading to the most recent glibc) will produce better performance out-of-the-box on recent Intel. My searches aren't turning up much information --- anyone know?

 _gcc and clang will vectorize that, but not exp / log_

I don't know how well it's currently working, but it looks like this might have changed recently: [https://sourceware.org/glibc/wiki/libmvec](https://sourceware.org/glibc/wiki/libmvec)

</comment></comment></comment></comment></comment></comment><comment><USER14> Despite the really long accepted answer, I don't see many suggestions that actually fulfill the assignment. Playing with compiler flags feels like cheating IMO.

Pre-generating a bunch of random numbers in a linked list would be amusingly evil, though.

<comment><USER15> Yeah. I'd interpret the spirit of the question as not making the program do more work, but just arrange things such that the operations it does end up being more expensive. Maybe the problem is with the sample program. It just doesn't feel like the right one for the task.

For example if it had some actual data structures and operations on those, rather than just arithmetic on a few stack-allocated scalar values, introducing bad cache behavior would be a lot plausible. Another problem is that most of the time is going to be spent in pretty opaque library functions (exp, log, rand) whose execution cost is hard to affect just by rearranging data or operations.

<comment><USER13> That's what I found when writing my answer on SO. The only justification I could come up with for using memory was to generate all the random numbers first, then use them later.

I did include several potential microarchitectural slowdowns, like causing a store-forwarding stall by modifying just the high byte of `double` with an XOR to flip the sign bit. That isn't much of a stretch, but will hurt a lot

Multi-threading with a shared atomic loop counter is also really good, and maybe worse than you'd naively expect (i.e. easy to justify re: diabolically incompetent).

</comment></comment></comment><comment><USER16> > _" put on hold as too broad by ..."_

and yet again StackOveflow do their best to kill a problem/discussion with a lot of interest and interesting comments.

...sigh...

<comment><USER17> and yet again someone doesn't understand that Stack Overflow is not a discussion board.

<comment><USER16> It may not be a discussion board but of course almost every question has some degree of discussion. For example, even a specific question about how to initialize a list of lists in Python has more than 1 answer and there will be discussion about the "best" method.

There is almost never only 1 correct answer, in which case a discussion is needed and the answer may require your personal opinion as there may not be a single "best" answer. It would depend on circumstances - more discussion required...

</comment></comment></comment><comment><USER18> cue last week's hbo silicon valley for gilfoyle's efforts to 'deoptimize' his pre-processor for their box @ 200MB/s...

also several days ago there was a joke floating on my fb from a taiwanese programmer about how he gets raises every year by removing multiple loops in his programs to show efficiency improvement.

<comment><USER19> Reminds me of this story [http://thedailywtf.com/articles/It_Must_Be_Broken](http://thedailywtf.com/articles/It_Must_Be_Broken)

</comment></comment><comment><USER20> Increase the constants at lines 18 and 19 a bit ...</comment>


Windows 10 – WiFi always connected even during sleep? - <AUTHOR> I use Firefox and frequently leave it open for days with multiple tabs. GMail is one of those tabs.
Always put the computer on hibernate each night. In the morning I wake up the computer, log in.
I kept Firefox open, as well as my GMail tab....So how is it possible that without taking any action, within the second it takes to login, GMail can already show my Inbox completely up-to-date?? No need to refresh.
This bugs me to no end. Even prior to login, or in sleep mode, the computer is remaining connected to WiFi and the Internet. Not only that, but any applications running are also thus connected, and updating? This doesn't strike me as a very secure scenario.
Already tried all the common tips - disable connected standby, power management for the wireless adapter, unchecked "connect automatically" for the wifi networks. Nothing works.
I previously had this approach work, 12 months ago or so - WiFi would remain off even after logging in, at which point I had to manually select the network I want to connect to. This was the case even if the computer went to sleep for 5min - manually connect to the network again after login.
I don't always want to be "connected".  Is this paranoia, or is it impossible to be "offline"?
(Windows 10, all up-to-date, running on Dell XPS 13 latest gen.) <comment><USER2> Well I find this to be true in Mac as well. And honestly I find it irritating. Though the WiFi part is still fine but the the Bluetooth bugs me a lot. If we have the Bluetooth switched on and put the laptop to sleep the Bluetooth is still on and tends to get connected to my Bluetooth earphones. You can find me constantly switching my earphones on and off hoping to get it connected to the right device. (Ofc I started keeping my Bluetooth on Mac off now a days)

</comment><comment><USER3> You can't trust your computer nowadays. You need to keep everything locked down. It's sad, and it's one of the reasons we're not going to get a singularity any time soon ...

</comment><comment><USER4> There is actually a new capability in good notebooks that behaves like that, kind of the same we have with cellphones, "always on" for notebooks, maybe googling a bit will get you how to turn it off</comment>


Hacking WiFi to inject cryptocurrency miner to HTML requests (CoffeeMiner) - <AUTHOR> http://arnaucode.com/blog/coffeeminer-hacking-wifi-cryptocurrency-miner.html <comment><USER2> I've thought about adding something like this to my guest wifi to mine some cryptocurrency - but quickly dismissed it as most guests would need to use a charger soon(ish) and thus using my electricity :P

<comment><USER3> It's also a scummy thing to do and sets a terrible precedent. Do you want friendly customers on your WiFi or do you want angry customers who feel cheated when they learn the truth?

<comment><USER4> I think it was basically a joke, and I think his "guests" are friends, not customers, and would therefore be likely to appreciate the joke even if he actually did it.

<comment><USER3> Ah, okay :)

</comment></comment></comment></comment><comment><USER5> I find it funny how, even after publishing this post, the author hasn't configured a http -> https redirect for his own site.

<comment><USER6> Your sense of humour must be very unique.

</comment></comment><comment><USER7> Excellent write up. That's why we need SSL/TLS with HSTS. Pure HTTP, specially in public WiFi, is dead.

<comment><USER8> Since we won't get https everywhere soon, is WPA2 on a public Wifi with a publicly known key a workaround? Should prevent plain MITM?

<comment><USER9> If you control the AP, you should disallow client to client communication. Most AP's and routers allow this and it would mitigate this risk.

<comment><USER3> Is there any way to mitigate this without limiting abilities of people on the network? It kind of destroys the point of a LAN.

<comment><USER8> Did you ever use LAN functionalities in public Wifi (e.g. Starbucks)?

<comment><USER10> I did once at a hotel. Someone on the LAN kept what appeared to be his entire MP3 collection in his Shared folder. So I downloaded the whole thing.

Turned out he had crap taste in music and I ended up deleting my copy.

</comment></comment></comment></comment></comment></comment><comment><USER11> This is why it's important to always use a VPN when connecting to an untrusted wifi, such as a coffee shop or airport wifi. Either pay $3 /month to a provider or setup your own with something like pivpn.

<comment><USER12> One problem I have with that is: by the time I'm conected to WiFi, I don't know how much traffic has already passed through before I could active the VPN. How many background tabs suddenly realized that they had internet again and started sending information (and how many of those used insecure, third party scripts?). Many apps/programs also seem to happily start phoning home as soon as they got WiFi.

I used to have little snitch[1] set up custom rules depending on where I'm connected (allow only local network on unknown WiFi's until connected to a VPN) but that never really worked well because some WiFi's allowed third party IP addresses (to tracking scripts or their home page) which meant I got to tracking down this IP and adding a temporary rule for that. Suddenly quickly connecting to hotspots often became a tedious 10 minute process. This also had the positive side effect that I could prevent A LOT of apps from phoning home but at the end it was not worth all the hassle (because almost nothing just worked) and I decided not to install little snitch for my current installation. The only thing I really miss it is when I connect to my phone hotspots because I'm always afraid application XYZ decides to download an update and eat my (very limited) mobile bandwidth.

Furthermore there is no way to do that on my mobile phone where I have even less control over. My current solution is to never connect to free WiFi networks in the first play and in the few cases I need to, just hope that the provider is not evil. This sucks when I'm on vacation, though, because I'm at their mercy.

[1]: a very flexible application based firewall which allows you to set which app is allowed to connect to with ip/dnsName:port [https://www.obdev.at/products/littlesnitch/index- en.html](https://www.obdev.at/products/littlesnitch/index-en.html)

</comment></comment><comment><USER13> Given the recently disclosed vulnerabilities, instead of a cryptocurrency miner, it could be a Spectre exploit trying to scan and exfiltrate data from the computer's memory. We might be now at the point where disabling all Javascript for non-HTTPS pages is a good default.

<comment><USER14> Why do HTTPS pages get a pass? Between CDNs and ad networks, there's a ton of code out there. At some point, we decided that a magical protective box could make it okay for random people on the internet to run code on our machines. We keep finding this premise to be flawed, with Applets, and with Flash, and now with Javascript, and we always say "oh, if only we had a better protective box, it would have prevented this specific form of attack". Maybe the premise is flawed. Maybe no box is strong enough. Maybe we should stop running code from websites.

<comment><USER15> > Maybe we should stop running code from websites.

Which would mean that even something as simple as an up-vote on a comment would require a full page refresh from the server. The lag would probably kill most social media.

You could be on to something there.

<comment><USER16> > as simple as an up-vote on a comment would require a full page refresh

There are certainly ways to solve this declaratively.

<comment><USER17> One of them being making all form submits asynchronous and not reload the entire page. Or the server sends only the bytes that it changes back.

</comment></comment></comment></comment></comment><comment><USER18> This is (one of the reasons) why we need https.

</comment><comment><USER19> I guess all it takes is one request to a non-https site?

<comment><USER20> Well, one persistently active tab with a non-https site open.

I suppose you could configure the rogue AP to have one of those registration pages but the registration page tells them the WiFi will only work so long as they keep that tab open.

</comment></comment><comment><USER21> Interesting method, but yes; wouldn't HTTPS mitigate this script from being injected? Trying to get awareness for my own original miner written from scratch [https://www.sparechange.io/](https://www.sparechange.io/) Interesting learning WASM.

</comment><comment><USER22> Some buildings (hostels and shared accomodations) have shared internet (secured with WPA2). This type of attack might be particularly profitable in such situations.

<comment><USER23> Wouldn't WPA2 protect users from an attack like this?

<comment><USER24> Not if the attacker knows the password (which is probably just listed on the wall in the hostel)

</comment></comment></comment><comment><USER25> Does someone need to have control of the router to do this? Or how could it work otherwise?

<comment><USER26> It's in the article. He spoofs the MAC address to trick the gateway into thinking the attacker is the victim.

</comment></comment><comment><USER27> (OT: petteralexander's name shows a different color than other usernames. Why?)

<comment><USER28> New users are green, iirc 50 or 100 karma points required to get normal grey

<comment><USER29> Feels like they are more superior than us based on his color.

<comment><USER3> That's probably a cultural thing. In some cultures, red is superior to green.

Really, it's not about them being better or worse than a user with more Karna. It's just a signal to everyone that this person might be advertising or AstroTurfing.

<comment><USER30> What?! I didn't know it indicates that the user is "untrusted", I'm from Sweden and my impression has been that the green users are super users of some kind.

<comment><USER17> That was my impression as well. I never knew that. Guess it's good to know though.

</comment></comment></comment></comment></comment></comment><comment><USER31> Won't modern browsers block this anyway?

</comment><comment><USER32> Https everywhere extension fix it right?

</comment><comment><USER33> This is great! Wow!</comment>


RSA: Can Adobe Stop the Hate? - <AUTHOR> http://www.csoonline.com/article/558913/RSA_2010_Can_Adobe_Stop_the_Hate_

<comment><USER2> _"We understand that the reason Adobe is such a big target for the bad guys is that it's so ubiquitous. Something like Reader or Flash player is installed on just about every single machine out there that's connected to the Internet. That means the bad guys don't have to work so hard because if they can find a problem to exploit it can be directed at every machine. As a result, every bad guy on Earth is looking for something to exploit in our software."_</comment>


MongoDB Is Abusing JSON - <AUTHOR> http://smsohan.com/blog/2013/01/17/abusing-json/

<comment><USER2> Here's what this query looks like in RethinkDB (also based on JSON documents):

      r.table('orders')
       .pluck('cust_id','ord_date','price')
       .groupBy('cust_id','ord_date', r.sum('price')).
       .filter(r.row('reduction').gt(250))

We use the hard-coded attribute 'reduction' because _groupBy_ automatically gets compiled to our distributed map-reduce infrastructure. There is currently no _as_ command (though it could easily be simulated with _map_ ). I'll add a GitHub issue for this shortly, since we should add sugar for it.

<comment><USER3> The beauty of JSON is you could incredibly easily write such jquery like chained expression to generate the mongo query. JSON is a good ascii representation of structured data thats compact, quite readable and reasonably human editable.

Its not an abuse of JSON to use it as a way of representing queries, but its probably a shame that Mongo haven't provided a better way of generating queries.

<comment><AUTHOR> Totally agree with you.

</comment></comment></comment><comment><USER4> Aggregation is absolutely one of Mongo's weaknesses. It's not great at ad-hoc aggregation like MySQL or whatnot is, and the fact that it tends to lend itself to denormalized data makes SQL-style reporting clunky at best.

It does a lot of things better than SQL, too. Consider, for example, the query "Give me a list of all posts with all of these tags, by any of these authors, sorted by post date descending"

        db.posts.find({
          tags: {$all: ["foo", "bar", "baz"]},
          author: {$in: ["Joe", "Jane"]}
        }).sort({post_date: -1})

In SQL, you'd end up with something like:

        SELECT posts.* FROM posts
          INNER JOIN post_tags t1 ON t1.tag = "foo" AND t1.post_id = posts.id
          INNER JOIN post_tags t2 ON t2.tag = "bar" AND t2.post_id = posts.id
          INNER JOIN post_tags t3 ON t3.tag = "baz" AND t3.post_id = posts.id
          WHERE posts.author = "Joe" or post.author = "Jane"
          ORDER BY post_date DESC;

Its strength is denormalization; since you can denormalize entire lists or maps of data into a document, and then index and query on them, you can end up performing queries that would be ridiculously ugly and tedious in SQL.

<comment><USER5> not to take away from any of your great points in this post but isn't it the same as:

        SELECT posts.* FROM posts
          INNER JOIN post_tags pt ON pt.post_id = posts.id AND pt.tag IN ('foo', 'bar', 'baz')
          WHERE posts.author IN ('Joe', 'Jane')
          ORDER BY post_date DESC;

My SQL is rusty, I could be missing something but they seem essentially equivalent if you use the SQL helpers such as IN.

<comment><USER4> No, because that'll select any post that contains any of those three tags, not the posts that contain all three tags. AND vs OR.

It's worth noting that I threw the SQL query a bone by denormalizing post_tags into one table. In a properly relational DB, you'd have a tags table, a posts table, and a post_tags join table, so the query gets even hairier (or you have to do two queries).

        SELECT posts.* FROM posts
          INNER JOIN post_tags pt1 ON pt1.post_id = posts.id
          INNER JOIN tags t1 ON t1.tag = "foo" and pt1.tag_id = t1.id
          INNER JOIN post_tags pt2 ON pt2.post_id = posts.id
          INNER JOIN tags t2 ON t3.tag = "bar" and pt2.tag_id = t2.id
          INNER JOIN post_tags pt3 ON pt3.post_id = posts.id
          INNER JOIN tags t3 ON t3.tag = "baz" and pt3.tag_id = t3.id
          WHERE posts.author = "Joe" or post.author = "Jane"
          ORDER BY post_date DESC;

Yikes.

It's _also_ worth noting that this generates a _massive_ temp table to be sorted, which is very likely going to end up causing you to have to do a filesort. In practice, you'd probably break this down into 3 queries (forgive my mixing languages):

        $tag_ids = SELECT id FROM tags WHERE tag IN ("foo", "bar", "baz")

        $post_ids = SELECT posts.id FROM posts
          INNER JOIN post_tags pt1 ON pt1.post_id = posts.id and pt1.tag_id = $tag_ids[0]
          INNER JOIN post_tags pt2 ON pt2.post_id = posts.id and pt2.tag_id = $tag_ids[1]
          INNER JOIN post_tags pt3 ON pt3.post_id = posts.id and pt3.tag_id = $tag_ids[2]
          WHERE posts.author = "Joe" or posts.author = "Jane"

        $posts = SELECT posts.* FROM posts WHERE id IN ($post_ids) ORDER BY post_date DESC;

Easily doable in both languages, but Mongo's denormalized structure makes this sort of use case a _ton_ simpler.

<comment><USER6>
      SELECT * FROM post
      WHERE id IN
      (
         select post_id from post_tags
         INNER JOIN tags ON post_tag.tag_id = tag.id
         WHERE tag.tag in ("foo","bar","baz")
         GROUP BY post_id
         HAVING COUNT(*) = 3
      )
      AND author IN ("Joe","Jane")
      ORDER BY post_date DESC;

Of course normalizing tags is silly as they are natural keys so it should be as you first stated:

      SELECT * FROM post
      WHERE id IN
      (
         select post_id from post_tags
         WHERE tag in ("foo","bar","baz")
         GROUP BY post_id
         HAVING COUNT(*) = 3
      )
      AND author IN ("Joe","Jane")
      ORDER BY post_date DESC;

<comment><USER4> Are subselects still ridiculously inefficient in MySQL? It's been a long while since I've used them. I do like how those queries read, though.

<comment><USER6> Sorry I only use proper DB's like PostgreSQL and MSSQL who optimizer's have no problems with this ;).

Also what I like about the count approach is you can do "show me all the posts with at least 2 out of the 3 tags matching" fuzzier search if you desire.

</comment></comment></comment></comment></comment></comment><comment><USER7> I've been through the hassle of programatically piecing together complex SQL queries, and I'd far rather be able to just put together hashes that represent my query.

SQL was originally designed so that people who were savvy but not necessarily developers were able to query databases, but I can't think of the last time my boss would have wanted to run some random query against our production database.

<comment><USER8> This is a daily occurrence at some companies

</comment></comment><comment><USER9> There is always <http://querymongo.com/> which will convert SQL to a MongoDB query.

<comment><USER4> I actually dislike this tool a fair bit, because it gets people to continue thinking of Mongo as "Mysql plus WEB SCALE" or whatever. It's a totally different database and doesn't do well with highly-normalized relational-style data, and the idea of an "automated converter" seems to reinforce the idea that it's just a drop in for MySQL that automatically solves all your scaling woes, when that's just utterly and completely false.

Trying to shove a MySQL square into the Mongo triangle just _isn't going to work out that well_.

<comment><USER10> You mean SQL not MySQL.

Also that might be one thing it does but it also allows people to transition from SQL queries they know to mongodb queries. It helps the learning process.

<comment><USER4> No, I mean MySQL. The tool linked is specifically tooled for MySQL queries.

I fully support it as a learning tool, but Mongo's query language isn't all that hard to pick up if you already know how SQL works. Getting stuck into the mindset of "just write relational DB-oriented software and then generate some Mongo queries for it" is an awful practice that is sure to lead to much pain and suffering.

</comment></comment></comment></comment><comment><USER11> I'm just finishing off a project that was built using Mongo and I've run into this as well.

Other gotchas too, like feeling like you can store any old json structure in your db when you can't.

Dots are reserved because they're part of the query syntax. Fair enough, but it's pretty crappy to have to unpick a whole data structure because it was fine until a random bit of UGC was entered (that's where my last fews hours just went).

It does feel like the data and the query syntax are too crossed over to me.

</comment><comment><USER12> You picked an ugly mongo query, and there are many. You compared it to a concise SQL query, and there are many that are not.

MongoDB's limit(x) and skip(y) are a shitload nicer than most of Microsoft's ideas about pagination. It was only in SQL Server 2012 that they came up with "OFFSET" instead of "google it".... [http://stackoverflow.com/questions/2244322/how-to-do- paginat...](http://stackoverflow.com/questions/2244322/how-to-do-pagination- in-sql-server-2008)

<comment><USER13> There are extremely few modern cases where pagination at the database layer is a good approach.

<comment><USER14> How about any cases where you don't want to transfer all the rows you're skipping over the wire? As in, nearly all cases?

<comment><USER13> You wouldn't be "skipping" those rows because they would be transferred once. The database is almost always the most constrained part of a solution, and the various hack pagination techniques are almost always (not 100%. More like 98%) a naive mistake.

<comment><USER15> I am really interested in this argument, because I am currently converting a mysql based web search to use apache solr. I have been thinking about all the arguments about pagination a lot.

The thing I don't know about, though, is the "database is almost always the most constrained part of a solution". On this simple site search, we tend to notice the apaches taking a lot more cpu than the databases calls. I guess when our site gets bigger, you'd imagine it is easier to scale out the apaches than the database (with sharding), but we would still have a lot of room to improve the mysql layer anyway (memcached for example).

</comment></comment></comment></comment></comment><comment><USER16> JSON is used as a query language because it's fast, easy to parse and easy to generate dynamically. If you have a query interface for users, SQL is probably a better choice, but Mongo chose JSON for performance reasons.

If you don't like dealing with it directly, use something like MongoEngine so you're not working with the raw queries, or if having readable, easy to understand queries is important, use a SQL database.

Everything is a compromise, with Mongo's query language you're sacrificing readability for performance.

( This is not a comparison of a SQL database to Mongo, just the time it takes for a SQL engine to parse the query into an execution plan )

<comment><USER17> > JSON is used as a query language

Maybe a little more accurate to say JSON is used as a base layer for the query language.

JSON is "JavaScript Object Notation". But the "meaning" of the query is in the objects being denoted, not the notation used to represent them as text. So comparing Mongo's use of JSON to SQL is apples-to-oranges.

We could encode SQL as JSON too:

        {"query": "SELECT * FROM things;"}

or

        {"query": [
            {"SELECT": "*"},
            {"FROM": "things"} ] }

without affecting the expressive power of the SQL language one bit.

</comment></comment><comment><USER18> I don't think it's fair to take JSON out of context from the rest of the query API. Or maybe it's being overly generous, I'm not sure. Either way, this is the same way object literals are constructed in JavaScript. So is the beef w/JavaScript?

For me the Mongo shell is just enough so-called "richness" and "expressiveness" (Try it yourself: <http://try.mongodb.org/>). There's a certain magic to passing objects to functions (and being able to, say, read the body of a function by typing that object into the CLI).

</comment><comment><USER19> TL;DR: JSON sucks for representing queries.

<comment><USER20> It's not JSON, but rather that MongoDB's query language sucks. I'm not entirely sure how to fix it - perhaps make it slightly more verbose and meaningful?

<comment><USER19> Probably there are better ways to express a query in JSON compared to how MongoDB does it, but I'd take a step back and ask whether a nested map is the best approach to think about it.

<comment><USER4> I actually prefer Mongo's query language to SQL, because it makes sense to my programmer brain. It's basically "push operand, push values", which makes a ton of sense. It's just plain old Polish notation.

</comment></comment></comment></comment><comment><USER21> Meh. I count roughly the same number of tokens either way. I really don't think this is that big of a deal.

</comment><comment><USER22> I feel the same about the query language of Mongo. The first goal of a query language should be ease of use. With mongo having to type all those extra characters quotes, brakets, square brakets, colons is very annoying. You need to type a lot to get any reasonable output.

<comment><USER23> You might like coffeescript.

        db.orders.aggregate [
          $group:
            _id:
              cust_id: "$cust_id"
              ord_date: "$ord_date"

            total:
              $sum: "$price"
        ,
          $match:
            total:
              $gt: 250
        ]

</comment></comment><comment><USER24> MongoDB is a NoSQL-type database, so it wouldn't make sense to have a SQL query interface... I think they did a good job with the API for not using SQL.

Plus, the API isn't really _abusing_ JSON. It isn't pretty, but it's not abuse.

<comment><USER25> People seem to be confusing "NoSQL" and "non-relational". Mongo happens to be both, but there's nothing fundamental that puts the two together. You may not get the full capabilities of SQL with non-relational data (JOINs, etc), but there's no reason that non-relational data stores couldn't parse normal SQL and execute the appropriate queries.

You can make a relational database that doesn't support the SQL syntax, and you can use SQL syntax to interact with schemaless data (for added fun, try throwing JSON in a mysql/postgres text field).

I'd agree with the article saying this is an abuse of JSON, though. It's a format to represent data; more accurately, potentially-nested key:value stores, arrays, and scalar types. A query is not data (unless you're one of those "my database has a 'queries' table" types)

<comment><USER26> How is a query _not_ data?

<comment><USER25> In the same way that code isn't data.

Yes, it's bytes in memory like any other data, but it's the means rather than the end. You care about the data, and the query (or code) is what you use to get it.

</comment></comment></comment></comment><comment><USER27> I think it's great. I've spent too much time parsing SQL strings into well typed data structures, and you get it for free with Mongo.

<comment><USER15> I kinda agree (about the free part), but I just wanted to say that if you are parsing sql strings you really should get an ORM. Plug in a well tested, optimised library and never think about parsing sql strings again. It also makes things easier to test because you get nice separation of concerns.

</comment></comment><comment><USER28> Mongodb queries are much easier to dynamically build than SQL. And this is so because Mongodb uses JSON for queries.

</comment><comment><USER29> have you seen Elasticsearch API?</comment>


What Are The 2 Most Important Traits in An Entrepreneur? - <AUTHOR> http://blog.eladgil.com/2011/04/what-are-2-most-important-traits-in.html

<comment><USER2> 1\. Persistence 2\. Persistence

Everything else is negotiable.

I'd disagree with tenaciousness as opposed to persistence because I think it implys a zeal that is important for early success, but less so for success in the long run.

<comment><AUTHOR> Persistence is a good way to put it too. Seems like the interpretation of the terms are similar. From Google "define persistent" or "define tenacious"...

Search Results te·na·cious/təˈnāSHəs/Adjective 1\. Not readily letting go of, giving up, or separated from an object that one holds, a position, or a principle: "a tenacious grip". 2\. Not easily dispelled or discouraged; persisting in existence or in a course of action: "a tenacious legend".

per·sist·ent/pərˈsistənt/Adjective 1\. Continuing firmly or obstinately in a course of action in spite of difficulty or opposition. 2\. Characterized by a specified habitual behavior pattern, esp. a dishonest or undesirable one.

</comment></comment><comment><USER3> As a single founder, I'm finding that the most important trait is not giving up.

Just keep trying.

March on.

</comment><comment><USER4> 1) Not reading articles about 'x most important traits of an entrepreneur'.

2) Not giving up.

</comment><comment><USER5> I think the two traits are simply a complete belief in your idea, and the willingness to work as hard as need be to make it successful. I think that as long as you have these two things(and your idea doesn't suck) that you'll be successful.

</comment><comment><USER6> I'm curious as to why you think charisma is sometimes a negative influencer of success?

<comment><AUTHOR> If you meet a number of founders of really big companies, in many cases they are sort of awkward, introverted people. The salesy, flashy people may be really good at convincing, but sometimes they can get away with less execution. This can be a good or bad thing depending on the type of startup and the market it is in.

There are also obvious examples of highly charismatic people who build big companies (e.g. Steve Jobs).

As an aside, I think Mike Moritz (investor in Google, Yahoo!, PayPal, etc.) also mentions something along the lines of "awkward/shy founders are often the ones who do big things" in a recent video interview (maybe the TechCrunch one?)

I am not sure what the cause is. Maybe inward focus helps build a great product and ignore feedback from people trying to influence your vision in the wrong direction? While charisma suggests being strongly attuned to others, whose feelings you may hurt by forging your own path? I dont know...

<comment><USER7> You're equating tech entrepreneurs with entrepreneurs on the whole. There's a _big_ business world outside of tech.

Case in point: Richard Branson. Arguably one of the most charismatic entrepreneurs around.

<comment><AUTHOR> Agreed, charisma can be a huge asset. But it is not necessarily an indicator of a good entrepreneur, and it is not a pre-requisite. I think the other 2 traits are :)

</comment></comment></comment></comment><comment><USER8> Thanks for choosing the word "tenacious." I get so tired of hearing that entrepreneurs need to be "stubborn." Tenacity differs from stubbornness, and I'm really happy to see you make that distinction.

<comment><USER9> Same goes for words like hustle or pigheadedness. I think grit maybe an even better way to state it. Stubbornness and tenacity can work for or against an entrepreneur or business. (comment inspired by reading [http://andymckenzie.blogspot.com/2011/04/when-can-we- measure...](http://andymckenzie.blogspot.com/2011/04/when-can-we-measure- grit.html))</comment></comment>


The Big Four: Culture, Innovation and Sentiment - <AUTHOR> http://tastethecloud.com/content/big-four-culture-innovation-and-sentiment <comment><USER2> brilliant analysis</comment>