Nnn – a terminal file manager for programmers - <AUTHOR> https://github.com/jarun/nnn <comment><USER3> Show HN is for sharing your own personal work. We got a complaint saying that this is not the submitter's work, so I've taken Show HN out of the title.

Posting as a Show HN something you didn't create is effectively taking credit for someone else's work, so please don't do that!

Please read the rules: [https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html).

<comment><USER4> Hi, author of `nnn` here. I didn't submit it but I am actively responding to the comments and it is actually helping me to get feedback and feature ideas.

I am grateful OP posted it. Thanks for fixing the title.

<comment><USER3> Thanks, that's good to hear.

</comment></comment></comment><comment><USER5> A lot of good features in nnn. Unfortunately, for me, the huge deal breaker is the (sorry, but I have to say it) _terrible_ choices for one of the most important hotkey-functionality combinations.

I am not talking about some rarely used keys/functions, but the problem lies at the very core of the experience: the simple hjkl, vim-style navigation.

You see, as in vim, k/j ('up'/'down') obviously move up/down in the list of dirs/files. h ('left') moves up the directory hierarchy ('Parent dir' as nnn's help aptly describes the function.) Of course, now l ('right') moves down the directory hierarchy: IF you are sitting with the highlight on a DIRECTORY entry, obviously if moves down into that directory.

But here is the kicker: what if you have as your current highlight something OTHER than a directory, that is a FILE ?

It OPENS the file. (The open operation is done through xdg-open, so whatever that is configured to recognize) If xdg-open is not configured defensively, in the case when the file is executable it runs the program. Or if the file has some associated program, it is launched with the file as an argument. (this was happening even on my machine, and I have a relatively conservative xdg- open config) At the minimum it launches an editor, opening the file in read/WRITE. In nnn's help, listed among the keys activating this 'Open file/enter dir' is the <Return> key, in addition to the usual move right keys (so 'l' and right arrow). A clear expression that in the authors intention those keys have equivalent functionality.

So imagine you're browsing around the file system as fast as your fingers and eyes allow (after, that's the beauty of the fast text/ncurses interfaces, yes?). You just want to examine stuff, look at files, poke into configurations, maybe for a archive you just downloaded. Archive that might or might not have executable stuff and/or potentially harmful things. You've located the dir path you want to go into, you dive in and before you realize you need to hit the brakes on that 'l' key, you end up moveing RIGHT on SOMETHING (it happened so fast you're not even sure what it was), and that SOMETHING was executed/loaded, now some application is opened, and on top of that you were continuing to type so that application receives your keys and does who-knows-what (well, depending on what was launched).

WTF ??? The first time it happened, I could not believe such a UI design decision: mixing NAVIGATION and OPEN operations on the SAME KEY. OPEN is a sensitive, potentially disruptive operation: there SHOULD be some minimal friction provided in the interface before one is allowed to proceed.

It moved this nnn from potentially one my favorite tools, into a totally love/hate relationship. Yes, "love" because indeed it does some things well. But the mistake of this RIGHT/OPEN snafu is enough to more than cancel any other qualities.

<comment><USER6> That's the way it works in most file managers... Usually a click would open a directory or a file just the same

<comment><USER5> First of all: when using a _mouse_ 'a click would open a directory or a file just the same'. A mouse/trackpad has pretty much _one button_ (or equivalent) dedicated to interaction, so it is expected to overload some functions on it. Besides, "navigation" with the mouse is a different thing: you don't need to "move" the selection so opening is a clear voluntary decision, decoupled from navigation.

But focusing back on _keyboard_ operation... It is true that many text based file managers do this 'open file on right navigation move'. In my opinion that doesn't make it right: I personally consider a bad thing that can lead to startling, unnecessary and potentially sensitive application launches; a separation between movement (hjkl, arrows) and opening (obvious choice here, <Return>) is a sensible design.

Another point of view: the unwanted file-open-by-mistake situations are a form of involuntary file previews. As nnn's own 'design considerations' state [0], previews are to be avoided: "previewing ... is a wastage", potential "slow/faulty ... access makes the experience very painful" and "Users should see only what they _explicitly_ want to see".

Lastly, there is an important aspect that mitigates the problem for 'most file managers': customization.

Here's a quick glance at the players in this text-fm arena:

\- ranger and vifm, the larger managers: a cursory look in their help/configs hints that they can be heavily customized \- out of the box ranger displays a preview of the selected file; that acts a big visual hint that you don't need to 'navigate' further; however I agree that default previews are bad \- ranger and vifm remember the position along the path, so you don't get shunted moving back down \- noice has about the same behavior as nnn (nnn was forked from it), but at least it has a standard suckless style config.h header allowing customization by recompiling \- speaking of suckless style, a last resort solution is to do just that with nnn, making your own fork: its header largely keeps the structure of the original

And here are examples that work more along the lines I prefer:

\- rover has the best behavior IMO: it can be similarly configured using environment variables, but by default it uses hjkl simply for navigation and NOTHING ELSE; it has separate, dedicated keys for open, preview or other things \- midnight commander has the same separation: arrows (actually just up/down) strictly for navigation (in local dir ONLY, no less); explicit directory navigation and opening with <Return>; there is no implicit 'file open'

[0] [https://github.com/jarun/nnn/wiki/nnn-design- considerations](https://github.com/jarun/nnn/wiki/nnn-design-considerations)

</comment></comment></comment><comment><USER7> I do not use a terminal based file manager - I find that a better workflow for me is to use sshfs to create a local mount on my laptop and then just use the Finder to browse ... which gives me preview, etc.

HOWEVER, there is a specific use-case - renaming a bunch of files in a directory - where I do use a "file manager" and that file manager is 'vimv'.[1]

vimv is a very simple tool - you run vimv in a directory and suddenly the vi editor opens up with the contents of that directory. You then use the vi editor to edit the filenames any way you like. Upon :wq, the files in that directory are renamed.

Very useful - and in many cases, much faster than crafting a sed/awk one liner to do the renaming (especially if some of the renaming is arbitrary or random).

[1] [https://github.com/thameera/vimv](https://github.com/thameera/vimv)

<comment><USER8> Obligatory mention of emacs wdired-mode, which sounds very similar. I mention it because I used emacs for almost a decade before noticing that there's a wdired mode as well as dired mode.

<comment><USER9> There also sunrise mode, which combines (w)dired with ortodox file managers. Easily one of the best ways to deal with files. Particularly in combination with Tramp...

<comment><USER10> What is sunrise mode? Are you referring to [https://www.emacswiki.org/emacs/Sunrise_Commander](https://www.emacswiki.org/emacs/Sunrise_Commander) ?

</comment></comment></comment></comment><comment><USER11> The search is extremely cool and fun to use. Directories all seem to load very very quickly. It's got vidir integration, giving it bulk-renaming. I wasn't familiar with vidir, but the ":bulkrename" command in ranger was my favorite part of it. After a few minutes of poking around with nnn, I ended up installing it on all my machines. The only weird thing I noticed was that I couldn't find a straight-forward way to delete files. The first thing I came up with was to hit ! to go into a shell and then just use rm. Not sure if I was missing something or if this was deliberate to make sure you don't accidentally delete files.

<comment><USER4> In the just-released version 2.1, you can just select a file and press `^X` to delete it.

<comment><USER12> I am testing Nnn via Homebrew and I am also missing the ability to delete. The in-program help lists '^X' as Quit.

Is anyone else who installed via Homebrew able to delete?

<comment><USER4> The DEL key wasn't handled in rename prompt. The fix is available on master branch.

<comment><USER12> Just noticed you mentioned 2.1 and Homebrew installed 2.0. That is probably why I am not seeing it.

<comment><USER4> It's not available in 2.1 either. Last night a user reported this after release of 2.1. :)

</comment></comment></comment></comment></comment></comment><comment><USER13> This looks incredibly cool, I love the speed and minimalism. It doesn't seem as configurable and feature-packed as vifm[1], so I'll probably stick with that, but it certainly fills a good niche.

> nnn vs. ncdu memory usage in disk usage analyzer mode (400K files on disk):

I assume that's because nnn isn't keeping the entire directory structure in memory, which means that browsing to a subdirectory involves rescanning the entire directory. That's a fair trade-off, but an unfair comparison.

1\. [https://vifm.info/](https://vifm.info/) \- I'm surprised this hasn't been mentioned yet in this thread.

<comment><USER4> Author of `nnn` here. Thanks for the appreciation.

Please let us know what you like in vifm which isn't available in `nnn` and we will consider the features. However, I must say the last thing we want to see in `nnn` is feature bloat. You can extend it as you wish through scripts which `nnn` supports in any number.

No, `nnn` scans the complete dir tree otherwise du won't work. It rescans because data on disks keep changing e.g. on our server where 17 people run VMs. It can be easily changed to static but the rescan is very fast and users specifically asked for it. The memory usage is less because `nnn` uses much less memory for all operations in general.

<comment><USER13> I appreciate your work, but you're not being very honest with your claims.

nnn is _not_ keeping information about 400K files in memory in that benchmark. As a result, the rescan is necessary when changing directory. The rescan may be fast in many cases and in some cases it may even be what you'd want, but I can also name many cases where you certainly won't want it (large NFS mounts being one example).

Sorry for the pedantry. I spent a fair amount of time optimizing ncdu's memory usage, so I tend to have an opinion on this topic. :)

<comment><USER4> I think we are saying the same thing in different lingo. I am trying to say, you do not need to store it if you can have fast rescans.

Coming to memory usage, if you store the sizes of every file you need 400K * 8 bytes = ~3 MB.

Now `ncdu` uses ~60 MB and `nnn` uses ~3.5 MB. How do you justify that huge gap?

> but you're not being very honest with your claims

No, I am completely honest within the limits of my technical understanding. Your tool uses 57 MB extra which would be considerable on a Raspberry Pi model B. To an end user, it's not important how a tool shows the du of `/`, what's important is - is the tool reasonable or not? I don't know how `ncdu` manages the memory within, I took a snapshot of the memory usage at `/`.

In fact, now I have questions about your very first line beginning with `This looks incredibly cool` and then the comparisons of it with different utilities in negative light. (I must be a fool realizing it now, I should have seen it coming.)

<comment><USER14> Your calculation (`400K * 8 bytes = ~3 MB`) is way off. What would be the point of storing only the size? You need to map it back to the file.

60MB gives you about 150 bytes for file path or file name and its size, which sounds plausible.

<comment><USER4> I can think of at least 3 possible algorithms to use much much less memory even with a static snapshot of the complete subtree. And all of them are broken because the filesystem is supposed to stay online and change. It's realistically useful to scan an external disk to find the largest file etc., but not accurate on a live server, a desktop with several ongoing downloads, video multiplexing etc.

<comment><USER14> Earlier in the thread you suggested that it's hard to justify ncdu using 60MB while it takes only 3.5MB to store 400K * 8 bytes numbers. The number you came up with is just silly and overlooks actual complexity of the problem.

Given that you are making an implicit judgement about the other program, don't be sloppy with your estimates.

<comment><USER4> > don't be sloppy with your estimates

I'm not. You can, and I'm sure eventually you will arrive very close to the approximation.

I'd been a big time fan of `ncdu` for years and even wrote in an e-journal about it once. Maybe that's why the sharp adjectives became more difficult to digest. Anyway, good luck!

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Every time I see interesting file managers it makes me want to try and like them but I always end up abandoning them after either a few minutes or a few hours. I don't know why. I'm quite sure I could be more productive using a file manager, even a graphical one. But it's been more than ten years that the only file manager I actually use is Bash and the GNU coreutils.

<comment><USER16> Moving or copying files a single time might be a bit faster using a GUI (or TUI). But when you factor in starting the file manager and navigating to the right place, it's not a big difference.

And when you need to do a more complicated operation you're looking at a shell oneliner vs. multiple (possibly repetitive) manual steps in the UI.

If simple copying of files made up a great deal of my daily work, I'd certainly use a file manager (if I couldn't write a script to do it).

<comment><USER17> The primary benefit of any GUI over a functionally equivalent CLI is the improved discoverability, especially for rare, one-off tasks. CLIs over GuI primarily offers better repetition.

Any comparison between the two systems should stem from there: you've only offered half of it.

Eg getting a remote virtual disk set up in OSX's finder is much easier than on terminal, without googling/general research, for a one-off drive. Getting 50 such disks up and going is far easier on terminal. (CLI has a better O(), but worse constant factor.)

<comment><USER18> I disagree partly here: hitting tab twice in a shell usually lists available commands, possibly with some prefix. A GUI has the advantage of context, in that eg menu items of the disk utility make sense as you know they'll be some operation on disks.

On CLI eg I don't remember the mkfs command for a file system, I type mkfs and hit tab twice.

It could be better though, of course.

<comment><USER19> That is the problem right there, you need to know mkfs exists to start with.

<comment><USER18> Well one needs to know the Disk Utility exists as well, or disk management thingy in Windows.

<comment><USER19> Sure, but that is available on a simple right mouse click on the device.

In any case you are focusing too much on mkfs, I can think of plenty of other examples, many of each aren't even portable across UNIX environments.

While GUIs at least can offer navigation and visual cues as means to discover features.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> Very nice! I often end up running MidnightCommander but this would be enough in most cases. I really like the idea!

<comment><USER21> +1 for mc! Using it for almost 2 decades now. First on linux, now on macOS.

<comment><USER22> I use it on my RPis, but now I am going to try nnn if it runs on ARM. MC is legendary though and the fact that supports multiple transfer protocols is a very helpful.

<comment><USER4> `nnn` integrates with lftp. I have added a wiki page on how to automate transfers or copy selection easily.

</comment></comment></comment></comment><comment><USER23> My friends have been using it for a few days and they seemed quite happy with it. The only thing that stops me from using it is that I need thumbnails, so I am stuck with spacefm for now.

<comment><USER24> Any particular reason spacefm stands out to you over the rest?

<comment><USER23> I tried multiple file managers in the past. Dolphin is really good but it has a lot of KDE dependencies. Thunar had quite a few bugs that annoyed me, for example when I configured backspace to go to the parent directory it activated even when I was editing the path. In the past I used to use Nautilus until it replaced the quick search feature with a recursive search thing, in addition to removing the ability to change keybinds, plus I kept getting crashes and getting a lot of lag when visiting a directory with 10k+ images with it later on. Most of the other file managers that I tried either did not have thumbnails or they did not support tabs.

Spacefm does not have any of the above issues, which is why I am currently using it.

<comment><USER24> I use Thunar and avoid Nautilus pretty much for the same reasons. I like Thunar because it keeps things simple, and I have bash for anything complex, which a lot of my file operations tend to be. I'll check out spacefm and see what I think, thanks for the recommendation.

</comment></comment></comment></comment><comment><USER25> Emacs Dired + Tramp -> no need for any other tool.

One super awesome Dired feature is C-x C-q to edit a folder like a file (all Emacs commands work here, like macros which are easier & faster to get right than most shell commands). To save your changes after editing, press C-c C-c.

<comment><USER26> I would love to hear other dired features that people use. I'm just now getting a comprehensive understanding of how great dired is.

<comment><USER22> Does it support file transfer over ftp, sftp, s3, etc?

<comment> ISO-morphism Yes, that's where tramp comes in, although s3 might need a separate plugin.

For me the best part of dired is "do-what-i-mean" mode, dired-dwim. You basically get the output of `ls -la` and edit it as a text file. When you save, dired figures out changed names/permission bits/etc and does what you mean to those files.

</comment></comment></comment></comment><comment><USER27> Lack of file previews makes it a deal breaker. Ranger is a bit slower, but it does so much more.

<comment><USER28> I agree. Would be very nice if this was an option at least (that could be turned on by configuration). On the design considerations [1] they state:

> Previewing large files on selection is a wastage of processing power and > time. A slow/faulty disk or over the network access makes the experience > very painful.

You only need the first lines of the file, not the whole file, so this argument is not valid.

> It's also a risk to privacy and confidentiality. Users should see only what > they explicitly want to see.

Not valid if preview is off by default. A hotkey could toggle preview mode.

Yes, you can open each file in a pager. Exit the pager, go to the next file and repeat. That is slow and painful. Being able to browse a code repository with file preview is the main reason I will probably not use this and stick with ranger. It's a deal breaker for me.

I really like that it's fast (I've tried using ranger on the USB Armory – it was very slow), and if preview was added as an option I would seriously reconsider.

[1]: [https://github.com/jarun/nnn/wiki/nnn-design- considerations](https://github.com/jarun/nnn/wiki/nnn-design-considerations)

</comment></comment><comment><USER29> > nnn is probably the fastest and most resource-sensitive file manager you > have ever used.

Volkov Commander is written in assembler, binary has 32KB with zero deps, memory requirements far bellow 1MB. It has better UI and more features.

<comment><USER4> It's for DOS and is a shareware.

> better UI

I doubt that from the screenshots. But it's a personal preference.

</comment></comment><comment><USER30> How does this compare to [https://github.com/ranger/ranger](https://github.com/ranger/ranger) ?

<comment><USER31> more minimalist, smaller, a tiny bit more responsive. Probably a lot less capable since ranger seems quite featured.

<comment><USER4> In fact every operation would be slower in an interpreted or scripting language. It becomes evident on a Raspberry Pi, Linux subsystem for Windows (even on i7, yes), when opened within vim as a file chooser, or on Termux env (Android). `nnn` finds its users in all these constrained environments.

Also `nnn` is not a feature by feature replacement for ranger. The last thing we want is a feature bloat. And we are very open to useful feature requests. Let us know if you are missing something.

<comment><USER31> We really should all develop on slow machines.

<comment><USER4> develop > test

<comment><USER31> that too. I did so for website. If it loads fast and render neat under elinks Ill keep it.

</comment></comment></comment></comment></comment></comment><comment><USER32> Why doesn't nnn remember where you are in a directory? It seems natural to me that "hl" should preserve your state, e.g. "hhll" should take you back to the same place you were in.

Was it an intentional design choice to make "l" put you back on the first entry in the directory? Why?

<comment><USER4> Yes, we didn't want to remember the last file selected in every directory visited. In other words, it's like remembering the state of every directory visited. `nnn` selects the parent dir when you come out of a child dir.

<comment><USER33> You don't need to remember the state of _every_ directory visited. The functionality GP is asking for is similar to the forward button in the web browser. I.e., if you're starting in `/a/b/c/d`, to be able to go

      /a/b/c/d -> /a/b/c -> /a/b -> /a/b/c -> /a/b/c/d

However, if you instead go into `/a/b/x` after the first two steps, the "forward history" would be lost.

<comment><USER4> Ah, OK! I thought every file in every dir. To do this you would have to remember each absolute path which is 4K max per path (or allocate dynamically and free every time when changed). We try to keep `nnn` light on memory.

I must admit this is the first time I am hearing it's a problem. Probably because of the abundance of other navigation options available in `nnn`.

</comment></comment></comment></comment><comment><USER34> How is this better than Midnight Commander?

<comment><USER4> Author of `nnn` here. I'm not very familiar with MC (sorry about that) as I don't use it so can't provide a meaningful/unbiased feature comparison. However, you can find the list of features in `nnn` here: [https://github.com/jarun/nnn#features](https://github.com/jarun/nnn#features)

In addition, last night I added a vim plugin to use `nnn` as a file picker within vim.

<comment><USER34> Thank you for your reply, and no worries for not being familiar with mc. I addressed the question generally to HN, hoping for some feedaback from anyone who happens to have used both.

EDIT: One nice thing that mc has is that it lets you open a remote shell via ssh. I use that a lot (I find sc syntax a big pain) so I'd miss that a lot in nnn. Sorry if that feature is included and I failed to spot it.

<comment><USER4> > Sorry if that feature is included and I failed to spot it.

Don't be, it's not in yet. It's on the ToDo list. I would really appreciate a PR.

<comment><USER34> A PR? Well, I wouldn't completely write it out, but my C is really rusty and I am being buried under a ton of stuff I'm expected to do (I'm doing a PhD, so I basically have no free time).

Is it just you on this project?

<comment><USER4> It was for anyone who reads the comment.

</comment></comment></comment></comment></comment></comment><comment><USER35> Reminded me of xtree gold

[https://en.wikipedia.org/wiki/XTree](https://en.wikipedia.org/wiki/XTree)

</comment><comment><USER36> You would have my attention if it could rename files like I'm used (on Emacs) with Dired, specifically the Wdired mode... ^__^;

<comment><USER4> `nnn` supports in-place file renaming. It also comes with `vidir` integration.

<comment><USER36> Briefly reading at the description vidir seems similar to Emacs' Wdired and I suppose it's an useful and valid choice, but I've decided long time ago that for me Vi/Vim is an hard pass... ^__^

</comment></comment></comment><comment><USER37> Why _for programmers_? What secret features support programming in particular?

<comment><USER38> I see it as using commands vs direct manipulation, keyboard vs mouse, and - <USER2> a result - prioritizing efficiency at the expense of slower learning curve. Pretty much like vim vs notepad.

</comment></comment><comment><USER39> Works great in Termux on the Gemini. Plus there's a decent man page

<comment><USER4> Thanks for the appreciation! I myself have the habit of using `nnn` on Termux.

</comment></comment><comment><USER40> Does anyone who has used both nnn and ranger (the one I use, and am very happy with) care to explain what are the selling points of that fm compared to this one?

<comment><USER41> Check other threads and replies above

</comment></comment><comment><USER42> How launch executable file from _nnn_? For example:

      > 2018-11-15 09:44   167.8M* Krita-4.1.5-x86_64.AppImage*

<comment><USER4> There was a feature to run executable scripts in `noice` (on enter) but it was dropped in `nnn` as potentially dangerous. At this point you can spawn a subshell and launch it manually.

</comment></comment><comment><USER6> Is there any way to remap the key bindings?

<comment><USER4> Not without re-compiling.

<comment><USER6> Thanks. So it's not much use for non-qwerty'ers as is, I'll have a look at patching it.

<comment><USER4> For several operations, there are multiple keybinds. But I can think of something interesting. If you can come up with a keybind profile for non- querty, we can have a separate config for that set.

</comment></comment></comment></comment><comment><USER43> i'm stuck with long-ago-forked version of vfu [https://github.com/cade-vs/vfu- dist](https://github.com/cade-vs/vfu-dist) Works well, as old-school as it may look.

</comment><comment><USER44> Anyone have any idea of what I am supposed to use as a file opener on OpenBSD?

<comment><USER4> If you can confirm please share so I can update the program to use the default opener on OpenBSD.

I think it might be open(1): [https://man.openbsd.org/file.1](https://man.openbsd.org/file.1)

same as in iOS. But i need the confirmation before making the change. Please raise an issue in the project page for the same.

</comment></comment><comment><USER45> Any chance this has/might get support for Windows natively?

<comment><USER4> `nnn` needs ncurses. AFAIK, it's not available on Windows yet. I use it daily on Linux subsystem for Windows and it supports Cygwin.

<comment><USER45> Excellent point, thanks for the reply, I'll give it a spin on WSL.

<comment><USER4> Have fun!

</comment></comment></comment></comment><comment><USER46> Is there a browser-based file manager somewhere?

<comment><USER47> Local or remote?

Konqueror and w3m are both web-aware local file managers.

</comment></comment><comment><USER48> Still have not done anything better than mc.

<comment><USER4> Does mc have a navigate as you type mode? Just curious.

</comment></comment><comment><USER49> Recommend vifm or why not tmux which is very configurable.

<comment><USER4> ... and much much slower, yes.

I see you vouching around for alternatives in every comment thread here. No issues. But please explain what you are seeing in other utilities that's missing in `nnn`? I would be repeating myself, but we are very open to reasonable feature requests.

<comment><USER49> Done that in main thread about vifm above.

Sorry, but my comments were invincible for long time than suddenly appeared. Thought it was something wrong then all comments was visible. Hard to know if you don't know HN comment system.

<comment><USER4> I didn't realize that. Thanks for the note!</comment></comment></comment></comment>


Text Summarization for Humans - <AUTHOR> https://explaintome.herokuapp.com/ <comment><AUTHOR> This project is open sourced by Sang Han on github too [https://github.com/jjangsangy/ExplainToMe](https://github.com/jjangsangy/ExplainToMe) . Feel free to make changes to this awesome tool.</comment>


Algorithmic Botany: Publications - <AUTHOR> http://algorithmicbotany.org/papers/ <comment><USER2> I love it! From a quick glance down the list, I already found a beauty: Animating Persian Floral Patterns.

[http://algorithmicbotany.org/papers/persian- flowers.pdf](http://algorithmicbotany.org/papers/persian-flowers.pdf)</comment>


I made a cheap HHO Generator. Device helps in reducing fuel consumption. AMA - <AUTHOR> https://techama.com/i-made-on-my-own-very-cheap-but-effective-hho-generator-this-little-device-159099/ <comment><USER2> Since it take more power to split up water than you get by recombining it, I severely doubt it.</comment>


I'm not sure it's Mou against the world - <AUTHOR> http://larryhynes.net/2014/09/mou-against-the-world.html

<comment><USER2> I got my Mou licence the same way the author of the blog post got his and I'm still very happy with Mou for my casual markdown need.

I don't blame him for selling a few things on the side. It's not like they'll take up a lot of time once you organized the manufacturing etc. They just sit there and you ship them once someone orders. I don't think that's a problem to do on the side. Can't say I'm a big fan of the fundraising idea. I'd rather just buy it once it's finished, I don't even expect upgrade pricing.

Just to add to the article, he's also selling MouStand.

[0] [http://moustand.com/](http://moustand.com/)

</comment><comment><USER3> I'd like to suggest haroopad[1], similar to Mou, more powerful and available on Lin/Mac/Win. Based on node.js

[1] [http://pad.haroopress.com/](http://pad.haroopress.com/)</comment>


Upper bound on age of YC founders - <AUTHOR>

<comment><AUTHOR> YC and PG have been effectively convincing young hackers it's not too early to start a startup. Startup is the new NBA - flooded with young talents, putting school aside for a shot at the big league.

While I still believe it's never too late to start a start up, is there an upper bound on the age of YCombinator? Would love to know the statistics on the age of mature founders from YC.

<comment><USER2> In most batches the oldest founder is around 33 or 34. We don't have a rule about this; it just works out that way.</comment></comment>


Ask HN: Hacking away right now - what travel questions to solve? - <AUTHOR>

In between a hackathon, building a travel app that mashes foursquare 'tips' with curated Quora, to produce "Dos & Donts" about places & activities. What questions / concerns do you have about places & things-to-do when you travel ?<p>[http://www.banjarey.com/what_travel_concern] <comment><USER2> I hate to say it, but this post honestly sounds like it comes from a startup idea Markov chain.

Anyway, I'm fond of the Luxe Guide [<http://www.luxecityguides.com/>] model: highly selective high-quality recommendations for a particular city/area. Not "here are 125 restaurants" like most tour books, but "here are the 10-12 you can't go wrong with as a visitor with only a few days."

<comment><AUTHOR> Hey zachliption,

Why would you say that ? Just want to understand.

The idea is to build a very selective "Dos & Donts" around a place / activity - this is not a recommendation for place (125 restaurants) but more like "what's good to eat here" or 'the chef's speciality is this' and other low- level details.

In any case this is just a weekend thing, but I would love to understand what would you have in a travel app that would give you more value than tripadvisor (lets say).</comment></comment>


Ask HN: Will cash and the dollar bill disapear: Tillify.com - <AUTHOR>

We at Tillify HQ are debating whether cash will disappear.
Will mobile payments and mobile card reading apps like square take the dollar bill out of our pocket.
What will this mean in the real world? Everything we buy and do is declared and stored? Is that a good thing?
Will everything be paid for with a tap of our mobile device, or a card?
Parking meters, buss faryes a hot dog in the park.
Tell us what you think.
Tillify.com is a new bootstrap start-up, we build applications that bring point of sale to your device, not just a store-front.
@tillify http:www.tillify.com <comment><AUTHOR> If cash dies selling your surf-board to your friend at the gym will be done using your card or a mobile payment. Does this mean that like Paypal those transactions will be monitored, controlled.

Does this mean one-day they might occur a tax?

Will our constant need to simply everything we do make our lives more controlled and ultimately more expensive?

</comment><comment><USER2> One of the things that sets cash apart from most electronic substitues is anonymity. Any solution that wants to dethrone cash will need to take that into consideration. BitCoin (arguably) does this, but most of the other electronic cash substitutes do not.

</comment><comment><AUTHOR> We agree. The loss of cash is ultimately the loss of our anonymity. Sounds dangerous doesn't it.</comment>


One Year Without Social Media - <AUTHOR> http://www.stevepavlina.com/blog/2015/07/one-year-without-social-media/

<comment><USER2> Nice article</comment>


Vercel (Guillermo Rauch's Blog) - <AUTHOR> https://rauchg.com/2020/vercel <comment><USER2> I liked the name "Zeit" much better, but this is still exciting.</comment>


Rumours swell over new kind of gravitational-wave sighting - <AUTHOR> http://www.nature.com/news/rumours-swell-over-new-kind-of-gravitational-wave-sighting-1.22482 <comment><USER2> This might be a naive question, but I'd like to clear it out.

Do gravitational waves travel at the speed of light?

I know the theory says nothing can travel faster than light. I also know that photons can be seen as quanta or as waves. So my guess is that gravitational waves travel at most, at the speed of light.

But do they? or do they travel slower? faster? Is there a doppler effect for GWs?

I ask because I would think ripples in the space-time fabric itself might be a bit different than light waves or other more studied phenomena.

Can anyone point me in the right direction?

<comment><USER3> The speed of light is sometimes referred to as the speed of causality, and it seems like it's more of a fundamental speed limit on the propagation of events or information through space.

<comment><USER4> IIRC, everything moves through spacetime at c. Things with mass like people, planets, etc, move through the time portion as well as the space portion. As you go faster through space, you travel less through time, though at non- relativistic speeds you don't notice (GPS satellites do have to account for this). Electromagnetic waves have no mass, they don't travel in time, so the entire portion of their travel takes place in space, so we say they travel at the "speed of light."

<comment><USER5> > Electromagnetic waves have no mass, they don't travel in time, so the entire > portion of their travel takes place in space, so we say they travel at the > "speed of light."

This part comfuses me. If they don't travel in time, how do they have a speed? Light is a type of electromagnetic wave right? And it takes many years to travel to us from a nearby star.

If we can measure or calculate the time it takes for light from some place to reach us, does that not imply traveling through time?

<comment><USER6> Think about a wave on a lake. It may appear to be moving in time. The water particles certainly move up and down. But if nothing is in it's "path" is the wave really moving? It's actually just there, the wave undulates and that creates the perception of motion, but really the thing you see moving is just a visual effect on the surface of a field the size of the entire lake. A field which is not moving at all.

Photons are similar. You see the peak of the wave moving around, but the wave itself is everywhere and eternal... until other forces get involved anyway.

<comment><USER7> This picture is incorrect: the electromagnetic wave has a mechanical momentum in the direction of its propagation, which means that something is moving in that direction.

<comment><USER6> Does it have momentum before we measure it? I thought momentum was a property of the collapse event, not a property of the wave?

<comment><USER7> According to the classical electrodynamics - it sure does. From the quantum mechanical point of view, it also does - in the sense that we can always measure it (i.e. it is an _observable_ ). The "property" in this case is not so much a particular outcome of such measurement as much as the expectation value; actually, I'm afraid that the use of the word "property" in this context can only lead to confusion as it effectively conflates several different things: the (quantum-mechanical) state, the observable, and the particular value observed.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> My understanding is that black hole mergers are not expected to have any optical-wavelength emissions, whereas neutron star mergers should have emissions across the electromagnetic spectrum. Is that distinction part of the excitement here?

<comment><USER9> The matter and light radiated away from black hole (BH) mergers will all come from the accretion discs of each BH. The accretion discs of the BH masses for which LIGO (and Virgo) is most sensitive will generally be fairly sparse, so the emissions will generally be fairly dim.

For a pair of similar-mass neutron stars (NS) that merge into a BH, you can treat region of spacetime close to the merger as being filled with extremely dense accretion discs. The density of matter leads very bright emissions, and the available geodesics produce radiation that will escape to infinity rather than being quickly absorbed close to the source (including within the dense matter around the newly formed BH as it settles into an accretion disc). The picture is slightly different where the NS masses are highly unequal.

The Max Planck Institute for Gravitational Physics at Potsdam Germany has done many numerical simulations of NS mergers. NASA's animated one [1] where the NS are of substantially different masses:

[https://www.youtube.com/watch?v=vw2sLcyV7Vc](https://www.youtube.com/watch?v=vw2sLcyV7Vc)

Systems of mass that are barbell-shaped (a pair of heavy masses connected via an arbitrarily thin bar; for orbiting stars and BHs we take the limit as the bar goes to zero volume and zero mass) will radiate gravitational waves when they are spun about an axis on an axis perpendicular to the bar. These spinning NSes will be radiating gravitational waves with increasing amplitude, and these radiated GWs remove angular momentum from the rotating system, allowing the NSes to move closer to each other. By the start of the video, the GW radiation being emitted should eventually be detectable by instruments at enormous distances; that radiation has allowed the two NSes to reach a critical proximity.

At this point the smaller NS disintegrates under tidal stress and its additional mass-energy-momentum that then begins falling onto the larger NS causes the larger NS to collapse into a BH. A large proportion of the smaller NS remains in the region near the new BH and is swept up into an accretion structure alsong with s small proportion of the larger NS that did not get trapped behind the horizon. The accretion structure is briefly very bright, especially in gamma rays, as the matter from the smaller neutron star self- collides until it is entrained into a dense disc. Those early gammas will be visible at enormous distances (e.g. we can pick up extragalactic NS mergers with sky-scanning instruments searching for gamma ray bursts [2]).

By comparison, the smaller of a pair of mass-mismatched black holes cannot disintegrate (everything is stuck within each BH's horizon), and the larger of the pair is likely to have a sparser accretion disc, so the amount of disc- disc collision will be relatively low. The reshaping of the accretion material around the merged BHs may be driven principally by the dynamical spacetime around the BH, with only occasional collisions. While such collisions can be arbitrarily energetic (at the point where their geodesics intersect bits of matter may be moving ultrarelativistically with respect to each other), there are unlikely to be enough such emissions to be reliably detectable at large distances.

\- --

[1] [https://arxiv.org/abs/1001.3074](https://arxiv.org/abs/1001.3074)

[2] There is a timing-coincidence argument about a short gamma ray burst detected in this way and a detection by LIGO & Virgo that is circulating around the rumour mill. Peter Coles blogged some detail [https://telescoper.wordpress.com/2017/08/23/ligo-leaks- and-n...](https://telescoper.wordpress.com/2017/08/23/ligo-leaks-and- ngc-4993/) This may be an NS-BH merger, in which the picture is again somewhat different, and depends on the density of the BH's accretion disc both in terms of its matter content and in terms of the momentum (e.g. if the NS and BH are counter-rotating, collisions will be more frequent and more energetic).

<comment><USER10> Thanks for this well informed comment. It links together some of the observing strategies and motivations really well.

</comment></comment></comment><comment><USER7> While I would have a trouble imagining a quantum of the space-time curvature (the graviton), it is not hard to see that the changes in the curvature could propagate in the form of waves. So, while yes, an experimental discovery of these waves is an important event in history of science, I am left curious as to whether it adds anything to our present understanding of Nature...

<comment><USER9> > trouble imagining ... the graviton

Let's start with a graviton as a gauge boson, mediating the gravitational interaction similarly to how the photon is a gauge boson mediating the electromagnetic interaction.

First, let's start with "gauge". We can use as an analogy an air pressure gauge, one that measures relative air pressures. Let's take it to a place with a standard pressure (say, in conditions which are effectively STP) and tune our pressure gauge so that it reads "0" at that air pressure. As we wander to and fro reading our calibrated gauge, we'll see pressures that are zero, positive or negative. If we climb a tall hill we'll see a negative reading. If the temperature drops we'll see a positive reading.

If we contrive things so that we can take a reading of a generalized pressure with our pressure gauge everywhere in the universe at all times, we can construct a (classical) gauge field. In deep space, the gauge will read strongly negative. At the bottom of the ocean, or deep in Jupiter's atmosphere, or at the core of the sun it will read strongly positive. In the early universe, it'll be strongly positive; in the far far distant future away from the black hole that will dominate our patch of de Sitter vacuum, it will read strongly negative. We'll get zero values in some places, like near the Earth's surface through a lot of Earth's history, or in the upper reaches of Jupiter's atmosphere through a lot of its history.

Our choice of "0" is not ideal, because "0" is only rarely the value at any point in our gauge field that permeates all of spacetime. Instead we should set "0" as the value in extragalactic space, because then "0"s will dominate the field (indeed it is possible that all readings will then be non-negative). In effect, when we set our "0" at STP we normalized the gauge field; when we decided instead to set our "0" in extragalactic space, we renormalized it. We could obtain an ideal renormalization if we could sample the whole of spacetime and find the lowest reading of our pressure gauge, but we can certainly get rid of practically all negative values by taking far fewer samples in regions where we think the lowest readings might be.

Once we have settled on a decent normalization, we could look at the propagation of nonzeros and study their statistics. If they follow the Bose- Einstein statistics, we'd call them "bosons". If they follow the Fermi-Dirac statistics, we'd call them "fermions". If they follow some other statistics, we'd assign them yet another name. (Our choice of generalized "air pressure" probably follows some odd statistics.)

Perturbative quantum gravity works something like this. [2]

We have a background spacetime with a metric; we have a gauge that measures the deviation from this metric. It'll be "0" at every point where the arrangement of stress-energy _exactly_ matches the metric, and nonzero elsewhere. We are interested in modelling the gravitational interaction as the arrangement of nonzero values in our field. Patterns of nonzeros around gravitationally interacting matter themselves evolve (under a suitable decomposition of spacetime into 3+1 space and time) and interact like (classical) waves (made up of many molecules), and upon some study we can determine that these waves in our gauge field form patterns that strongly suggest they have a rotational symmetry of two, which we expect on theoretical grounds too because the metric is a rank-2 tensor field so particles representing the (change in the) metric field should be spin 2.

Conveniently, in a quantum gauge group theory, a particle with spin 2 is attractive of a particle with the same charge and repulsive of a particle with the opposite charge. (Compare with spin 1, where same-charges repel and opposite-charges attract). [4] So we can identify the _nonzero_ numbers in our metric gauge field with gravitons. This is amenable to study with perturbation theory.

Unfortunately General Relativity is a non-linear theory and in our perturbatively quantized gravity, when you have a lot of high-energy gravitons they spawn more gravitons. We would want to apply Wilson's thinking on renormalization and reset our gauge to "0" in a cluster of these high-energy gravitons by finding some suitable ground value in the cluster. This is _extremely_ successful up to a point [1], but as the energies of the gravitons increases we have to take more measurements to find a suitable ground value, and eventually we have to take an infinite number of measurements to find one. This is what is meant when you read "gravity is perturbatively non- renormalizable".

There are, as you suggest with ("... space-time curvature ..."), other values related to the gravitational interaction that we can turn into a quantum field [3], but most suffer a highly similar fate: in some conditions we have to do an infinite amount of work to make our field values sensible _and_ match observables.

Finally the gauge field that we built on the metric is fully relativistic and generally covariant, so it works with any system of coordinates, choice of units, slicing of spacetime into 3+1, etc. that we want, up to diffeomorphisms (we have to remember that we chose a static background spacetime). So even though it gives us useless readings in some regions of a spacetime containing strong gravity, perturbative quantum gravity is a useful and standard tool. However, it is not considered a candidate for a fundamental theory (barring some unforessen advancement in renormalization theory) rather than an effective theory and moreover by implication it undercuts General Relativity's claim to be a fundamental theory too.

\- --

[1] Relativists tend to define "strong gravity" at this point, since we get correct results from renormalization at any energy lower than it. Strong gravity only appears very close to gravitational singularities (and in the case of black holes, that means well inside the horizon). If we are using the path-integral formalism then we'd find that we have "strong gravity" in this sense in every Feynman diagram containing at least one loop of gravitons.

[2] [https://arxiv.org/abs/gr-qc/0206071](https://arxiv.org/abs/gr-qc/0206071)

[3] [https://www.wikiwand.com/en/Canonical_quantum_gravity](https://www.wikiwand.com/en/Canonical_quantum_gravity)

[4] One might ask, "is there oppositely-gravitationally-charged matter anywhere"? It's an OK question, and people have discussed it seriously. Sabine Hossenfelder has touched on this a few times on her blog, including [http://backreaction.blogspot.com/2017/04/why-doesnt-anti- mat...](http://backreaction.blogspot.com/2017/04/why-doesnt-anti-matter-anti- gravitate.html) although while the photon has no electromagnetic charge, the graviton itself (in perturbative quantum gravity) has gravitational charge (this reflects the non-linearity of General Relativity).

</comment></comment><comment><USER11> The idea of two neutron stars colliding gives me shivers. To see something like that would be mind boggling..

</comment><comment><USER12> "We are working hard to assure that the candidates are valid gravitational- wave events, and it will require time to establish the level of confidence needed to bring any results to the scientific community and the greater public. We will let you know as soon we have information ready to share."

\-- Statement from Ligo [http://www.ligo.org/news/index.php#O2end](http://www.ligo.org/news/index.php#O2end)

"This did not, in fact, blow my sox off."

\-- Astronomer Peter Yoachim [https://twitter.com/PeterYoachim/status/901175225819176961](https://twitter.com/PeterYoachim/status/901175225819176961)

</comment><comment><USER13> I wonder if the gravitational wave would be physically noticeable if one were near the event.

</comment><comment><USER14> > swell ... wave

Someone had fun writing that headline.

<comment><USER15> No doubt they will "crest" just before the announcement :-). And opinions will "undulate" over whether or not they are valid results.

Perhaps we just need more drama in science classes to be more inclusive, "Will this acid change the Ph level of the solution? Or will its buffering protect it? Find out after the break ..."

<comment><USER16> You will never believe what happens to this falling feather in a vacuum!

</comment></comment></comment><comment><USER17> As a non-science person who occasionally watches PBS documentaries, could this have any implications for Hawking radiation?

<comment><USER18> Potentially this neutron - neutron star merger or ones that are detected in the future will result in the formation of a black hole and in that event we may get more data to help refine models of black holes and those refined models may give us insight into the mechanics of black hole radiation - but it would be a bit indirect and we'd have to get lucky. Also Hawking radiation is "very slow" and may never be measured directly in interstellar black holes

</comment></comment><comment><USER19> Normally, the scientific community is pretty careful about not revealing results before they're fully baked (press notwithstanding). Seeing how that control has broken down for this incident, is it correct to infer that astronomers have pretty much lost their minds over the possibility of capturing a neutron star merger?

Edit: calm down, people, I'm excited too.

<comment><USER20> LIGO didn't announce because they want to be sure. But one of the most valuable aspects of LIGO is as a trigger for optical astronomy of transient, time-sensitive events. So, when they see something, they have to tell a wider astronomical community---the other observers who look in non-gravitational channels---though they might not be ready to make the information public. Each observatory comprises hundreds of people. Suddenly your message isn't contained to just the hundreds of people loyal to LIGO, but to thousands of people. Moreover, these observatories tend to be publicly owned and funded, and transparency and modern open-science practices often mean live updating of the status of these observatories. Finally, there are the scientists who have nothing to do with this physics, but fought hard for some observation time and had their scheduled observations interrupted for something high-priority. These people can infer (or are often told directly) why their allocation was preempted.

</comment></comment><comment><USER21> Someday could we ride these waves to Interstellar travel?

<comment><USER22> No. The effect of the waves is so small that it required a gargantuan project to detect the strongest events. The LIGO detector is possibly the most sensitive instrument of any kind humans have made.</comment></comment>


Ask HN: College degree required? - <AUTHOR>

I've been browsing job boards for a research project and I come across time and again a requirement for a college degree to land the position.  Yet when I read the description of what the employer wants of its employee it is not clear that one (the degree) has anything to do with the other (the job).
For instance, a job posting will typically ask for a candidate (aside from particular work skills) to:
-have strong written and communication/verbal skills -be able work independently and effectively multitask in a fast paced environment with minimal supervision -professional, extremely well organized, thorough, and detail-oriented -be proactive (aka go-getter) -have an analytical/sales/managerial aptitude
While a degree may signal some of the above criteria it doesn't follow that the college experience adds to what the student already is capable of by the time he reaches graduation.  If anything college may erode his value to employers by allowing him to skate by on a minimal course schedule whose work product is corroded by grade inflation.  Students are taught that a minimal effort is all that is needed to be rewarded with a decent grade.  In the real world that is decidedly NOT true and subpar performance is the quickest way to get you fired (outside of government).
It could be argued that what once was something special and where a college degree signified something important is no longer the case.  For all intents and purposes a college degree doesn't tell you anything about what kind of future employee that person is going to be and if he'll bring the abilities and qualities you want in the job.
What is confounding then is why society has placed an ever greater importance on college when its value and worth has declined.  College used to be valuable when access to information was scarce and communicating with and meeting new people was hard to do but the Internet has liberated us.  Now we are overwhelmed with content and spend a good portion of our time on social networks and user forums.  Therefore, the inherent "worth" of college has fallen precipitously even though employers paradoxically have required ever more of us to have such credentials.
My answer is that a college degree had conferred unto its recipient an "illusion of competence" whereby employers would take as a given that the graduate was an above average worker and place his resume at the top of the pile, so it "paid" to endure four years of coursework and thousands of dollars in tuition costs to obtain that signal.
As students put in the minimal effort necessary to graduate employers find out that graduates are nothing special and the marginal value of a degree declines.  Therefore, the illusion that had held up throughout the 20th century has now been shattered in the 21st leaving employers frustrated with underwhelming candidates and graduates despondent over minimal job prospects and burdened with significant student loans. <comment><USER2> What is the question?

The issue I have with the school-is-useless line of thinking is that the underlying assumption is that the dropout is a Good Will Hunting autodidact whose genius is confined by the educational system, and is shrewd enough to realize his educational ROI would be higher if he taught himself.

In reality, dropouts and people who don't go to school are often just people too stupid, lazy or psychologically messed up to accomplish anything.

The brilliant self-taught definitely exist, but in my experience they almost never spend time posting to online forums about how school is a waste of time.

This is coming from someone who dropped out thinking he was a genius autodidact and then went back to school when he realized he was just being stupid and lazy.

<comment><USER3> The poster's main issue is not that people who didn't go to school are generally better than people who did, but that people who went to school aren't generally better than people who didn't. I.e., because of grade inflation and the trend to not make students do much work, people who went to school are just as likely to be lazy and stupid as people who didn't. A degree isn't a signal for anything anymore.

</comment></comment><comment><USER4> You seem to be commiting the common geek error of assuming that people are generally logical, and that they agree with your basic values. They aren't, and they don't.

College degrees are demanded, even when irrelevent to the job at hand, because that is a traditional way of getting better people. Most employers will accept and follow this as a cultural best practice because they don't have any reason to spend the time questioning it, and it is so engrained in our society that it is unlikely to get pushed aside by logic anyway. "College is useful" is as obvious to most people as is "1 + 1 = 2". If I show you a proof that the latter is not true, you will assume that the proof is flawed, not your belief. Most people are like that about most of their beliefs.

I do absolutely agree with you that college degrees are overrated most of the time. I think that you're overoptimistic about public illusions being shattered and all that though. The college industry is enormous, both economically and culturally, and will not go down easily.

</comment><comment><USER5> If the position is a entry level position, then there are two options I see for hiring the person: 1) Based it on his/her education and courses. If they went to the same school as I did, then I am able to ask about certain courses/instructors. 2) Based on any projects they have completed on their own (open source/freelance/personal)

You have to have one or the other. If you don't have either, then I'm going to choose someone else.

For anything over entry level, I'm looking for work experience.

To answer your question (which is why do companies ask for the a degree) is that it is used as a filter for them. If you have ever done any hiring, you will know that you can be flooded by unqualified candidates and its a major time sink.

</comment><comment><USER6> If you're looking for people who like to learn and grow, maybe it makes sense to for them at a school. Just sayin'

</comment><comment><USER7> The only question mark is in the title...

Apply anyway. Most companies will take years of experience over a degree (or at least some schooling).

</comment><comment><USER8> what question are you asking

</comment><comment><USER9> School requires you to show up 9-5 monday - friday, take initiative (homework, projects, etc), and make an effort (debatable). These skills transfer well into working for any size of company. Self-educated people are great workers, but they might tend to be lone-wolves, rockstar, or wizard programmers who work well by themselves but can't hack it in a group of (college educated) programmers. Software engineering is all about communication, there are tons of nerds, but nerds who can communicate are a valuable commodity.</comment>


Software for moral enhancement - <AUTHOR> http://kajsotala.fi/2016/09/software-for-moral-enhancement/ <comment><USER2> The trouble I see is that we all have different versions of morality. I, for example, believe that tobacco is a positive social good, cigarettes are a social ill and anti-smoking campaigners are utterly despicable; that is the precise opposite of what many people believe: my version of a morality-guiding app would encourage folks to do things that others abhor (and their version would encourage folks to do things I abhor) and discourage folks from doing things they approve of (&c.).

<comment><USER3> This isn't necessarily a problem--open source apps could be made configurable to aid people in controlling their behavior toward _almost any_ goal, with focus on goals that fall into forms of morality. The most basic forms of this are apps that just provide data--what decisions people make with that data are up to them (although there is an underlying problem of how filtering and presenting data shapes people's decisions). But in a larger sense, a behavioral modification tool could be used equally by pro-smoking and anti- smoking advocates.

It's a bit off-topic, but I'm curious about your pro-smoking stance. Could you give your reasoning? I have some reasons of my own why I'm not _as_ anti- smoking as most people, but I wouldn't go so far as to say tobacco is a social good, and I'm a bit confused by you also saying cigarettes are a social ill.

<comment><USER4> With all these competing, possibly contradictory moralities, how do you see this reliably making the world a better place?

PS. Understandably, we might not all even agree on what a better world would look like.

<comment><USER3> Personally, I think empowering individuals to control their own lives is an inherent good, so within my personal definition of morality, it makes the world a better place from the start.

However, I'm well aware of the tradeoffs--en masse, individuals will collectively make some pretty stupid decisions (i.e. continuing the destruction of the planet) that might make the world a worse place. We need more than programs that allow personal choice to make the world better--we need education, social change, better laws, funding, etc.

The fact is, there's nobody running the show who could coordinate all of this to ensure that programs that enable personal choice are a net positive. We can only do our part and hope that people in other areas of expertise do their part.

</comment></comment></comment></comment><comment><USER5> I like this as a thought provoking piece. The examples seem less than stellar though. It begs the interesting point of the moral value of the Internet.</comment>


Ask HN: Are There Any Good Alternatives to Google Forms for Surveys? - <AUTHOR>

While I applaud Google to make such a good survey system (with Google Forms), ideally I would like to avoid going through their services. Are there any survey systems out there that do not suck ? And no, SurveyMonkey does not cut it, and I'd like a modern interface too when I create my surveys (Google Forms makes it very easy). Any Suggestion ? <comment><USER2> I am biased, but my team and I are developing a new platform for the reasons you stated above. Check it out at [http://microhero.com/surveys](http://microhero.com/surveys)

Any feedback is much appreciated!

</comment><comment><USER3> My friend likes Zoho Web Form Builder:

[https://www.zoho.com/creator/online-form- builder/](https://www.zoho.com/creator/online-form-builder/)

Also depending on how much you use [https://www.cognitoforms.com/](https://www.cognitoforms.com/) has a free plan.

Here are some more:

[http://www.typeform.com](http://www.typeform.com)

[http://www.wufoo.com](http://www.wufoo.com)

[https://kwiksurveys.com](https://kwiksurveys.com)

<comment><AUTHOR> Oh, I did not know Zoho did Forms. They look like a good alternative indeed! Thank you!</comment></comment>


A Tale of Query Optimization - <AUTHOR> https://parallelthoughts.xyz/2019/05/a-tale-of-query-optimization/ <comment><USER2> I can definitely relate to the struggle of debugging slow SQL queries. I've been optimizing Postgres for the last three years for my day job. There's nothing like spending hours trying to determine why a query is slow, only to discover some bizarre incantation makes the problem go away.

I had one case where adding OFFSET 0 to a query, which did not change the semantics of the query, made the query ~10x faster. It turned out that due to a reference to a large column being inlined in a dozen different places, Postgres was detoasting[0] that column a dozen times more than it needed to. Adding the OFFSET 0 prevented the subquery from being inlined and made it so Postgres only detoasted the column once.

In general, I think optimizing a database is way harder than it needs to be. A lot of it is black magic and it requires a ton of expertise that many teams don't have. Even though I've been doing it for years, I still struggle with figuring out what indexes I should create to make a query fast.

I recently started a startup in an attempt to make optimizing databases easier. I built a Postgres extension that records information about queries as they run. It's basically the same information you would get for EXPLAIN ANALYZE, but you get it for every query at query time. Then, using this information, I'm able to make specific recommendations about how you should optimize your database. I can give advice on how you should structure your schema and what indexes you should create in order to maximize the performance of your queries. This way, even teams that don't have much in-house Postgres experience won't have to spend hours or even days trying to figure out how to make their queries faster.

If this sounds interesting to you, I would be happy to chat. You can reach out to me at email@example.com. It's my goal to use the expertise I've built up over the last couple of years to make scaling Postgres a truly painless experience.

[0] [https://malisper.me/postgres-toast/](https://malisper.me/postgres-toast/)

<comment><USER3> Isn't there some irony in fact that SQL was meant to abstract away the "how" of getting records? If you need to optimize through implementation specific quirks, then wouldn't a more direct api be preferable?

<comment><USER2> I have a number of thoughts on this. A common claim I've heard is that query optimizers are better at producing efficient query plans than all but the best programmers. I can't find any concrete studies on this, but it's plausible. Databases are great at handling slight variations of a query and a data set. I've had times where I thought Postgres was going to execute a query one way, but it wound up executing it in a different way that happened to be more efficient.

Given that query optimizers are usually better than humans, it makes sense to use them for the general case. The question then becomes what happens for the exceptional cases. Cases where the database chooses a bad plan that a human could tell is obviously going to be slow. For these cases I think it's up to the database implementors to decide what to do.

Some databases have support for query hints, a way to force your DB to do what you want. The Postgres team has avoided implementing query hints for two reasons:

\- In the long term queries that make use of query hints will be suboptimal. (Personally I don't buy this. I've ran across cases in Postgres where the planner will only give you a very inefficient query. I would rather use query hints to get the query to work for the time being than never have a working query at all.) \- Implementing query hints would overly complicate the planner. I find this to be pretty reasonable.

As usual, having a query optimizer is a tradeoff. It seems to me like in most cases it's better to have one than to not.

<comment><USER4> Even if you assume that the human programmer is better than the query optimizer, which I think you'll agree is assuming rather a lot, the query optimizer still has important advantages. It can react to changing conditions.

The optimal query plan for a given query can change as data is added or deleted. Moreover, there can be multiple optimal query plans at the same time for "the same query" (or at least the same parameterized SQL statement).

I'm not suggesting that optimizers don't produce bad plans that have to be worked around. My point is that comparing the optimizer to an expert human only makes sense as a thought experiment. Most bad query plans are caused by bad selectivity estimates/stats, not bad cost models.

<comment><USER3> This is starting to sound a lot like the discussions around optimizing C compilers. Only in that case you can drop into assembly if necessary.

</comment></comment></comment></comment></comment><comment><USER5> Having hit this sort of problem too many times, I now recommend a DNS style FQDN in all my click-stream data.

The domain stored as ".com.google.mail" etc.

This breaks down a lot of these sort of queries with a simple enough '.com.google%' filter and which gets evaluated internally even on min-max zones.

Gets turned into ">= '.com.google.' and <= '.com.google.\u0ff'" in byte comparisons.

Even then it doesn't really help with the array syntax - because the condition doesn't get pushed there.

<comment><USER6> Arguably domain names written the way they usually are has been wrong since the beginning, for obvious reasons.

</comment></comment><comment><USER7> It isn't clear to me why so few of these optimization articles talk about temporary tables. They are my go-to solution for many situations.

Temporary tables are great because they break the problem down into smaller pieces. They make the query much more understandable. They are way easier to debug. They are way less sensitive to failing because the query optimizer decides to do things differently because table statistics have changed. They will often eliminate the need for exotic syntax.

You also have the opportunity to add secondary indexes to these tables.

When you create a temporary table, you are essentially doing the work of the optimizer yourself. That's perfectly ok if you know things about the data that the optimizer doesn't or hasn't successfully figured out on its own.

They're not for every situation; they only work well when the intermediate tables are small and fit in memory. But that is very frequently the case.

<comment><USER8> One reason you may not see CTE's or temp tables included in these articles is because their specifics are tightly coupled to the schema of your DB. It's hard to write a generic example in Stack Overflow that would work for someone else.

But like you said: if you know the DB schema then temp tables make SQL easier to read, write, and reason about.

<comment><USER9> I'm not sure what you mean by this - it is possible to write any subquery as a CTE

<comment><USER10> Might refer to MySQL. It didn't support any CTEs until version 8.0, from April, 2018.

</comment></comment></comment></comment><comment><USER11> Optimizing a large database (billions of rows) has been the bane of my existence over the last couple months. But, when you finally get that 10x / 100x performance improvement, it's all worth it... until the next thing.

<comment><USER12> I've recently started working on a service with a MySQL database of a few billion rows. I'm used to developing on lower KVs like RocksDB or something.

Any broad, general advice? Cool tricks you've learned? Bulk deletes and a relatively high continuous write rate are killing me right now.

I kinda hate how manual batching deletes and then having to verify them takes.

<comment><USER13> Deleting lots of rows is perhaps the biggest weakness you are running into. What I do for some cases is "LIMIT 4000", sleep for 1 second, and repeat until no rows affected (tweak the numbers for your situation obviously) (usually this is for a periodic expiry/cleanup task).

<comment><USER14> It's probably worth mentioning that at least in MySQL, this is dangerous if using statement based replication.

[https://dev.mysql.com/doc/refman/8.0/en/replication-sbr- rbr....](https://dev.mysql.com/doc/refman/8.0/en/replication-sbr- rbr.html#replication-sbr-rbr-sbr-disadvantages) [https://dev.mysql.com/doc/refman/8.0/en/replication- features...](https://dev.mysql.com/doc/refman/8.0/en/replication-features- limit.html)

<comment><USER13> Great point.

I happen to use row-format mixed (default on RDS), so that can make it OK. Also, often I have some other related stuff to cleanup, so the pattern is more like: SELECT id, stuff FROM table WHERE expired LIMIT 1000; do cleanup; if all successful: DELETE FROM table WHERE id IN (id, id, ...); so the delete is deterministic for statement-based replication.

</comment></comment></comment></comment></comment><comment><USER15> Query optimisation is such an underated skill, imo it should be prioritised as a core skill for any backend developer

<comment><USER16> It seems there are a lot of new "lead" developers out their that don't even understand rdbms, much less how to optimize them. A lot these days seem to be coming into the field thinking that SQL is old hat and nosql is the future.

You should see this newly rebuilt e-commerce system I saw recently all built on mongo. The hoops jumped through and special functions created and work- arounds were just astounding. They basically had to build their own rdbms layer in node to interface with their mongo datastore.

All this because the new hotshot developer convinced management that this was the only way to solve the DB bottlenecks they were having.

...because "MySQL will just never be able to query the product table in less than a few seconds since it has 'millions' of rows".

An entire system rewrite to solve a problem that probably could have been solved with a few indexes. And now they are going to have to implement some things in MySQL again, because all the node.js code is getting overly complicated.

<comment><USER17> > "lead" developers out their that don't even understand rdbms

Just lead ? I am interviewing "Full stack rockstars" who haven't who think SQL is unfit for every purpose. Frustrating, but a good filter.

</comment></comment></comment><comment><USER18> This article could be improved by including the Postgres EXPLAIN output so the reader could follow along the deductive process.

<comment><USER19> Im confused - did they not EXPLAIN their query? That's the first step in ANY slow query debugging!

I kept waiting to see EXPLAIN output so I could say "oh, there's your table scan" and nothing ever showed up.

</comment></comment><comment><USER20> > Then, for a small subset of data, I manually verified that all results were > correct.

This could have been automated with a full join. with A as queryA, B as queryB select A full join B on all applicable fields where A is null or B is null. Alternatively, you could also do: A except B union all B except A. Either way you get the rows in A that aren't in B and rows in B that aren't in A -- no manual check needed, and you could have run that on the entire set instead of or in addition to counting rows.

</comment><comment><USER21> I'm a bit confused why they didn't use EXPLAIN ANALYZE at all in this case. Debugging performance issues in Postgres without it makes it unnecessarily hard. I'm not entirely sure without being able to try it myself, but I suspect it should have immediately identified in sufficient detail where most of the time in that query was spent .

</comment><comment><USER22> The frustrating thing about such optimizations is how badly they fit with the rest of the software development and testing workflow.

Ideally, you want to write a test that prevents a performance regression, but then you need to set up a test database with the right data characteristics (but not real data, for privacy concerns), need the hardware to handle that, need to maintain that database through future schema changes and so on.

It would also be nice to have some kind of mechanism that can tell you that the optimization is no longer necessary when you switch to a new DB version with an improved optimizer.

Most organizations don't have the engineering capabilities or capacities to provide this kind of tooling and infrastructure, so the benefits of CI/CD often don't apply to such changes.

</comment><comment><USER23>
        Then, for a small subset of data, I manually verified that all results were correct.

Where's your test suite man? :)

</comment><comment><USER24> I wonder if a CTE with both sets and `intersect` between them would be more efficient than &&.

<comment><USER11> > [https://www.postgresql.org/docs/11/queries- > with.html](https://www.postgresql.org/docs/11/queries-with.html) The > optional RECURSIVE modifier changes WITH from a mere syntactic convenience > into a feature that accomplishes things not otherwise possible in standard > SQL.

Am I incorrect in thinking that "standard" CTEs don't have an impact on performance. When I saw this in the article and how it had no performance improvements it reaffirmed my assumption.

<comment><USER25> PostgreSQL materializes all CTEs so it has a big impact on performance. It's a hack that works around lack of optimizer hints in PostgreSQL to force the execution plan you want. Only in PostgreSQL 12 did they change this: [https://www.postgresql.org/docs/devel/release-12.html#id-1.1...](https://www.postgresql.org/docs/devel/release-12.html#id-126.96.36.199.5.3.4)

<comment><USER11> > > [https://dba.stackexchange.com/a/13117](https://dba.stackexchange.com/a/13117) > Also, a CTE should never be used for performance. You will almost never > speed things up by using a CTE, because, again, it's just a disposable view.

<comment><USER25> Information about SQL in general should not be confused with information about specific RDBMS.

That SE answer is about MS SQL Server.

What I wrote was specific to PostgreSQL which does a specific weird thing using CTEs where they are materialized.

In SQL Server, CTEs are not materialized and are just used to organize the query, to make it more readable.

If I wrote about PostgreSQL heap organized tables, would you object and say I'm wrong because MS SQL Server tables are index-organized, clustered on the primary key?

<comment><USER26> Good answer, but fwiw SQL Server has heaps too :p

</comment></comment></comment></comment></comment></comment><comment><USER27> Unbelievable that this is even getting upvotes. Anyone with a basic understanding of relational databases could see that nested o(n) ilike filter was the problem. I kept reading in the hopes that maybe something actually interesting was going on, besides ineptitude. Disappointing.

When you have a huge list like that in a filter, the answer is to treat it like a table and perform a join. Then the query planner can optimize the ordering of the loops. But this is just mind-boggling ignorance dressed up as a "debugging horror story".

<comment><USER28> Get over yourself.

This is a well-written article in which the author lays out their thought process for optimizing a slow SQL query. The article certainly provides more value to someone who may not be an RDBMS expert than your comment does.

</comment></comment><comment><USER29> I think the root of your issue is the %text% leftsided % is horribly slow.

<comment><USER30> Addressed in the article:

> The pattern matching query itself is taking only 5 seconds independently. > Matching millions of unique URLs is clearly not a problem.

<comment><USER10> As _thomaswang_ says, that assumes that the query planner recognizes that it can hoist that query part out of the loop ([https://en.wikipedia.org/wiki/Loop- invariant_code_motion](https://en.wikipedia.org/wiki/Loop- invariant_code_motion))

If it doesn't, and runs it again and again, time will add up (possibly by less than 5 seconds for each time it gets run because the data could stay in memory)

That theory was only disproved at " _Move the sub-query into a CTE_ " section.

Also, I'm not familiar with Postgres, so I don't know whether it could significantly affect timing, but the query benchmarked at 5231.765 ms isn't identical to that in the larger query. The latter has _::text[]_ added.

I also would try and replace

        AND r_time > to_timestamp(1547585600)
        AND r_time < to_timestamp(1549177599)

by

       AND r_time BETWEEN to_timestamp(1547585600+1) AND timestamp(1549177599-1)

because the query optimizer might not detect that in this query.

<comment><AUTHOR> BETWEEN is soemthing that is on the "Don't do this" page of Postgres. :)

[https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use...](https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_BETWEEN_.28especially_with_timestamps.29)

the ::text[] addition had no impact on perf. I actually constructed this post from my notes at the time so might missed have a few things here and there.

The bit about query planner recognizing things is exactly what the challenge is to be honest. Like I said at the end, the good mental model of SQL helps a lot, and mine is decent-ish I hope but can still learn a lot. :)

<comment><USER10> That's why I added the _+1_ and _-1_ fragments :-) I thought those computed the smallest larger/largest smaller time stamp.

I didn't realize that time stamps weren't integers in PostgreSQL, though, so that is a bug.

Are you sure your query, which uses an interval that is open at both ends, is correct?

</comment></comment></comment></comment></comment><comment><USER31> I find this a little bit funny,

> I was extremely suspicious of the EXISTS optimization, since it changes the > logic to exit early.

Does this this mean that if one has found an answer, it is somehow sensible thing not to return the answer right away?

<comment><USER32> With a database, that depends if you want "all the answers" or "any answer". Exists is the latter.

</comment></comment><comment><USER33> Thanks - good investigation. We're replatforming a legacy app from Oracle to Postgres in the near future and I'm sure we'll see stuff just like this. Definitely saving so I can refer back.

</comment><comment><USER34> This is pretty great. I've been meaning to find a good story and helpful guidelines for making queries faster as I taught students about SQL.

</comment><comment><USER35> A funny bit:

>I started googling for a way to do set-intersection in Postgres without using the && and it wasn't going anywhere.

That's what SQL was made for! It's called 'join'. Perhaps is wasn't the best idea to store session's urls in an array and not in a table. (It's called normalization.)</comment>


Ask HN: What would you do the next time internet goes out? - <AUTHOR> <comment><USER2> lament not working on mesh networks and praying my failing will be cut short by the internet getting back up soon.

in other words, the same as everyone here.</comment>


Libclub: Lightweight Peer-to-Peer networking engine for real time applications - <AUTHOR> https://github.com/inetic/libclub <comment><USER2> Thanks for this publication. One question, what the consensus method is implemented in your engine?</comment>


Selling What They Preach - <AUTHOR> https://www.theatlantic.com/entertainment/archive/2017/03/selling-what-they-preach/519156/?single_page=true <comment><USER2> I'm normally a big fan of The Atlantic's longform pieces, but this one really missed the mark with me. It certainly doesn't justify its length, and it only gave passing mention to the simple fact that companies sell what people want to buy.

People seem to want hope right now, and there is endless hand-wringing about how we can fight the rising nationalistic nastiness in the world. Rebuild stronger local community ties? Be more mindful of small acts of kindness? People have their own ideas, but that's what they want to buy, so that's what companies are using in their ads.

I think the author got the causal relationship backwards. Marketers aren't arbiters of what people think is good or right. They just do their best to express those values as they see them in the current zeitgeist.

<comment><USER3> > companies sell what people want to buy

That doesn't really sum it up. There are lots of things people would love to buy, but there are strong enough disincentives to marketing them that they don't get sold. A simple example is raw milk, but there are myriad reasons why a company would not sell a product, and often times those reasons line up across an entire industry.

In marketing specifically, there are cultural practices that make certain products easier to market than others. It's not that those products are unmarketable, or that people don't want to buy them, the tools to do it just don't exist.

For example, I could invent a marketing tool for marketing some of these products, adding 0.01% to the typical marketing professional's revenue... but learning that tool isn't really worth their time. They can manage maybe a half dozen tools max. so each of those needs to provide tens of thousands of revenue per year. If we start from a presumption that marketing will be done by people earning $50-200k per year, only tools that fit into that lifestyle can exist.

Of course, there could be a market of part time freelance marketers capturing those smaller contracts, but there's no incentive for the existing marketing industry to build the tools they'd need.

In fact there's disincentives: having a big step-up in the cost to get started makes it harder for newcomers, which decreases competition for professional marketers, which makes it easier for them to keep their contracts. And for the tool builders (software companies, etc) it is just more manageable to focus on one or two tools that capture the lions share of the market. Product teams get bogged down in trying to build too much, so management has an incentive to keep it simple.

A startup could capture that value, but as startups grow there is pressure for them to go after the big fish clients, and again that causes them to turn away from the long tail.

Open source will get to it, but the open source software development world is still in its infancy compared to the IP-based software development world. And OSS is currently in a crisis due to software shifting from downloadable executables to software-as-a-service. We'll pull out of that, but until then the end-user OSS market is languishing.</comment></comment>


Ask HN: I Just Lost All My Karma On HN. What Happened? - <AUTHOR>

All the Karma i collected (which was 13 btw, not a big deal, just wondering why it was resetted to 1) just "vanished" together with my email and personal profile data. Any idea about what happened? Thanks<p>Update: Just noticed that all my submissions and comments were deleted as well. <comment><USER2> Could your comments have been on posts later marked dead? I see one comment from you still available from 90 days ago (I have showdead on so I'd be able to see if you were hellbanned).

I'd wager it's either that many of your comments were on a thread voted down / flagged dead, or it's simply a temporary bug as others have guessed.

<comment><AUTHOR> Hellbann wouldn't make any sense. I also doubt the voted down option for threads/comments. As many suggested i think it's a bug or anything similar. If doesn't happen again i simply won't bother, otherwise i'll investigate.

Most of the time i'm a ghost reader on HN and i don't really pretend to understand all the dynamics on this community. This is why i asked pubblicly what the cause could be as maybe there was some very obvious reason i just wasn't aware of.

Thanks a lot for your help, much appreciated :)

</comment></comment><comment><USER3> ... I still don't know what HN Karma is for or how it's gained.

<comment><USER4> You get karma for every time someone upvotes you.

When you have 500 or so, you get the privilege of downvoting.

<comment><USER5> It's more of a curse than a privilege, I find myself hitting downvote now instead of replying with a well thought out answer to tell them why they are wrong.

The downvote should require a comment.

</comment></comment></comment><comment><USER6> It's probably just a temporary bug.

</comment><comment><USER7> You are reborn in the next life</comment>


Lighthouse AI: smart security camera with 3D sensing, computer vision and NLU - <AUTHOR> http://www.businessinsider.com/lighthouse-security-ai-camera-2018-2 <comment><USER2> Additional insight from the founders can be found here: [https://blog.light.house/home/2017/5/11/introducing- lighthou...](https://blog.light.house/home/2017/5/11/introducing-lighthouse)</comment>