Nano 5.0 - <AUTHOR> https://lists.gnu.org/archive/html/info-gnu/2020-07/msg00010.html <comment><USER2> Thank you for you work on Nano! I'll never understand the elitism around text editors in the Linx world but am glad a user-friendly option like Nano exists.

<comment><USER3> Indeed. I kinda wish they would go the rest of the way and implement the standard keyboard commands that every other editor in the world (apart from vi/emacs) uses though.

Ctrl+c, ctrl+v, ctrl+x, ctrl+a, shift+arrow/home/end/pgup/pgdn/etc. etc to select text. It would completely eliminate the learning curve to edit text on linux, which seems like it could only be a good thing.

<comment><USER4> vim works quite well with standard keyboards. Put this in your .vimrc:

set nocompatible

behave mswin

<comment><USER5> Or use `evim`.

Btw, 'set nocompatible' is never necessary in a vimrc file. When Vim detects a user vimrc file, it automatically sets nocompatible.

</comment></comment></comment></comment><comment><USER6> A bit of a plug, but micro has been a nice alternative. Mouse support is enabled by default, ctrl+c/v works as expected, you only need to remember ctrl+e for other commands, as it opens a prompt where you can just type save/quit/etc.

Not in the repo though, which can be a no deal for many.

<comment><USER7> Oh right, it uses the operating system clipboard? That's kinda handy.

In other terminal editors I sometimes have to do a slight pause, to work out whether I'm using the editor-in-built yank chain, or pasting from operating system clipboard using the not-as-standard-as-Ctrl-V terminal emulator binding, Ctrl-Shift-V.

<comment><USER8> And then there's Vim which, depending on how it was built for the respective distro, doesn't seem to accept any way to copy or paste text.

As a daily Vim user I find this to be a huge mistake in terms of user friendliness; Vim provides ways to set the clipboard as default register and other quality of life improvements, but until then a beginner has to figure out some of the weirdest keybindings to make it work, or even recompile the program because that specific distro didn't include xclip support for some reason. It is infuriating.

<comment><USER9> I understand your practical frustration, but it can be very tricky with terminal apps to reliably interact with the X display. It's basically a completely foreign interface that requires the environment to be setup properly.

But in general I just install gvim (even if I only use Vim in the terminal) since it normally means that it'll have proper X support.

<comment><USER10> Middle click paste should "just work" and be transparent to terminal apps. It's not encouraging that vim is willing and able to break this in its default settings.

<comment><USER9> I could be misremembering but I believe that middle-click paste just makes the terminal feed the contents of the X selection to stdin of the running process. I could be wrong (because there's so much arcane knowledge in terminal emulators that I won't even pretend to have scratched the surface) but I'm not sure if Vi(m) can even tell where the input comes from. Well, maybe it could by using the speed at which the characters come in I suppose, although even that might end up being unreliable over remote connections and the like.

In my humble opinion expecting good X integration for a terminal application is bound to result in frustration. I use vim in the terminal but I also never use the mouse and I configure Vim not to interact with the X selection unless I specifically instruct it to using "*. If you want better integration with the graphical environment why not just use gvim? That's what it's for.

<comment><USER11> > middle-click paste just makes the terminal feed the contents of the X > selection to stdin of the running process.

Recent versions of vim hijack this by default (`set mouse=a`). This makes the middle click paste the default vim register instead of the X selection. To paste from X, you must shift+click.

Personally, I can't understand why anyone would want this, but anyway. The best part is that you can't easily disable this globally, i.e. by putting `set mouse=` in /etc/vim/vimrc. Because if a given user does not have a ~/.vimrc of their own, then defaults.vim gets loaded _after_ /etc/vim, overriding anything set there.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Thanks to nano team for great work. It's my favorite terminal editor for last 10 years. Too bad many systems has vi as default editor

Most liked features: kut lines, page up/down, easy search, syntax highlighting, editing long lines

</comment><comment><USER13> When I was in high school and used my first Linux machine (a Raspberry Pi), Nano was always the recommended editor when I had to edit config files while not knowing what I was actually doing. Nano is great for those use cases for sure.

But as I've spent more and more time with Unix systems, Vim was become by go to choice. The customizability and power of Vim is great, but there is a steep learning curve if you've never used a Unix machine before and you're just trying to get WiFi working.

With that said, use what you like, all text editors do the same job in the end: edit text.

<comment><USER14> > When I was in high school and used my first Linux machine (a Raspberry > Pi)...

Thanks for making me feel old! My first experience with editing text on a *nix system was vi on my older borther's AT&T SVR4 workstation in the 1980s when I was junior high (my apologies if that reference makes anyone else feel old!).

<comment><USER11> Actually you just made me feel really young! My first taste of Unix was a dialup shell account in the early 90's. IIRC, vi was the only editor available.

</comment></comment></comment><comment><USER15> I wonder which editor the Nano devs use. It would make for an ironic anecdote if any used Vim or Emacs.

Not to say they couldn't edit their own source in their own editor, I'm sure it's quite up to the task. But I am curious to know.

<comment><USER16> I would imagine they use a full feature IDE, not a minimalist editor.

It's like expecting the maintainers of notepad.exe to use notepad itself during development and not Visual Studio.

</comment></comment><comment><USER17> Nano has always been my go to editor. It comes natively with most install and is very easy to use. It has all the shortcut listed on page so it is quite user friendly out of the box.

After learning vim for a bit, I find nano quite superior for my use case mainly editing files inside terminal on remote servers.

<comment><USER18> > Nano has always been my go to editor. It comes natively with most install

While vi comes with all installs

> is very easy to use. It has all the shortcut listed on page so it is quite > user friendly out of the box

Tell that to the users that don't know what ^ stands for

> After learning vim for a bit, I find nano quite superior for my use case > mainly editing files inside terminal on remote servers.

I just don't understand how this is possible, except if you're doing very basic edits (even in that case...)

<comment><USER19> >I just don't understand how this is possible

I use nano as my daily driver, and here is how:

\- I have a nanorc I copy around but am quite comfortable without. Here it is. You can use it under WTFPL-1. Feel free to fork it, but I'm not accepting pull requests.

      set multibuffer
      set nohelp
      # Defaults for older versions
      set nowrap
      set smooth
      set morespace
      unset boldtext

\- I use ^K and ^U to cut and paste lines of text.

\- If I need to edit multiple files, I use ^R

\- If I need to find and/or replace, I use ^W (find), optionally with ^R (replace) and/or M-R (Regexp).

\- Sometimes I need to go to a specific line/col, so I use ^_

Basically there's nothing else I want my text editor to do. I want minimal magic.

<comment><USER20> I know it's not the point you were answering too, but this is exactly why I don't think I will be comfortable in nano, ever. I already know vim, and I don't really bother to remember another set of shortcut just for nano.

When vi/vim aren't available, I usually just end up using sed.

<comment><USER21> The only thing you need to remember with Nano is that ^ means Ctrl. The rest you can find by reading the menus or the built-in help.

Which is why I prefer Nano over vim, I don't edit _that_ much from the console so I only have one easy thing to remember.

</comment></comment></comment></comment></comment><comment><USER22> Really wish all linux systems had it installed out of the box.

vi/vim is difficult for newbies. And the crowd that uses vi/vim will know how to install their editor of choice while the newbie doesn't necessarily

<comment><USER19> What systems don't come with nano these days?

<comment><USER23> CentOS/RHEL didn't ship with it forever (at least to 7), I don't know if that changed with 8 but it was one of the reasons I learned enough vi to be dangerous, as I had to debug a bunch of weird networking stuff on VM guests that were CentOS 5.5.

<comment><USER24> You might be happy to know that Fedora recently changed Nano to the default editor, which means it will likely land in RHEL as the default eventually.

</comment></comment></comment></comment><comment><USER25> "nano -w" is a great editor. "nano" without -w should be banned like "rm -rf /" and require some long DWIM switch to even start.

<comment><USER26> Not wrapping should be the default for every editor. AFAICT, it is not the default for any editor.

I don't think nano is unique in this respect.

<comment><USER27> > Not wrapping should be the default for every editor. AFAICT, it is not the > default for any editor.

It appears to be the default with emacs — at least, hard-wrap off appears to be the default. Soft-wrap is on, but … I can't imagine that you dislike soft- wrap, do you?

<comment><USER26> I do! It makes it difficult to see what's going to happen when I invoke operations on lines. And it encourages my colleagues to create long lines that would play better with diff tools like git diff if broken out.

</comment></comment></comment></comment><comment><USER28> Checked what came with osx

        >>> nano --version
        GNU nano version 2.0.6 (compiled 14:41:52, Apr 17 2020)

comfortingly my freshly brewed 5.0 looks identical when editing...

Also:

        >>> ls -l /usr/bin/pico
        lrwxr-xr-x root wheel 4 B Thu Jan 23 17:02:12 2020 pico ⇒ nano

<comment><USER29> And of course here's why: [https://git.savannah.gnu.org/cgit/nano.git/commit/COPYING?id...](https://git.savannah.gnu.org/cgit/nano.git/commit/COPYING?id=d0035b4ab28c061c6ecabfd2945e05d55d6f3f70)

<comment><USER30> Related, for more context: [http://meta.ath0.com/2012/02/05/apples-great-gpl- purge/](http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/)

</comment></comment></comment><comment><USER31> I started using Nano because of the Digital Ocean blogs. I still use it 4 years later, everywhere from side projects to firefighting in production. It's so simple and easy to use and I'm looking forward to 5.0 changes!

</comment><comment><USER32> For config file editing, nano is great.

Vim's 'return me to the last location in this file'[0], however, is great when repeatedly editing large config files in this same place.

Nano should try to implement it, if it hasn't already.

[0] [https://stackoverflow.com/questions/8854371/vim-how-to- resto...](https://stackoverflow.com/questions/8854371/vim-how-to-restore-the- cursors-logical-and-physical-positions)

<comment><USER33> That would be:

        ## Remember the cursor position in each file for the next editing session.
        set positionlog

The flag was there since 2011, but I only started to enable it a few months ago. Pretty nifty feature, although it took me some time to get used to the new flow.

</comment></comment><comment><USER34> Nano is my dog in the Vim-vs-Emacs holy war.

</comment><comment><USER35> Nano is great as an idea - a editor with almost no features which is small, fast and is everywhere. Howerer, it suffers from the same as all open source projects. Working on it is fun, so they add new feautres with each version. Now 'nano' is not nano anymore. It even has had syntax highlighting for some time already.

<comment><USER36> > Now 'nano' is not nano anymore.

Nothing beats plain old 'vi' (NOT vim!).

<comment><USER37> Do distros still come with original vi? My understanding is that when you type 'vi' you get a minimal version of vim on most (all?) distros theses days.

<comment><USER36> 'vi' is still 'vi' in OSX, and I believe the "original" 'vi' is accessible as 'nvi' on the usual Linux distros. (e.g., [https://packages.ubuntu.com/focal/nvi](https://packages.ubuntu.com/focal/nvi))

<comment><USER38> nvi is new vi, released as part of 4.3BSD-net2. The only place I have encountered original vi is on Solaris. Solaris vi does horrible things like crashing if you maximize your xterm too big...

</comment></comment></comment></comment></comment><comment><USER39> I wonder how many people here know about mc (as in midnight commander) and its builtin viewer/editor.

<comment><USER40> I love Midnight Commander. It's the first thing I install on any Linux system.

</comment></comment><comment><USER41> I despise Nano. I support the effort of anyone working on open source, but really don't like using the editor personally. As a result, it's really frustrating when setting up a new box and VIM isn't the default.

<comment><USER21> > As a result, it's really frustrating when setting up a new box and VIM isn't > the default.

However having Nano as default is the better choice. You as an expert know how to change the default. Newbies who don't likely will prefer a discoverable editor like Nano.

<comment><USER42> I agree. And if you only occasionally need text editor on a linux server, like when you change some config file or update cronjob and nothing more, nano is perfectly adequate, without needing to learn all the vim shortcuts.

<comment><USER43> > And if you only occasionally need text editor on a linux server, like when > you change some config file or update cronjob and nothing more, nano is > perfectly adequate

That's precisely when nano is the least adequate: "sudo -e" or "crontab -e" won't pass the "-w" flag to nano, and without that flag, the automatic hard line wrapping can easily corrupt config files which depend on line breaks for their semantics (a crontab being a great example, where each line is a separate entry).

<comment><USER42> Thanks, I check that flag. Anyway, I never encountered the problem you are mentioning...

</comment></comment></comment></comment></comment><comment><USER44> nano is the best.

<comment><USER45> But it's not standard. Ed is the standard editor.

<comment><USER44> It's the standard of how my brain operates.

</comment></comment></comment><comment><USER46> I was always been annoyed whenever an editor other than vi opened in a console/terminal. Recently I've been learning an alternative keyboard layout which has gone well--except that my vi muscle memory is completely broken. I think I'll appreciate this now and accept it gladly even when my new vi memories are formed.

</comment><comment><USER47> Wow, those are useful improvements... especially the toolbar. A bit slow in arriving, but still good.

> For the color names red, green, blue, yellow, cyan, magenta, white, and > black, the prefix 'light' gives a brighter color.

Catching up to the Nineteen...eighties, I think? :-P

</comment><comment><USER48> Nano is the singularly greatest tool for Windows users to be productive administrating a Linux implementation. I love this editor and have been using it since it's first release.

</comment><comment><USER49> I am currently using vis editor[0] and I am liking it.

[0] [https://github.com/martanne/vis](https://github.com/martanne/vis)

</comment><comment><USER50> I've often thought about contributing to the nano project. Having written a few editors myself I still have a passion for it. I use nano for quite a bit.

</comment><comment><USER51> If you are an emacs person but need something like nano for quick edits on remote machines, try "mg".

I think it's the default editor on some BSDs.

</comment><comment><USER14> I don't do a ton of editing in a non-graphic environment but enjoy confusing the other C# coders at work by editing a code file in vim with ConEmu from time to time. I never thought to see if Nano was also in ConEmu until reading the linked announcement and these comments -- and it is! I'll have to play with Nano a bit more but it seems like the difference between editing in Notepad versus VS Code: good for really simple stuff but the power of vim is needed if you're doing something non-trivial.

</comment><comment><USER52> Once, as a little personal challenge, I've used solely nano + fzf as my IDE and programming editor. I went pretty far with it :)

<comment><USER53> how far can you go? I've been on the lookout for a friendly vim like editor. at the same time I really like a IDE features

</comment></comment><comment><USER7> Interesting — I did not realise Nano was a GNU project again. In fact, Nano apparently did not leave GNU for very long.

<comment><USER54> For details: [https://nano-editor.org/news.php](https://nano- editor.org/news.php)

Leaves GNU in 2016 June 17, rejoins GNU in 2016 September 1

Some past HN insight (Misleading information in _my_ HN thread? It's more likely than you think): [https://news.ycombinator.com/item?id=11953044](https://news.ycombinator.com/item?id=11953044)

Blog post by Chris: [https://www.asty.org/whats-up-with- nano](https://www.asty.org/whats-up-with-nano)

"Request to move nano from gnu to nongnu" [https://savannah.nongnu.org/support/?109076](https://savannah.nongnu.org/support/?109076)

"[Nano-devel] nano to remain in GNU" [https://lists.gnu.org/archive/html/nano- devel/2016-08/msg000...](https://lists.gnu.org/archive/html/nano- devel/2016-08/msg00045.html)

<comment><USER55> >For details: [https://nano-editor.org/news.php](https://nano- editor.org/news.php)

That's a beautiful website.

</comment></comment></comment><comment><USER56> mcedit is what use instead of nano. Similar to the editor in Far Manager and Norton Commander.

</comment><comment><USER57> This editor is my daily driver. Has been for years and years. Glad it's still kicking!

</comment><comment><USER58> I'm bothered by the UI design of this webpage.

</comment><comment><USER59> The first thing I uninstall on a new system is nano.

<comment><USER60> I don't uninstall it but I don't really get the love, mostly because of the keybindings. Maybe I'm too young and don't know its legacy but they seem super esoteric to me and I've never seen them anywhere else. Emacs navigation bindings are similar to terminal bindings, and well vi is its own world but at least fairly widespread today.

<comment><USER61> nano is a GPL-licensed clone of pico, which in turn was originally conceived as the integrated message editor in PINE, the popular UNIX e-mail client first released in 1989. If you look at the overall interface of PINE you will recognize that it is consistent with pico/nano, but other than that there really wasn't much consistency in terms of keyboard commands between apps in 1989.

More importantly, while for instance ^S for "save" might seem more natural than ^O for "writeOut", remember that dumb terminals over slow lines were still very much a thing in 1989 and ^S was already used as a terminal control code for XOFF (software flow control) -- which is why nano throws that "XOFF ignored, mumble mumble" message when it catches ^S.</comment></comment></comment>


Show HN: Reddit/HN for Indie Music - <AUTHOR> http://hitkarma.com

<comment><AUTHOR> A site for independent musicians to share their music and collaborate with other artists. Fans of independent music can discover the best new talent.</comment>


Show HN: Use of AI to monitor 15M corporate websites - <AUTHOR> https://twitter.com/aiHitPulse <comment><USER2> While interesting, it appears it's a bot account posting multiple times per minute. I'll be shocked if Twitter doesn't ban this, and soon.</comment>


How We Beat C++ STL Binary Search - <AUTHOR> https://realm.io/news/how-we-beat-cpp-stl-binary-search/ <comment><USER2> For 8192 elements I also get result that their search function is faster than STL. But 8192 elements is tiny size of array where to search, for larger array I get opposite - STL is faster. Here is output for 8 * 1024 * 1024 elements (~8.4million).

gcc 5.2.0 on Windows x64 (i5-3210M)

        stl      : 881 miliseconds
        version 1: 880 miliseconds
        version 2: 1607 miliseconds
        version 3: 1260 miliseconds
        version 4: 1271 miliseconds

gcc 5.2.0 on Linux x86_64 (i7-4770S)

        stl      : 629.231 miliseconds
        version 1: 629.436 miliseconds
        version 2: 897.143 miliseconds
        version 3: 862.827 miliseconds
        version 4: 863.22 miliseconds

clang 3.6.2 on Linux armv7h (CuBox-i, Cortex-A9)

        stl      : 3380.29 miliseconds
        version 1: 3428.9 miliseconds
        version 2: 3433.65 miliseconds
        version 3: 3391.86 miliseconds
        version 4: 3376.91 miliseconds

Oh, and Visual Studio doesn't have "/O3" argument they say they are using for cl.exe.

<comment><USER3> This is fascinating. Do you have any idea what might cause this?

A shot in the dark: Their benchmarking uses a very simple "random" generation to choose the index to search for, which is actually just a linear scan (modulo the size of the array). Could it be that with the larger array, the generated sequence of test indices happens to work nicely with the CPU's branch prediction - after all, your results show a performance drop-off for the versions that use conditional moves, and the behavior depends suspiciously on the CPU microarchitecture.

</comment></comment><comment><USER4> There is no mention of the fact that the STL implementation is very generic, it only assumes operator++ and operator* on the iteration, and operator< on the value.

The "optimized" versions here all make more assumptions on the iterator, starting from operator+(int) in Version 1, so it no longer works on iterators with just "forward_iterator_tag". Further versions even restrict vector sizes (albeit to a very high number) and assign -1 to an unsigned integer (size_t). So this is something you can use in your project if you need the performance, but can't put it into GCC.

<comment><USER5> Just a nitpick: Assigning a negative value to an unsigned integer is perfectly safe and well defined (i.e. integer overflow for unsigned integers is well defined in the C++ spec).

Using -1 for size_t is sometimes even prefered to assign it's maximum value, since there is no cross-platform #define for it (most use either SIZE_T_MAX or SIZE_MAX).

In fact llvm's libc++ uses it for it's std::numeric_limits implementation and for std::string's npos.

<comment><USER6> I believe there is a standard cross-platform maximum value, and you mention it in your last sentence: std::numeric_limits!

<comment><USER5> Uhm... Using "-1" to assign the maximum value is in fact as standard and cross-platform as using std::numeric_limits, which was basically my entire point.

The difference between those is only "textual solution" VS "mathematical solution" and thus a matter of personal preference.

</comment></comment></comment></comment><comment><USER7> Loosely related: STL binary search suffers from the same weakness as several other comparator-based STL algorithms for certain classes of expensive comparators: [https://www.reddit.com/r/cpp/comments/36sqtq/more_efficient_...](https://www.reddit.com/r/cpp/comments/36sqtq/more_efficient_interface_for_algorithms_taking_a/)

</comment><comment><USER8> If you take version 4, stop the first while loop once the size is at most 32, and then do linear search from there, it's faster (on my machine). For example, version 4 is 102ms and this tweak puts it at 83ms. (On clang x64.)

</comment><comment><USER9> Isn't version 2 wrong, because

        size_t probe = (low + high) / 2;

may overflow?

<comment><USER10> Not if low and high are both unsigned (size_t is unsigned.) Even if it overflows, the result will be correct. See: [http://googleresearch.blogspot.com/2006/06/extra-extra- read-...](http://googleresearch.blogspot.com/2006/06/extra-extra-read-all- about-it-nearly.html)

<comment><USER11> That blog is wrong. The program won't be undefined( unsigned wrap is defined ), but the offset will not be correct.

<comment><USER12> So the article isn't wrong, but its misleading you into thinking this should be correct for the wrong reasons.

They are using signed integers as their indices, which means that the signed bit is always 0. Thus the addition after casting to unsigned will never overflow, and you can divide by two (shift by 1) and then recast to a signed integer, no harm no foul.

<comment><USER11> Actually it is misleading by them( and you ) to assume that in C, an unsigned int can represent values larger than the largest signed int value.

In other words, C allows that UINT_MAX == INT_MAX, in which case you will overflow.

If they made that assumption, they should explicitly mention it, but they didn't.

> Update 17 Feb 2008:... ...Now that we've made this change, we _know_ that > the program is correct;)

It seems the article is aware of the irony. Another update would be in order.

<comment><USER12> Sure, and if you expect your software to run on such a platform with any degree of confidence then you're right to consider it. Even better, tell us about a conforming implementation that you've used in product recently that has UINT_MAX == INT_MAX.

Also I'm not trying to mislead people into thinking that its a good way to implement this. The confounding bit from the article is they started in Java and ended up in C. If you were indexing with signed ints in C, C++, or any language that has unsigned integers then you already have a bug with or without the bad mean check.

<comment><USER11> You got it backwards there. Only if you know your implementation and plan to code only for it, can you even start to consider bending the C Standard, and not the other way around.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> (76% + 81% + 80% + 75% + 70%) / 5 = 76%

[http://www.cse.unsw.edu.au/~gernot/benchmarking- crimes.html](http://www.cse.unsw.edu.au/~gernot/benchmarking-crimes.html)

</comment><comment><USER14> My version:

      template <class T> INLINE size_t fast_upper_bound5(const vector<T>& vec, T value)
      {
          size_t index = 0;
          size_t size = vec.size();
          while (size > 0) {
              size /= 2;
              size_t probe = index + size;
              if (vec[probe] <= value)
                  index = probe + 1;
          }
          return index;
      }

Slightly faster with gcc:

      $ g++-mp-4.9 -std=c++11 -O3 -DNDEBUG -o blog blog.cpp
      $ ./blog
      size = 8192:
          stl      : 144.883 miliseconds
          version 1: 145.406 miliseconds
          version 2: 129.713 miliseconds
          version 3: 109.231 miliseconds
          version 4: 103.578 miliseconds
          version 5: 102.282 miliseconds

But sucks with clang, apparently because of the described problem with non- using cmov instruction.

      $ clang++-mp-3.6 -std=c++11 -O3 -DNDEBUG -o blog blog.cpp
      $ ./blog
      size = 8192:
          stl      : 147.466 miliseconds
          version 1: 145.978 miliseconds
          version 2: 145.547 miliseconds
          version 3: 113.546 miliseconds
          version 4: 106.968 miliseconds
          version 5: 144.231 miliseconds

</comment><comment><USER15> It should be noted that their version gives different results to std::upper_bound if your range contains duplicates.

</comment><comment><USER16> This is nice but what are the use cases for such an optimization ? My understanding is that such an optimization will not matter anyway compared to all the I/O and other expensive operations. And if a program is really too slow then the true optimization is to scale with threads or distributed computing.

</comment><comment><USER17> Great link.

The most educational bit for me was the careful structuring and eventual elimination of the if/else to shake out a conditional move rather than an unpredictable branch.

Modern optimizers and CPU scheduling engines are so powerful, a lot of received wisdom on how to code for speed is outdated; manual loop unrolling, for example, is rarely very beneficial. It's nice to see that there's still some room for craftsmanship in the most critical of paths. Structuring loops for autovectorization is another useful habit to get into.

<comment><USER11> Wouldn't a conditional move transfer the cost from branch mispredict to the cache miss.

I wonder how realistic their test were, and if the same results would be achieved with some cache trashing.

<comment><USER17> If you're going to miss when you go back to the array, you're going to pay that cost regardless. But without the mispredict, you'll be able to issue the load that much sooner.

I'm not familiar enough with Intel's architecture to know one way or the other, but it wouldn't surprise me if not mispredicting the next memory access saves you more than just some pipeline flushing: The CPU could speculatively issue the load you don't actually need, wasting resources that could be used to service the correct addresses.

Or did you mean something else?

<comment><USER11> Well a branch will fetch the incorrect cache line if it mispredicts, but a conditional move will fetch both cache lines every time.

<comment><USER18> The conditional move is the statement `low = v >= value ? low : other_low;` which either assigns `other_low` to `low` or does nothing.

And other_low is a variable, and not an arbitrary element of the list (which is a big difference with respect to cache), and it can also be seen from the assembly that it's stored in register. So there there is no "both cache lines" to fetch anything from.

<comment><USER11> The conditional move still depends on both values taken from the array, via other variables.

</comment></comment></comment></comment></comment></comment><comment><USER19> I think it's at the very least been hinted at below (and there are other good points), but note that their implementation makes copies. If the copy ctor of type T is the least bit expensive, the STL (implementation in glibc) for this same benchmark is far faster than theirs. Try it. :)

</comment><comment><USER20> One of the problems with the extreme complexity of submitting code to GCC, and FSF projects in general, (having to complete copyright assignments, which are very slowly handled) is that this is unfortunately unlikely to end up in libstdc++ (although I would be happy to see it there).

<comment><USER21> I think the fact this violates the standard would be the greatest difficulty.

</comment></comment><comment><USER22> Paul has been doing great work here: [http://repnop.org/pd/slides/bsearch.pdf](http://repnop.org/pd/slides/bsearch.pdf)</comment>


Xbox Adaptive Controller - <AUTHOR> https://www.xbox.com/en-US/xbox-one/accessories/controllers/xbox-adaptive-controller <comment><USER2> This is really impressive. I'm happy someone is pushing the controller forward, as someone who's worked in video game controller tech.</comment>


Ask HN: Include or not include side company on Linkedin? - <AUTHOR> <comment><USER2> "Side company" as in one you run yourself? Maybe, if it's relevant to your field, and you actually worked on/released a product. If it's just hobby projects, I don't see that being as relevant.

<comment><AUTHOR> Side company meaning company being created, on the side, to eventually be f/t job.

<comment><USER2> I see no pressing need to include it at this time.

<comment><AUTHOR> clintonb - can we schedule like 15 mins to talk about this over a call? I have a unique scenario I wanted to really get your honest opinion on.</comment></comment></comment></comment>


Code for Covid-19 modelling by the Imperial College/Neil Ferguson (main part) - <AUTHOR> https://github.com/mrc-ide/covid-sim/blob/bd87d475563cd54978325bf73ce45e80a7c8de65/src/CovidSim.cpp <comment><AUTHOR> This links to the first commit, before the MS team cleaned up things. Not sure about the differences, but this should be closer the the real thing.</comment>


The Service Mesh: What Engineers Need to Know - <AUTHOR> https://servicemesh.io/ <comment><USER2> One reason for the explosive interest in service mesh over the last 24 months that this article glosses over is that it's deeply threatening to a range of existing industries, that are now responding.

Most immediately to API gateways (eg. Apigee, Kong, Mulesoft), which provide similar value to SM (in providing centralized control and auditing of an organization's East-West service traffic) but implemented differently. This is why Kong, Apigee, nginx etc. are all shipping service mesh implementations now before their market gets snatched away from them.

Secondly to cloud providers, who hate seeing their customers deploy vendor- agnostic middleware rather than use their proprietary APIs. None of them want to get "Kubernetted" again. Hence Amazon's investment in the very Istio-like "AppMesh" and Microsoft (who already had "Service Fabric") attempt to do an end run around Istio with the "Service Mesh Interface" spec. Both are part of a strategy to ensure if you are running a service mesh the cloud provider doesn't cede control.

Then there's a slew of monitoring vendors who aren't sure if SM is a threat (by providing a bunch of metrics "for free" out of the box) or an opportunity to expand the footprint of their own tools by hooking into SM rather than require folks to deploy their agents everywhere.

Finally there's the multi-billion dollar Software Defined Networking market - who are seeing a lot of their long term growth and value being threatened by these open source projects that are solving at Layer 7 (and with much more application context) what they had been solving for at Layer 3-4. VMWare NSX already have a SM implementation (NSX-SM) that is built on Istio and while I have no idea what Nutanix et al are doing I wouldn't be surprised if they launched something soon.

It will be interesting to see where it all nets out. If Google pulls off the same trick that they did with Kubernetes and creates a genuinely independent project with clean integration points for a wide range of vendors then it could become the open-source Switzerland we need. On the other hand it could just as easily become a vendor-driven tire fire. In a year or so we'll know.

<comment><USER3> This is a good overview. However, I think that the reason that we see a lot of service variations is because the core tech - namely - Envoy, contains all the "hard" tech (the data plane) while creating a "service mesh", basically comes down to creating a management layer on top of it.

Another intresting note is that Google did NOT recede control over Istio to CNCF.

<comment><USER4> > _Envoy, contains all the "hard" tech (the data plane) while creating a > "service mesh", basically comes down to creating a management layer on top > of it._

I'd argue this is backwards. Envoy has a fairly tightly defined boundary with relatively strong guarantees of consistency given by hardware -- each instance is running on a single machine, or in a single pod, with a focus on that machine or pod.

The control plane is dealing with the nightmare of good ol' fashioned distributed consistency, with a dollop of "update the kernel's routing tables quickly but not _too_ quickly" to go with it. It's "simple" insofar as you don't need to be good at lower-level memory efficiency and knowing shortcuts that particular CPUs give you. But that's detail complexity. The control plane faces dynamic complexity.

</comment></comment></comment><comment><USER5> I'm going to sound like an old man but...

What amuses me about this is back in the day everyone thought the Mach guys were crazy for thinking things like network routing and IPC services be implemented in user space... and others mocked the OSI model's 7 layers as overly complex (e.g. RFC3439's "layering considered harmful").

Now we've moved all our network services onto a layer 7 protocol (HTTP), and we've discovered we need to reinvent layers we skipped over on top of it. We're doing it all in user space with comparatively new and untested application logic, somehow forgetting that this can be done far more efficiently and scalably with established and far more sophisticated networking tools... if only we'd give up on this silly notion that everything must go over HTTP.

<comment><USER6> Network-over-network is just another Inner Platform Effect.

<comment><USER7> Wonderful term. I've been aware of the phenomenon for a while but not its name.

Link for others:

[https://en.wikipedia.org/wiki/Inner- platform_effect](https://en.wikipedia.org/wiki/Inner-platform_effect)

<comment><USER8> Oh wow! I had a customer who was really bad at it. All! there Software was effected by it in one way or another.

Now I have a name for that at least :)

</comment></comment></comment></comment><comment><USER9> At my company, we were migrating all our apps to a kubernetes + istio platform over the past couple of months and my advice is this - don't use a service mesh unless you really, really need to.

We initially choose istio because it seemed to satisfy all our requirements and more - mTLS, sidecar authz, etc - but configuring it turned out to be a huge pain. Things like crafting a non-superadmin pod security policy for it, trying to upgrade versions via helm, and trying to debug authz policies took up a non-trivial amount of time. In the end, we got everything working but I probably wouldn't recommend it again.

It's funny that I was at kubecon last week and there was a start up whose value prop was hassle-free istio and the linkerd people stressed that they were less complex than istio.

<comment><USER10> I would go as far as to say I think the vast majority of people don't need a specialized service mesh. We unfortunately started with Linkerd and it actually is the cause of most reliability/troubleshooting issues. I don't think lack of complexity is actually a good selling point for it, because it's inherently more complex that not using a service mesh.

Istio may appear more complex but that's because it has a superior abstraction model and supports greater flexibility. We're beginning to migrate from Linkerd to Istio at this point. I had the same initial frustrations with podsecuritypolicy (and linkerd suffers from the same), but istio-cni solves the superuser problem, and I believe even the istio control plane is now much more locked down in the latest release.

However if I had my way I would be telling every team they don't need service mesh. We don't have any particular service large and complex enough to really take advantage of its sold features.

<comment><USER11> >"We unfortunately started with Linkerd and it actually is the cause of most reliability/troubleshooting issues"

Would you mind elaborating on what those Linkerd issue are/were that were effecting reliability and troubleshooting?

<comment><USER12> I'm also curious about this (author here btw). The majority of people we see coming to Linkerd today are coming _from_ Istio. They get the service mesh value props, but want Linkerd's simplicity and lower operational overhead. Would love some more details, especially GitHub issues.

</comment></comment></comment></comment><comment><USER13> My favorite use of this kind of system is to manage tls and acls. The service itself can be extremely dumb and just expose a unix socket.

The 10:1 ratio of microservices to developer sounds like hell though that's just too much to reason about.

</comment><comment><USER14> Good article - I must admit I'm vaguely familiar with the concept and this read certainly gave me some new insights.

One meta call out on the writing - I read and scrolled at least 30% through the page on my iPhone until the author explained why I should care about a service mesh I.e. what problems it tries to simplify or solve.

It seems to me there are some strong use cases here, but it's only worth your while if you're operating at sufficient scale.

For instance, if my team at some FAANG scale company is responsible for vending the library that provides TLS or log rotation or <insert cross cutting/common use case here>, and it requires some non trivial on boarding and operational cost, migrating to this kind of architecture longer term where these concerns are handled out of the box may be beneficial.

Still - it doesn't mean the service owners are off the hook. They still need to tune their retry logic, or confirm the proxy is configured to call the correct endpoints (let say my service is a client of another service B and for us, B has a dedicated fleet because of our traffic patterns). This is an abstraction. Abstractions have cost.

Trust but verify.

The trap people fall into is, "Here's a new technology or concept. Let's all flock to it without considering the costs."

</comment><comment><USER15> It's a pity "fat clients" are dismissed so quickly. I think that when your tech stack is uniform enough to use them, they can provide much more that service meshes, and do it faster as well. After all, why does "service is down" and "service is sending nonsense" have to be handled via completely different paths?

<comment><USER16> The main problem with fat clients is that for polyglot architectures (which most large companies that end up building a service mesh evolve into over time) you have to maintain a fat client library for every language. You can get very far leveraging existing tools like gRPC that codegens fatty clients for you but the quality of tooling is very uneven depending on the language of choice. By pushing all of this into the network layer you skip all of that.

<comment><USER15> Right, polyglot architectures have no choice, but this text talks about "5 person startups" as well. Surely they can keep the set of languages limited?

Plus, it's not either/or situation. A fat clients for Go + Node.js; and a proxy for all others. This way your core logic can enjoy increased introspection / more speed / higher reliability; while special purpose services get a proxy which allow interoperability.

</comment></comment></comment><comment><USER17> As someone who's familiar with the API gateway pattern, is it fair to say this is just another API gateway for internal services? Seems like it is but its also described in an extremely convoluted way with 'control planes' and such.

<comment><USER18> The service mesh is a bit different from an API gateway -- in it's current most popular implementations (linkerd[0] & Istio[1]), there are basically small programs that run next to _each individual instance_ of the programs you want to run. Linkerd has been around for a while and IMO there weren't _that_ many companies that were at a scale where they needed it (I didn't see it deployed that often), but it's basically that same concept, but on a more granular level -- if you delegate all your requests to some intermediary, then the intermediaries can deal with the messy logic and tracing so your program doesn't have to.

A better way to describe is "smart pipes, dumb programs". Imagine that all your circuit-breaking/retry/etc robustness logic was moved into another process that happened to be running right next to the program actually doing the work.

You can have both an API gateway _and_ a service mesh deployment -- for example Kong's Service Mesh[2] works this way. They're saying stuff like "inject gateway functionality in your service", but that only make sense if you sent literally every request (whether intra-service or to/from the outside world) through the gateway. _Maybe_ that's how some people used Kong but I don't think everyone thought of API gateways as a place to send every single request through. You'll have a Kong API gateway at the edge _and_ the kong proxies (little programs that you send all your requests through) next to every compute workload.

[0]: [https://linkerd.io/](https://linkerd.io/)

[1]: [https://istio.io/](https://istio.io/)

[2]: [https://konghq.com/solutions/service- mesh/](https://konghq.com/solutions/service-mesh/)

<comment><USER17> Hmm, is the assumption that, because you're deploying an instance of the mesh as close to the application as possible, you don't need robust logic between the application and the service mesh? I can buy that I suppose.

<comment><USER18> Yes kind of -- except not in between the application and the service mesh, it's between application and application.

Imagine that for every application there is _one_ small binary that runs and serves _all_ it's traffic, like a chauffeur. Your application stops talking to the outside world completely and sends all messages to the small chauffeur binary -- which then talks to _other_ chauffeurs, over the network.

Keeping with the chauffeur analogy, there is a "head office" which calls the chauffeurs on CB radio at regular intervals that lets them know which cars go where and how to start them/etc.

"head office" => "control plane"

"chauffeur" => "side-car proxy"/"data plane"

In the end what this means for your application is that you just make calls to external services (whether your own or others) and since _all_ your communication goes through this other binary, you get monitoring, traffic shaping, enhanced security, and robustness for free.

Another interesting feature is that if the side-car proxy can actually _understand_ your traffic, it can do even more advanced things. For example you can prevent `DELETE`s from being sent to Postgres instances at the _network_ level.

</comment></comment></comment></comment><comment><USER19> Every part of a service mesh could be baked into operating systems so that all this extra technology was just there by default. This would put a fair amount of start-ups out of business, but it would also mean a lot less people having to be hired to set up and maintain all this stuff. Devs could just... develop software, with a clear view into how their apps run at scale. And Ops wouldn't have to custom-integrate 100 different services.

This is really the future of distributed parallel computing, but we're still just bolting it on rather than baking it in.

</comment><comment><USER20> I'm evaluating using AWS App Mesh at the moment. We're a really small team so we're choosing Fargate vs Kubernetes- mainly because we don't have need of nodes nor want to deal with them.

The appeal of App Mesh for us was initially around using it to facilitate canary deployments. AWS Code Deploy does a nice job with Blue / Green deployments and that may suffice for us, but it doesn't support canary for Fargate. Is that enough reason to add the additional complexity in our stack? Not sure, looking for input.

Also, much of the documentation is focused on K8s. I'm murky on how to implement an internal namespace for routing. Most of what I've seen is like myenv.myservice.svc.cluster.local but its not clear to me that using that pattern is needed in the context of Fargate.

Consistent observability is valuable, but again Fargate can do that pretty well- it just doesn't mandate access logging so that would be left to the app itself.

We want to implement OIDC on the edge for some services, but App Mesh doesn't support that yet as other meshes like Ambassador, Gloo, and Istio seem to. Since App Mesh doesn't really act as a front-proxy on AWS, we'll still be using ALB to handle auth which is fine, I think. I get mixed messages about the need for JWT validation, but if so, that would need to be implemented in the app level with ALB fronting it.

Can anybody help me find resources to sort this out? I've been through the `color-teller` example time and time again, but it still leaves lots of open questions about how to structure a larger project and handle deployments effectively.

<comment><USER18> > The appeal of App Mesh for us was initially around using it to facilitate > canary deployments. AWS Code Deploy does a nice job with Blue / Green > deployments and that may suffice for us, but it doesn't support canary for > Fargate. Is that enough reason to add the additional complexity in our > stack? Not sure, looking for input.

Maybe you should write a script for this? It sounds like you're about to take on a _lot_ of complexity for just the ability to do canary deployments when you could probably hack up a script in a day or two.

> We want to implement OIDC on the edge for some services, but App Mesh > doesn't support that yet as other meshes like Ambassador, Gloo, and Istio > seem to. Since App Mesh doesn't really act as a front-proxy on AWS, we'll > still be using ALB to handle auth which is fine, I think. I get mixed > messages about the need for JWT validation, but if so, that would need to be > implemented in the app level with ALB fronting it.

JWTs are only required for client-side identity tokens (you can use opaque ids and other kinds of stuff for backends) -- it seems like you're also at the same time looking for something to take authentication off your hands? App Mesh doesn't do that AFAIK, it's _only_ the service<->service communication that it's trying to solve.

I think it might be a good idea to make a concise need of what you're trying to accomplish here, it seems kind of over the place. From what I can tell it's:

\- Ability to do Canary deployments

\- The ability to shape traffic to services (?)

\- Observability, with access logging

\- AuthN via OIDC at the edge

A lot of meshes do the above list of things, but the question of whether it's worth adopting one just to get the pieces you don't have already (which is only #2 really, assuming you scripted up #1), is a harder question.

<comment> shubha-aws > Namespaces: In order to identify the versions of services for routing, you > need independent virtual nodes and routes in a virtual router. You can reuse > the DNS names or use cloudmap names with metadata to identify the > versions/virtual nodes. > OIDC at ingress - App Mesh does not do this yet, > ALB / API Gateway is needed for this. App Mesh has this on the roadmap. > > Resources - You can reach the app mesh team with specific questions at the > App Mesh roadmap Github and we can help

</comment></comment></comment><comment><USER21> Re. "fat client" libraries:

> Sure, it only worked for JVM languages, and it had a programming model that > you had to build your whole app around, but the operational features it > provided were almost exactly those of the service mesh.

The thing is, all of our microservices communicate with each other using Kafka. Envoy has an issue open for Kafka protocol support [1], but it's a fundamentally difficult issue because adopting Kafka forces you to build out "fat client" code and building a network intercept that can work with pre- existing Kafka client code is non-trivial. On observability, Kafka produces its own metrics.

Granted, Kafka doesn't offer the same level of control. But Kafka does offer incredible request durability guarantees. We don't have "outages" \- we have increased processing latency, and Istio/Envoy and other service meshes can't offer that because they do not replicate and persist network requests to disk.

[1] [https://github.com/envoyproxy/envoy/issues/2852](https://github.com/envoyproxy/envoy/issues/2852)

</comment><comment><USER22> Opinionated read, but interesting. That being said, Linkerd wasn't the first service mesh — SmartStack predates it by three years. [1] Although they didn't use the (then-nonexistent) "service mesh" term at the time, it pioneered the concept of userspace TCP proxies configured by a control plane management daemon. I doubt the Linkerd folks are unaware of it, so it was a surprising omission.

[1]: [https://medium.com/airbnb-engineering/smartstack-service- dis...](https://medium.com/airbnb-engineering/smartstack-service-discovery-in- the-cloud-4b8a080de619)

</comment><comment><USER23> While nobody ever seems to want to hear it, the vast majority of companies utilizing service meshes and k8s are wasting huge amounts of time and money on things they don't need.

Unfortunately these technologies are at peak hype so everyone seems to be implementing them for their small to medium crud apps. But get very sensitive if you try and point it out.

</comment><comment><USER24> How many transactions per second before you need all that stuff? If you're not in the top 100 sites, it seems unnecessary.

<comment><USER25> It's not as much about load as it is about complexity; it starts to make sense when you hit some threshold number of internal services, regardless of the amount of traffic you're doing. You use a service mesh to factor out network policy and observability from your services into a common layer.

<comment><USER26> What is the threshold above which a service needs to exist at all, over a module in an existing codebase?

<comment><USER27> The point at which you have multiple teams working on the same codebase, and their velocity is suffering from communication overhead and missteps.

<comment><USER28> A few remarks:

* Codebase should be defined as 'the platform'. where one team will most likely never look at the code of other team's microservices. * this communication problems and overhead start the moment you go from 2 to 3 or more teams. * the term 'team' in this context should be interpreted very broadly. One dev working alone on a microservice should be considered "a team".

Also, things mentioned in the article: you don't want to implement TLS, circuit breakers, retries, ... in every single microservice. Keep them as simple as possible. Adding stuff like that creates bloat very quickly.

</comment></comment></comment></comment></comment><comment><USER29> This is quite interesting. I used to work in more devopsy kind of roles but at the current gig it has been almost entirely removed from my purview. It's impressive to step away for a few years and return to see so many changes, but the article laid out the concepts in an easy to understand manner.

</comment><comment><USER30> If one were to implement a service mesh of microservices wouldn't the services need to be versioned similar to how the packages used by a microservice are version-pinned?

<comment><USER31> Sort of, but only for major versions, and it's preferable to bake that sort of thing into the API itself. The API exposed by a microservice should only ever be updated in backwards compatible ways unless you can verify that you have no callers, which is hard. New functionality should be introduced using backwards compatible constructs like adding fields to JSON or protobuf. Breaking changes go in a new API. This is easily managed conceptually by having the microservice expose version information as part of the API. A FooService might define "v1/DestroyFoo" and "v2/DestroyFoo" with different calling contracts. Perhaps v1 was eventually consistent and returns a completion token that can be used with a separate "v1/CheckFooDeletionStatus", but now with v2 the behavior has been made strongly consistent and there is no "v2/CheckFooDeletionStatus". The v2 of the API can thus be thought of really as a separate API that happens to be exposed by the same microservice, and pre-existing callers can continue to call the (perhaps now inefficient) v1 API.</comment></comment>


Well Crafted Code, Quality, Speed and Budget - <AUTHOR> http://devteams.at/well_crafted_code_quality_speed_budget <comment><USER2> Curious if the author has read anything by Watts Humphrey? He spent his entire career at SEI arguing this point and attempting to develop the tools and processes to bear it out. This article reads like a repetition of his arguments from A Disicipline for Software Engineering (1995).

<comment><USER3> I learned in a discussion with "kragen" that "software engineering" sort of took on a life of its own that turned into a nightmare of management-focused stuff harmful to programming. Most programmers, especially mainstream crowd, have an aversion to the term from the permanent association. I started saying CompSci and focusing on prior techniques rather than "engineering" due to that permanent scarring. As in my other comment, I just focus on specific techniques and processes that help programmers do their job with evidence they work.

It's my suggestion to you. I'm going to try to dig up and read that paper out of curiosity as I know many useful things came out of software engineering research. Most won't, though, the second they see "SEI" or "software engineering." So, might be best for us to just drop that and focus on improvements to "programming" or "development" with specific techniques.

</comment></comment><comment><USER3> I'm mixed about the post, esp the science part. The science of developing robust software is great and pretty consistent going back decades varying mostly in specific tools and tactics. Mainstream programming just _doesn 't apply it_ although more adoption in past decade of key techniques. Here's some computer science from the 1960's-1980's used in robust and secure system development (esp Orange Book B3 or CC EAL6) people might want to copy. I'm taking an empirical route where I reference techniques that were applied to many real-world projects with lessons learned in papers or studies that were consistent. All one can do with limited data & these aren't in order of importance.

1\. Formal, non-English (eg math/logical) specifications of requirements or abstract design. English is ambiguous and misreadings of it caused countless errors, even back then. CompSci researchers tried formal specs with both English as a start and precise notations (eg Z, VDM, ASM's, statecharts) for clarity in specifics. Result was many inconsistencies caught in highly assured systems and protocol specs before coding even began.

2\. High assurance stuff often used mathematical (formal) verification. Whether that worked or made sense was hit and miss. More on it later. Yet, virtually all of them said there was benefit in restrictions on the specs, design, and coding style to fit the provers' limitations. Essentially, they used boring constructs that were easy to analyse and this prevented/caught problems. Don't be too clever with design or code. Wirth and Hansen applied this to language design to bake safety & comprehension in with minimal to low loss in performance.

Note: Led to Nick P's Law of Trustworthy Systems: "Tried and true beats novel or new." Always the default.

3\. Dijkstra's THE project showed that modular, layered design with careful attention to interfaces (and interface checks) makes for most robust and maintainable software. Later results confirmed this where each module must fit in your head and control graph that's pretty predictable with minimal cycles prevented all kinds of local-becomes-global issues. Many systems flawless (or nearly so) in production were built this way. Dijkstra correctly noted that it was _very hard_ to do this even for smart people and average developer might screw structuring up a lot. Solid prediction... but still worth striving for improvement here.

4\. Fagan ran empirical studies at IBM that showed a regular, systematic, code review process caught many problems, even what tests missed. Turned that into formal inspections with the periodicity and prioritizing tuned per organization for right cost-benefit. Was generalized to whole SDLC by others in high robustness areas. Improved every project that used it from then on. Exactly what parameters to use is still open-ended but periodically looking for well-known flaws with reference sheet always works.

5\. Testing for every feature, code-path, prior issues outside of code base, and common use-case. All of these have shown repeated benefits. There's a cut- off point for each that's still an open, research problem. However, at a minimum, usage-based testing and regression testing helped many projects achieve either zero or near-zero, _user-facing_ defects in production. That's a very important differentiator as 100 bugs user never experiences is better than 5 that they do regularly. Mills' Cleanroom process combined simple implementation, code review, and usage-testing for insanely-high, statistically-certifiable quality even for amateur teams.

6\. By around 60's-70's, it became clear that the language you choose has a significant effect on productivity, defects, maintenance, and integration. Numerous studies were run in industry and military comparing various ones. Certain languages (eg Ada) showed vastly lower defects, equal/better productivity, and great maintenance/integration in every study. Haven't seen many such studies since the 90's and most aren't constructed well to eliminate bias. However, it's grounded in science to claim that certain language choices prevent common negatives and encourage positives. So, it follows to adopt languages that make robust development easier.

7\. By the 80's or 90's, it was clear that computers were better at finding certain problems in specs and code than humans. This gave rise to methodologies that put models of system or code into model-checkers and provers to show certain properties always hold (the good) or never show up (the bad). Used successfully with high-assurance safety and security critical systems with results ranging from "somewhat beneficial" to "caught stuff we'd never see or test for." Back then it was unclear how applicable it was. Recent work by Chlipala, Leroy, et al show near perfect results in practice when specs/proofs are right and much wider application than before. Lots of tooling and prior examples means this is a proven way of getting extra quality _where high-stakes are worth the cost_ and where core functionality doesn't change often.. The CompCert C compiler, Eiffel's SCOOP concurrency scheme, and Navy team's EAL7 IPsec VPN are good examples.

8\. Static analysis, aka "lightweight formal methods," were devised to deal with specialized skills and labor of above. Getting to the point, tools like Astree Analyzer or SPARK Ada can prove absence of common flaws with little to no false positives without need for mathematicians in the company. Just a half dozen of these tools by themselves found tons of vulnerabilities in real-world software that passed human review and testing. Enough said, eh?

9\. Software that succeeded with testing often failed when random stuff came at it, especially malware. This led to various fault-injection methods like fuzz testing to simulate that and find breaking points. The huge number of defects, esp in file formats & protocol engines, found via this method argues for its effectiveness in improving quality. It ties in with stuff above in that well-written code that validates input at interface and preserves invariants throughout execution should simply disregard (or report) such erroneous input.

10\. Interface errors themselves posed something like 80+% of problems. This was noted as far back as the 60's in Apollo project when Margaret Hamilton invented software engineering, fault-tolerance, and specification techniques to fight it. Dijkstra and Hoare pushed for pre- and post-conditions plus specific invariants to document the assumptions of code during procedure calls. Modern version is called Design by Contract in Eiffel, Ada, and numerous other languages (even asserts in C). Many deployments and tests showed such interface checks caught many issues, esp assumption violations when new code extended or modified legacy.

11\. Concurrency issues caused all kinds of problems. Techniques were devised by Hansen (Concurrent Pascal) and later Meyer et al (SCOOP) to mostly immunize against them at language level with acceptable performance. Languages without that, especially Java, later got brilliant tooling that could reliably find race conditions, deadlocks, or livelocks. Use of any method inevitably found problems in production code that had escaped detection. So, using prior, proven methods to immunize against or detect common errors in concurrency is A Good Thing. Note that shared-nothing, event-driven architectures also emerged but I have less data on them outside that some (NonStop, Erlang) worked extremely well.

The above are just a few things that computer science established with supporting evidence from real-world projects so long ago that Windows didn't exist. Anyone applying these lessons got benefits in terms of code quality, security, and maintainability. The rare few applying most or all of them, mainly high assurance community, got results along lines of space shuttle control code with extremely, low defects or zero in production. So, given the past and present _results_ of these methods every time they're put to the test, I'm irritated every time another person talks like there's no good science to quality software. I just listed a bunch of it, it's been tested in production as scientific method requires thousands of time, tweaked probably hundreds, and core approaches remained even if tactics got modified.

Now people can feel free to use and improve on the science. CompSci continues to in every area I listed with a chunk of proprietary and FOSS developers using a subset of the techniques. Just need more uptake. Use what's proven. And do note that there's plenty of examples for specific design and implementation decisions for common types of functionality. Many things that were shown to work or not work that could be encoded in libraries, DSL's, templates, whatever. No excuse except for our field's continual failure to learn and hand down the lessons from the past.

<comment><AUTHOR> I think we agree more than you think ;) In my original article, I quoted some papers and studies. And on twitter, somebody occused me that "All science is crap". So I tried to write another article with only _my_ opinion on the subject - The article linked here on HN.

<comment><USER3> Reading it instead of skimming it shows there's indeed plenty of agreement even without the science part. The minimizing f_n reflects well what the science established.

Far as defects, problems people report are definitely a start. Fagan Software Inspection Process had the nice idea of standardizing in a list the kind of issues in code that would be treated as defects and fixed by priority. They could be lack of a bounds check, code style issue, whatever team thought was important. So, my definition for defect in such a scheme was "unacceptable code."

"Low defect potential." Interesting metric. Often a function of complexity, coupling, amount of state being thrown around, and handling of interface assumptions. Scientists should come up with ways of assessing stuff like that then see what effect it had on defects during changes among many codebases. Of course, the minimizing f_n stuff should help here.

"When you find a defect in production a week after deploying the code, it is probably still rather cheap to fix: All the original developers are still there, they still remember what they did, the documentation is still accurate."

That's actually a really good point. I don't remember seeing anyone else say it. I'll have to remember that.

"Well crafted code is self documenting. When I read the code together with it's tests, I want to be able to understand what is going on. Without reading some external documentation (wikis, ...)"

Well-crafted code section is overall good but I'm not sure about this. I've seen code that can work like this. Yet, I've also seen high assurance work where there were abstract specs of what it does w/ pre and post conditions that were easier to understand for the _use_ rather than modification of a module. Could also be fed into analysis tools. Some modern languages include that stuff in the code itself and can export interface documentation.

That's not even getting into issues like consistently using a filesystem where all kinds of extra commands are needed for non-obvious reasons that need a whole article to describe. A 1 line operation suddenly takes more like 10. So, I'd say make the code self-documenting where possible but there's potentially good reasons to have the other things. If they're needed or useful.

"Good software design is, in my opinion, less subjective than some other terms I described above."

I agree. You did alright explaining that.

"It's Not Much Slower Anyway The Microsoft study on TDD"

You're conflating quality and testing here. They're not the same. There's plenty of low-quality software with lots of testing. Likewise, Cleanroom (extreme example) normally resulted in high quality from app/code structure and verification via code revies before testing. Same with Fagan inspections and OpenBSD's auditing process. All of those had vastly higher quality than TDD projects far as I've seen. And that's without arguing against testing, as you know I'm for it.

Maybe renaming anything like that in your articles to specifically focus on benefit or cost of testing. So, this would be "TDD/Testing is not much slower anyway." The arguments you made for it are good and actually apply to all quality techniques. I think you intended that but set it up wrt to testing. So, maybe it's not the title and just something about that section in particular. Sounds nitpicking but my reaction is more that we need to always convey quality as a combo of prevention, code review, and testing. Code review & testing at the least. Gotta keep putting it into their heads until they can't forget. ;)

"Let's look back to "Minimizing F_n (For Large n)":"

"Executable specifications (a part of the test suite) make sure everybody understands the current functionality that is implemented. And that "documentation" cannot be outdated, because otherwise the tests would fail."

All good advice but I counter that this long-running problem is a management and SCM/VCS problem. One can implement a policy and rule that any change to code requires an update of the documentation. Then enforce that. A test suite is also a good idea but not a requirement for dealing with this problem. Not even enough as other verification methods (eg DbyC, static analysis) were needed to catch problems in this area that testing misses.

"At some point, your users will recognize that it takes longer and longer until they get software that contains the new features they requested. And they have to pay more and more for it."

A really, good point that few mention or argue satisfactorily. Should be one of the top points. Need more ways to illustrate this in numbers or visually so less technical people get it. Maybe just delivery time and cost themselves as a graph going up over time. ;)

"putting it all together"

Strong, sound conclusions. The science supports it up to a certain quality point where ROI diminishes. You're advocating a realistic one, though.

"A former colleague once said TDD would not make sense for them "because our iOS app only consists of a user interface and some server calls, and you cannot really test those. There is no logic in between. And it's really easy to test manually." Well, maybe you can get away with it when you have an app like that."

You can counter that software is rarely that simple. There's assumptions built in to various component, failure modes to account for (esp network), and effects of state build-up if it's isn't fully stateful. Hell, tell them a non- bloated web server is "simple" with an "interface and some server calls." Then show them the release notes detailing all the bug fixes. Murphy's Law says the "Simple thing are always hard." Not always, but often enough to justify verification activities.

Just popped in my head that it also helps when platform (eg iOS) breaks something with a change and you didn't know about it. Proper tests can isolate that immediately.

"Also, your actions now will come back and bite you later."

Very important. Most "short-term" jobs end up lasting a long time and even being reused in some way. Besides, getting in habit of doing it right makes doing it right more effortless. The delay between short-term and long-term should get smaller as verification activities become natural. I also like "unlimited downside" concept haha.

"And in many organizations, you will have a hard time to argue throwing away "perfectly working code" when the time pressure gets bad..."

Always worth remembering, too.

Overall, when I'm more thorough in the review, you have a good write-up with only a few things I take issue with. As you predicted. :)

<comment><AUTHOR> Thank you so much for your detailed reply! That's a lot to think about :)

Thanks for the hints about defects. I like "unacceptable code", but it's probably still a definition not everyone can live with :)

"It's Not Much Slower Anyway The Microsoft study on TDD"

"You're conflating quality and testing here. They're not the same."

You are right. My arguments here are only for TDD, and I actually did not want to write a TDD article :) I'll think about how to improve that section. Also, CleanRoom is really interesting, but I don't know very much about it and never experienced it.

"At some point, your users will recognize that it takes longer and longer..." "Should be one of the top points."

I was thinking about the order of the points a lot, and no particular order seemed _really_ right. It now comes at a point where I think I have established all the preconditions: Speed means lead time, cost depends on speed, high external quality means faster, high internal quality means faster. But you're right, the main points I want to bring across come very late in the article. As for the illustrations: Maybe I'll write another article about how users care about internal quality - Because I've worked in so many legacy code projects that I think I have a good view on that :)

I like your points about the iOS stuff and the short-term jobs. I'll think about them some more. I'm still not sure if I believe the "unlimited downside" idea 100% (because only very few things are truly unlimited), but it's an interesting concept.

<comment><USER3> " Thank you so much for your detailed reply! That's a lot to think about :)"

You've very welcome. :)

"Thanks for the hints about defects. I like "unacceptable code", but it's probably still a definition not everyone can live with :)"

I kind of came up with it off the top of my head. Point is that the code will be rejected on relatively arbitrary criteria. It might be known defects, coding style, what libraries were used... anything. Hard to find a word or phrase covering all that which people will agree with.

"Also, CleanRoom is really interesting, but I don't know very much about it and never experienced it."

This presentation...

[http://groups.engin.umd.umich.edu/CIS/course.des/cis376/ppt/...](http://groups.engin.umd.umich.edu/CIS/course.des/cis376/ppt/lec21.ppt)

...summarizes it for you in about 5 min of reading. The first attempts to make software in a scientific way, engineered software I call it, were done by Margaret Hamilton (founder of our field) at NASA plus academics like Dijkstra and Hoare. They all produced nearly defect-free systems in a systematic, often mathematical way. Harlan Mills took the lessons to industry when developing Cleanroom: a combo of lightweight formal methods, design constraints, spec/code verification, incremental development, and usage-driven testing. He wanted SW products to be as low-defect as HW clean rooms. Result was code at extremely low defect rate that was consistent enough for statistical certification. While popular, Cleanroom teams actually _warrantied_ their software.

Altran/Praxis is another company I know that (a) truly engineers software and (b) warranties it at specific defect rates. Defect rates are usually better than the Linux kernel and close to the Space Shuttle's control code. Here's a case study of their Correct by Construction process applied to high-assurance, certificate authority:

[http://www.anthonyhall.org/c_by_c_secure_system.pdf](http://www.anthonyhall.org/c_by_c_secure_system.pdf)

So, there's you two methods that already maxed out quality (and/or security) on real-world projects. Altran charges an acceptable premium for quality but Cleanroom often cost nothing extra or saved money for same reasons as you article says. Appears the modern programmers still have lessons to learn in software quality from the 1980's. Like I tell people, the old wisdom is often pretty solid even after you fix outdated parts. Our field is just terrible at passing it down. I try to fix that with posts like this so the smart, young people get a boost toward whatever great things they create. Also reduce endless rediscovering of fire and reinventing the wheel. :)

"I was thinking about the order of the points a lot, and no particular order seemed really right. It now comes at a point where I think I have established all the preconditions: Speed means lead time, cost depends on speed, high external quality means faster, high internal quality means faster. "

I'm going to let you keep working on that formula. You've got the right ingredients. Far as order, there might not be any. Many things in engineering or business are holistic where the parts all feed into each other. The goal is an emergent property of that. So, rather than an order, you'd visualize it like a bubble chart or something that with connections between the components and labels for the effect they have. Just something that's clear that they're all connected, all feed into each other, and ignoring one breaks the process for achieving the goal. Just my dos centimos.

"Maybe I'll write another article about how users care about internal quality - Because I've worked in so many legacy code projects that I think I have a good view on that :)"

You should. Matter of fact, you should do a lot more work on figuring out how to measure and convey the importance of eliminating technical debt. To laypersons, not geeks. Not enough work here despite its importance. Everyone is instead making arguments for geeks that already get it cuz they're knee deep in the crap or picked the job that didn't have it.

Great attempt here:

[http://tech.ticketmaster.com/2015/06/30/what-ticketmaster- is...](http://tech.ticketmaster.com/2015/06/30/what-ticketmaster-is-doing- about-technical-debt/)

Tool for metrics here that was interesting. Not tried it, though.

[http://swreflections.blogspot.com/2012/02/technical-debt- how...](http://swreflections.blogspot.com/2012/02/technical-debt-how-much-is- it-really.html)

" I'm still not sure if I believe the "unlimited downside" idea 100% (because only very few things are truly unlimited), but it's an interesting concept."

I just thought it sounded catchy. Not committing to it myself. ;)</comment></comment></comment></comment></comment>


Ask HN: Offline Reader for HN – Pocket and HN Comments - <AUTHOR> I don't know how other people use hacker news.
What I do is, I "open in new tab" on the "comments" of the titles I am interested in and I look through 2-3 pages. Then I go to each comment page & open the article and read it. The comments help in understanding the counter points & get better understanding.
So, is there a reader like pocket. It should save the article and the comments.
I was unable to find this function in any of the apps around. How does everyone read HN? <comment><USER2> I use Materialistic android app, it can save articles offline for future reading.

</comment><comment><USER3> Hack something up that solves your problem?</comment>


Slow, free range, idle parents can increase IQ and happiness - <AUTHOR> http://faircompanies.com/blogs/view/slow-free-range-idle-parents-can-increase-iq-and-happiness/

<comment><USER2> An interesting insight on this in _Freakonomics_ is that things we _do_ with our children don't affect their long-term development as strongly as things we _are_.

Take the effort to make yourself whatever kind of person you want your kids to grow up to be, and they are more likely to grow up into that person.

For example reading 50 books to your toddler is good for short-term amusement followed by teaching your kids that "books are for kids" leading to a belief that they've outgrown "silly books". By contrast letting your toddler see you read what you like teaches your toddler that "books are for grownups" and your kid will naturally model their behavior on yours.

My personal attitude is that I want my children to grow up curious, inquisitive and self-motivated. Supplying external structure can't supply those traits, I have to model them instead.

<comment><USER3> You wouldn't happen to have a link to the study you mention, would you?

<comment><USER2> No. You'll have to read the book to find the reference. Sorry.

</comment></comment></comment><comment><USER4> From the article: "In 1972, 87% of kids who lived within a mile of school walked or biked daily. Today, only 13% of children bike, walk or get themselves to school."

Interesting point. I remember growing up in the middle of a city, playing outside for hours at a time without my parents' supervision when I was 5. Then when I was 7 I was taking public transportation to school or, if the weather was nice, I would walk 5 or 6 blocks home.

But that was decades ago and in a different country. I wonder if in US today my parents would be reported for "child neglect" if they let me do the same things they let me do then.

<comment><USER5> Couple friends of mine let their two daughters, ages 5 and 10, walk to the park alone. A block away (within shouting distance), with a cell phone, and mom home and ready to come get them if there was any trouble.

Their ex-roommate called Child Protective Services on them.

They were eventually cleared, but the whole experience was a hugely stressful nightmare for them, with multiple visits by social workers and threats to take their children away. So yeah, your parents would probably be reported for child neglect if they let you roam free like that.

<comment><USER4> That is really sad and disturbing. Somehow I think most people nowadays would side with the person that did the reporting. I wish people watched less TV hype and instead used their heads to critically assess risks and dangers.

</comment></comment></comment><comment><USER6> Thoreau on Goethe (_A Week on the Concord and Merrimack Rivers_, Thursday):

"He was even too _well-bred_ to be thoroughly bred. He says that he had had no intercourse with the lowest class of his towns-boys. The child should have the advantage of ignorance as well as knowledge, and is fortunate if he gets his share of neglect and exposure."

But one should not minimize the level of surveillance that was there for the boomers. A lot of mothers did not work, and though playtime was not micromanaged, it was not that easy to get away with serious mischief.

</comment><comment><USER7> Seems fairly uncontroversial to me, but I'm not in the target demographic yet.

Can anyone enlighten me as to why reading Kipling to a child might be considered "recklessness" these days?

<comment><USER8> For some parents not enrolling their kids in every possible class (from sports to arts) or encouraging their participation in debate or math clubs is somehow failing to prepare them adequately for a competitive college/job market. The idea that actually stepping back and saying "just one extracurricular per year" might actually be better for your kids is controversial for "helicopter parents".

As for Kipling, I would assume the reference I took (from another critic) was referring to either Kipling's more adult books or reading the Jungle Book to very young kids. As a parent of a 2-year-old who is tired of reading and re- reading most of the very simple books in the young kids' section of the library (I agree with gvb regarding many modern storybooks), I'm inspired to pull out something more mature that I might enjoy more as well.

<comment><USER9> I'm at one far end of the spectrum, but I was reading Stephen King in 5th/6th grade, read Anna Karenina (chosen myself) in 7th grade for a book report and had read nearly all of Tolstoy's works by the end of 9th grade. My parents never had a punishment for failure, only failures without effort. The idea was to set your goals so high that they cannot possibly be accomplished. It still holds to today for me...when I don't reach the goals I set for myself, I'm not disappointed, I look at how good I actually did do. The only time this is an issue is when coming up with those annual and semi-annual review goals that determine are used to measure if you're going to get a reasonable merit increase or not.</comment></comment></comment>


Freeland – own virtual jurisdiction with brand-new society [pdf] - <AUTHOR> https://mfcoin.net/files/white-paper/white-paper-eng.pdf <comment><USER2> so another cryptocoin fork</comment>


The coming onslaught of iPad competitors--32 of 'em. - <AUTHOR> http://technologizer.com/2010/08/12/ipad-alternatives/

<comment><USER2> So start by crossing anything with windows 7 off of the list.

The most interesting to my mind is the Cisco one. They seem to be targeting the enterprise - I can imagine office's full of people carrying these. RIM did awfully well in this niche. Could Cisco do the same?

I am really really hoping that HP don't mess up their purchase of Palm and deliver something fantastic with WebOS too.

<comment><USER3> I guess it depends what you mean by interesting. I'm not convinced that anyone targeting the enterprise is ever going to be able to reproduce the quality of UI that apple has achieved over the past few years.

Android has proved a huge disappointment to me because their customers have actually turned out to be enterprises (telcos customising and bundling crappy out of date versions) rather than selling direct to end users. I guess Cisco might do well in some verticals, but I'm not getting excited about it just yet.

(agreed that Windows 7 is a dead end for pads)

</comment></comment><comment><USER4> The are others. What strikes me is that none of them realy compete on the key feature of the iPad: its autonomy.

<comment><USER3> What do you mean by autonomy? The big drawback of the ipad is that you do still need a mac/pc to sync and backup — mobileme is nice as far as it goes, but the ipad isn't standalone just yet.

<comment><USER4> I mean the battery lifetime. I use my iPad a lot, like now for instance. Mostly to surf the web. I don't really need backup. For me It is an interactive media.

I don't expect these other devices to beat the iPad on this aspect soon. I heard HTC is planning to increase their CPU speed. For me autonomy (energy) is much more valuable.

</comment></comment></comment><comment><USER5> Where's the Meego tablets?

</comment><comment><USER6> Extremely uninspiring.</comment>


HMS Terror's 'incredible' condition may offer clues to lost Franklin Expedition - <AUTHOR> https://www.cbc.ca/news/canada/terror-wreck-exploration-franklin-expedition-mystery-1.5252023 <comment><USER2> It makes me wonder how difficult it can be to do diving work in places like the north of Nunavut. Has anyone in Ycombinator done something similar? How is it like?</comment>


"Say your words" - Globally Scalable Social News System - <AUTHOR> http://nwzpaper.com/articleView?articleId=1

<comment><USER2> If I understand correctly, you are proposing a single globally scalable system that provides everyone a platform to project their voice. This has implications for every business in every industry not just the ailing news industry. As a business owner I can now directly engage in public discussion without having to pander to some news outlet.

</comment><comment><AUTHOR> The editor and publisher have left the building!

</comment><comment><USER2> This is it...exactly what the industry needs!

<comment><AUTHOR> What industry is that?</comment></comment>