Was Knuth Really Framed? - <AUTHOR> https://www.spinellis.gr/blog/20200225/ <comment><USER2> Somebody, somewhere, wrote the Text::LevenshteinXS module. Somebody, somewhere, had to write sed, awk, head, sort, tr...

It's all fine and dandy to say "look at these awesome tools that make tasks like these trivially easy. See how powerful Unix is". But this fails to consider that somebody, somewhere, has to be a tool _writer_ , not just a tool user. Knuth's code was a tool writer's code, exemplifying a technique (Literate Programming) that is aimed at long form code writers in general.

As with others before, the author fails to grasp that this is an apples to oranges comparison.

<comment><USER3> Just like apples and oranges can be compared as foods (and fruits), bespoke vs. reused solutions can be compared so software. The reused parts are of higher quality than their corresponding bespoke implementations _and_ can be composed to accomplish the same task as well as many others. It's a powerful lesson.

<comment><USER4> > The reused parts are of higher quality than their corresponding bespoke > implementations ….

I think that stating this as an inevitable and uncontested fact might be pushing the point.

<comment><USER3> Most of the components are multi-platform, partly or fully POSIX-standardized, battle-tested, blazing, fast, etc. It's some of the most widely-used and arguably greatest software ever written.

<comment><USER5> And some of it is absolutely terrible with several tools overlapping in use cases along with unclear perfornance hints. Even worse is the combination performing worse which may not matter on small inputs but has a huge impact when you crank up the size. Then there's the issue of understanding why performance suffers and in what cases it's better to roll your own custom solution that better fits the problem.

Reuse saves work when your problem maps perfectly to existing tools while what may seem like a minor difference will propagate down your program to end in countless issues from a codebase you don't know.

</comment></comment></comment></comment></comment><comment><USER6> I'll avoid repeating my comments from the previous thread ([https://news.ycombinator.com/item?id=22406070](https://news.ycombinator.com/item?id=22406070)) and say just the ones that I think didn't get enough attention:

1\. I can't speak for Hillel Wayne who used the word "framed", but I didn't understand his newsletter post as _Bentley_ having "framed" Knuth -- I understood his post as pointing out that in the popular imagination/folklore, the story had mutated over the course of years from the original setting (a program that Knuth was asked to write in WEB specifically as that was the point, and a review of that program by McIlroy evangelizing the then little- known Unix philosophy) to a "framing" where two people were competing to solve the same problem with the same available resources, and one of them did it in a "worse" way. (Also left this comment on the blog post above.)

2\. Here's a comment on the previous thread from someone who says they read the column when it was posted, and their reaction they say was one of cringing -- so at least at that time it probably wasn't perceived that way: [https://news.ycombinator.com/item?id=22418721](https://news.ycombinator.com/item?id=22418721)

3\. Much of the space taken by the literate program is for explaining a very interesting data structure that we could call a hash-packed trie (AFAICT, devised on that occasion for that problem -- a small twist on the packed tries used in TeX for hyphenation, and described in the thesis of Knuth's student Frank Liang). One cannot obtain this data structure by combining other programs, only by combining other ideas. (I mentioned this in the previous thread as well: [https://news.ycombinator.com/item?id=22413391](https://news.ycombinator.com/item?id=22413391))

4\. So as far as evaluating literate programming goes, the real question (and the answer is not obvious to me!) is: if you're going to write a program that uses a custom data structure (like this), _how_ should you organize that program? Should you write it as Knuth does, or as a conventional program (like I tried to do with my translation: [https://codegolf.stackexchange.com/a/197870](https://codegolf.stackexchange.com/a/197870))? And as for estimating the value of a new data structure in the first place: as of now (at that question), solutions based on a trie are about 200 times faster than the shell pipeline, on a large testcase. (The hash-packed trie, which Knuth calls "slow" in his program, is not so bad either, and it does economize on memory a bit.)

<comment><USER7> I have my own answer for #4 (which, to me, is the only interesting question about this affair). I've actually done a fair amount of literate programming on my own, although I only have a couple of examples that one can look at these days. Here is a small library for fluent matcher system for Jasmine and React: [https://github.com/ygt-mikekchar/react-maybe- matchers/blob/m...](https://github.com/ygt-mikekchar/react-maybe- matchers/blob/master/src/ReactMaybeMatchers.litcoffee)

You will see that I've included yet another monad tutorial :-) I don't link to this as a way of saying that I think this is a good example of LP. It's not really. I was experimenting quite a lot. However, I can tell you one thing about it: it is practically impossible to refactor.

As a result, I decided that LP is not particularly good for working on living programs. Or, at least, it is not conducive to my style of programming, which encourages refactoring. Nothing I write is "frozen". It is _all_ in flux and so the value of documentation is transient. Additionally, it is rare that a programmer wishes to read code from the top to the bottom. If they ever do, it's usually the first time they have read the code. After that, they will want fast access to the parts that they want to modify. Sorting out the code from the text becomes difficult. If you make a change, you also have to review _all_ of the text to make sure that you haven't clobber something that is referenced elsewhere. It will work well for something short, but it's not great for large projects.

I still do LP style things. Here is an unfinished blog post on ideas about OO: [https://github.com/ygt- mikekchar/oojs/blob/master/oojs.org](https://github.com/ygt- mikekchar/oojs/blob/master/oojs.org) However, to contrast with this, I would invite you to look at [https://gitlab.com/mikekchar/testy](https://gitlab.com/mikekchar/testy) where I put some of those ideas into action (especially see the design.md and coding_standard.md documents to show what constraints I chose in this experiment). Crucially, after this code had run its course, I'd changed a lot of my ideas and never went back to my blog post. For me, the actual code is far more instructive than the blog post ever was. Of course, I'm the author, so I understand what I was trying to say and I only need a quick peek at the code to remind me what I was thinking.

For me, that's the dilemma of LP: once you know what you want to know, the text is in the way. New people will benefit from the Gentle Introduction (sorry, couldn't resist the TeX reference...), but 99% of the time nobody will benefit from it. Is the other 1% of the time worth it. It may be, actually, but boy is it hard to convince yourself of that!

<comment><USER8> > Here is a small library for fluent matcher system

It doesn't seem like Literate CoffeeScript lets you reorder the code blocks which, as I understand things, is the fundamental part of Literate Programming - code follows documentation, not the other way around.

(Although, to be fair, 99% of things I've seen labelled as LP aren't either. There's only WEB, CWEB, and noweb I can think of that'd count just now.)

<comment><USER7> Yes you are absolutely correct and it's definitely a big problem from an LP perspective. Babel gives me a bit more leeway, though. But from the perspective of "is this worth it", not having it reordered actually makes it _easier_ to work with, in my opinion. I think if you had tools that allowed you to work with the generated code and be able to jump back and forth to the sources it might be OK.

</comment></comment></comment></comment><comment><USER9> > perl -a -MText::LevenshteinXS -e 'print distance(@F), "\n"'

Step 2: draw the rest of the owl

<comment><USER10> The rebuttal itself is a lot more comical than any joke: literally countering Knuth's "code should read and flow like prose for full understanding" with "f you, look at what I can do with my 20 years of hacking and thousands of lines of code I've never seen, written by someone else".

</comment></comment><comment><USER11> I really like the _idea_ of the Unix model as well, but you're not going to be able to use it effectively to write an actual application. If you're writing a word processor and you need to find the levenshtein distance between the most frequent word pairs (maybe some measure of how alliterative/consonant/assonant your document is?) then you're probably not going to be building the word processor using the Unix model, and even if you are (the closest you can get is probably using Tcl/Tk?) then it's still best to write out what you're doing as clear as possible. Note that it took me about 5 minutes to figure out what the shell pipeline presented in the article actually does, and multiple times my reasoning about it lead me to think "wait, does this actually do what it's supposed to do?"

<comment><USER12> acme could be seen as a word processor in the unix model. It's easy to pipe text to composable commands.

I can put the Levenshtein perl one-liner in the blue bar and middle click it to get the distance of two highlighted words.

<comment><USER13> ACME is a word processor in the Oberon System model, which has very little to do with UNIX model and plenty with Xerox GUI based REPL environments.

</comment></comment></comment><comment><USER14> Thank you for adding to the considerable weight of literature completely missing the point of Dr. Knuth's literate software, which is:

tr, sort, uniq, and sed, should all be literate programs.

They would be easier to read, reason about, modify, and extend. At this point, tooling for literate programming lags considerably compared to illiterate programming, and that's entirely because of the determination to miss the point exhibited here.

Too bad really.

<comment><USER15> As I wrote in other thread - try to read the TeX source

[https://mirror- hk.koddos.net/CTAN/systems/knuth/dist/tex/tex...](https://mirror- hk.koddos.net/CTAN/systems/knuth/dist/tex/tex.web)

and compare it with coreutils source code

[https://github.com/coreutils/coreutils/blob/master/src/tr.c](https://github.com/coreutils/coreutils/blob/master/src/tr.c)

what's easier to read and understand?

<comment><USER14> There are several orders of magnitude in complexity between tr and TeX, making such a comparison fruitless.

That said, have a gander here: [http://brokestream.com/tex.pdf](http://brokestream.com/tex.pdf)

Not nearly as good as the hardcover, which has a proper table of contents and index.

The key is to imagine 40 years of progress along these lines. I can't imagine our default target would be paper.

<comment><USER16> The PDF still doesn't help much. The expositionary style of breaking out inner code blocks from their call site harms the ability to understand what's happening. It's nearly impossible to follow in the raw source. Hyperlinks don't improve matters much and the PDF rendering doesn't have rational layout for details like numeric tables.

Try unraveling the numeric code in Metafont:

[http://tug.ctan.org/tex- archive/systems/knuth/dist/mf/mf.web](http://tug.ctan.org/tex- archive/systems/knuth/dist/mf/mf.web)

[http://www.tug.org/texlive//devsrc/Master/texmf- dist/doc/gen...](http://www.tug.org/texlive//devsrc/Master/texmf- dist/doc/generic/knuth/mf/mf.pdf)

</comment></comment></comment></comment><comment><USER17> BTW, Spinellis MOOC about unix tools must be great: [https://www.edx.org/course/unix-tools-data-software-and- prod...](https://www.edx.org/course/unix-tools-data-software-and-production- engineering)

I'm really a fan of Spinellis, his books are excellent: [https://www.spinellis.gr/pubs/index.html#book](https://www.spinellis.gr/pubs/index.html#book)

Effective Debugging book is a must read for any software developer. The Elements of Computing Style is useful for any knowledge worker. Code Reading is probably the only important book in the subject.

The only problem with his books is that they are rather expensive, specially for developers who doesn't earn in dollars :-(

<comment><AUTHOR> That is indeed a pity. I try to compensate by making as much material as possible openly available, such as through the MOOC you mentioned (I've been working for five years on it), through my blog, and through open source software and content.

</comment></comment><comment><USER18> He, self admittedly, slightly cheated by evoking perl to do the "hard" bit.

TFA is not so much about Knuth but mostly about unix being highly capable.

</comment><comment><USER19> Yes he was. The modification to the problem being discussed here is really besides the point.

</comment><comment><USER20> If I can use Perl, I can do it in one line of bash.

<comment><USER21> The author did it in one line of Perl, using an existing library. How is that different from using awk? Yes awk is widely deployed but so is CPAN. In any case deployment isn't part of the argument for using the UNIX philosophy.

<comment> 08-15 What exactly is the UNIX philosophy?

As far as I can see, it's roughly "Data structures are hard, so let's pretend everything is ACSII text. Now we can use a really difficult systems programming language (C) to build functions with weird calling conventions ("tools") and glue them together with an awful scripting language (sh)."

'awk' fits into this framework awkwardly. It implements a restricted pattern (go line-by-line, match actions to lines), it doesn't want to be a full programming language, even though it really is.

But 'perl' is a programming language, and it wants to be one. Once you have 'perl', what is the point of using a reasonable scripting language (perl) to build functions with weird calling conventions and gluing them together with an awful scripting language? You're better off writing functions(!) with normal calling conventions (a library) and gluing them together using the good scripting language.

That logic taken taken to its conclusion replaces the shell with a clean language, encourages libraries instead of "tools", and embeds the 'awk' pattern into said language instead of relegating it to an incomplete secondary scripting language. In one word: 'scsh'.

<comment><USER22> > What exactly is the UNIX philosophy?

I believe it is the idea of writing small tools focused on doing one thing well with reusability in mind as opposed to writing larger complicated tools that do multiple things.

[https://en.wikipedia.org/wiki/Unix_philosophy](https://en.wikipedia.org/wiki/Unix_philosophy)

<comment><USER13> A cargo cult philosophy never adopted by commercial UNIX clones and adored by UNIX FOSS, where the man page of each GNU tool describing the available set of command line arguments looks like a kitchen sink.

</comment></comment></comment></comment></comment><comment><USER23> The previous episode: [https://news.ycombinator.com/item?id=22406070](https://news.ycombinator.com/item?id=22406070).

<comment><USER24> According to a sibling comment this is the follow up to this post. Sibling comment here:

[https://news.ycombinator.com/item?id=22436816](https://news.ycombinator.com/item?id=22436816)

<comment><USER23> Sorry, I'm afraid I don't understand.

</comment></comment></comment><comment><USER13> > In my everyday work, I use Unix commands many times daily to perform diverse > and very different tasks. I very rarely encounter tasks that cannot be > solved by joining together a couple of commands.

Others just use a REPL instead, where tr, sort, uniq, and sed get to be function calls with a threading macro.

<comment><USER25> Meh, a shell is just a REPL and a pipe is just a threading macro. What's your point?

<comment><USER13> The UNIX shell is a primitive REPL, without the capabilities of the REPLs developed at Xerox PARC, TI and Genera, regarding structured data, debugging tools, function composition, inline graphics, ability to directly interact with OS APIs.

A chariot in the age of cars.

<comment><USER25> A car without infrastructure is just a fancy box. A chariot without infrastructure is a rideable horse.

A rideable horse in the age of broken, disparate, infrastructure.

But, at the end of the day, it all depends on what you're trying to accomplish. I use repls, shells, notebooks, etc, on a regular basis. Unix tools solve some problems. Repls solve other problems. Notebooks another. What's important, to me, is to be able to be able to make the most out of them all, despite their flaws, because they're simply the tools that we have in our toolchain. It would be a shame to not learn our own tools, when they can offer us so, so much.

</comment></comment></comment></comment><comment><USER26> Everyone knows Donald Knuth would have finished TAOCP faster if he only understood how to use UNIX pipes.

</comment><comment><USER27> I think its entirely fair to say that Knuth was in a frame to demonstrate one thing, and implement something second. That he was ideating about the subject didn't - necessarily - prevent a successful implementation.

Certainly, as another set of eyes, the lower character count matters most, though.

</comment><comment><USER28> I want to read the follow up article where the challenge is to create a typesetted document. Bentleys criticism includes a single like Shell script invoking Latex.

</comment><comment> jagged-chisel "Through this demonstration I haven't proven that Bentley didn't frame Knuth; it seems that at some point McIlroy admitted that the criticism was unfair."

</comment><comment><USER29> I feel like 'by Jon Bentley' should be restored to the end of the title to help differentiate it from the earlier posting to which this is a reply.

</comment><comment><USER30> To me the question doesn't depend so much on whether Knuth was "framed".

The meaningful criticism leveled at Knuth's code was that it was monolithic. It's true that it was long because he wrote it from scratch, but that's not enough to force you to be tightly coupled.

Did Knuth try to make his code reusable? Was it reusable? I think those are the key questions.

<comment><USER31> That's not really a meaningful criticism. As others have pointed out, the point of Knuth's exercise was not to optimally solve the technical problem, but to demonstrate the effectiveness of Literate Programming (or the lack thereof). The technical problem was just a strawman, so that Knuth had a non- trivial program to demonstrate. With this in mind, McIlroy's pipe example isn't a critique of LP at all -- if anything, it was just a distracting advertisement for the Unix style of composing programs in the shell.

What McIlroy could have examined -- and chose not to at the time -- is whether awk, sed, tr, and friends could _themselves_ be written in a literate style, and whether such a rewrite would have achieved the goals that Knuth was setting out for LP.

Knuth could have chosen to break his monolith into multiple, loosely-coupled programs, and then written then all in an LP style. But would that have really made the demonstration any more effective?

<comment><USER30> > Knuth could have chosen to break his monolith into multiple, loosely-coupled > programs, and then written then all in an LP style. But would that have > really made the demonstration any more effective?

I would say yes. Clearly loose-coupling isn't necessary for a program that small. And no, it isn't always optimal.

But I have clear memories of being asked to re-do an intro CS assignment three times because it wasn't in a properly object-oriented style. Modularity is not a necessity all of the time, but it is important sometimes. Demonstrating the potential to write reusable code seems just as important as demonstrating anything else. (If the conclusion is "LP helps clarity, but you can't write libraries", is that even positive?)

As far as I can tell, "this code has only one useful point of entry" was a key part of the anti-LP argument leveled by McIlroy. After all, isn't the goal to demonstrate that LP works for people who aren't Donald Knuth?

<comment><USER31> I think McIlroy missed the mark here. Single points of entry, and tightly- coupled code, might be reasonable criticisms of Knuth's personal style, but I don't see them as inherent limitations of the LP approach. You could write multiple useful, interesting narratives about a library's core elements -- algorithms, data structures, etc., -- and then write a simple appendix documenting the entry points / API. The style itself doesn't have to get in the way of good program structure.

My own critique of LP is really more about the act of writing itself. Many people, programmers included, just aren't skilled at it! Knuth's literate programs are interesting because he's got something interesting to day, and his writing style is engaging. But I wouldn't enjoy having to read (or maintain!) a literate program that was written by a poor writer in a dull, meandering style.

Also, Knuth seems to think that the literate style ought to make us into better programmers, simply _because_ we're writing prose along with the code -- that the combination somehow unlocks a better understanding of the problem, how to solve it, and how to explain it to others. That sounds inspiring, but I'm not sure it's really true in the general case. Perhaps more research is needed to find out. :)

</comment></comment></comment></comment><comment><USER32> There's a careful balance when using tools and libraries. It's obvious that they are a good choice sometimes, but I've been surprised at the number of times where a tool/library that looks like a perfect fit is actually not, and the whole problem needs to be reconsidered and I end up writing a lot of original code.

</comment><comment><USER33> My Unix philosophy to showing people what a web-based 'ls' command looks like is this:

      $ import fs && import util && comstr --nowrap ls | pretty | less

(Works here: [https://dev.lotw.xyz/shell.os](https://dev.lotw.xyz/shell.os))

<comment><AUTHOR> This looks intriguing. Can you please explain what I'm looking at? I feel like Bowman waking up at the end of "2001: A Space Odyssey"

</comment></comment><comment><USER34> The premise was that piping together shell commands was "better engineering" than a computer program that captured the authors thoughts? Which is more robust for debugging, producing diagnostics, error handling and reporting, extending, and code reuse?

Laughable and sad at the same time, because ACM would publish that.

</comment><comment><USER35> > In fact, one of the reasons I sometimes prefer using Perl over Python is > that it's very easy to incorporate into modular Unix tool pipelines. In > contrast, Python encourages the creation of monoliths of the type McIlroy > criticized.

Python's Mario tool makes it easy to use Python code in pipelines.

[https://github.com/python-mario/mario](https://github.com/python-mario/mario)

</comment><comment><USER36> I find it comedic that someone called LaTeX error handling "phenomenally good".

<comment><USER6> It's my comment you're referring to (linked from the post). The full sentence was "BTW, TeX's error handling is phenomenally good IMO; the opposite of the situation with LaTeX." You've reversed the meaning(!) but I stand by my original comment: I invite you to try plain TeX (instead of LaTeX) for a few weeks/months, and see how you feel about the way it handles errors.

Unlike LaTeX, where the (TeX) error messages usually appear arbitrary / incomprehensible / unrelated to what you're doing, in TeX (IMO) all the error messages are very informative and include a lot of information and give you ways to recover from your problem and poke around, get more context, etc. First you'll have to have read a manual (or I recommend _A Beginner 's Book of TeX_ by Seroul and Levy), but my claim is about the user experience in the steady state.

Of course, part of the reason is that LaTeX is much more complicated than the low-level things one may be doing with plain TeX. Another reason is that the LaTeX authors were working with severe constraints, one of which was of their own choosing: their choice of using TeX macros as a "programming language" (which it was never intended to be, and at which is it horrible). Nevertheless, a big part of the reason is that they were trying so hard to make things "easy" for the user in the typical case that they didn't care as much about ways in which things can go wrong and how surprising errors can be.

</comment></comment><comment><USER37> Did Knuth ever reply personally on whether or not he felt he was framed?

</comment><comment><USER38> > In contrast, Python encourages the creation of monoliths of the type McIlroy > criticized.

unfair and gratuitous criticism of python... i have seen and written many small tools you can run using "python -m".

<comment><USER39> As someone who uses both languages extensively, I disagree.

You are right that Python is great for writing small tools that you can run, just like Perl.

But Python does not lend itself to writing them inline in a command line like was done here. Perl not only does, but has a number of useful features specifically added to fit this common use case. 3 of which were used in this example. (-a for autosplit, -M to load a module, and -e to have the code passed as an argument on the command line rather than having to have it saved to a file.)

Secondly, Perl lends itself to being used as a "better shell" while Python does not.

What I mean is that anything that can be written in bash can be trivially rewritten in Perl, and the program that you get tends to be substantially more maintainable if the bash script is at all complex. In such a rewrite there usually isn't a good reason to change the structure of the program and make it into a single Perl program.

By contrast Python has focused on the "One True Way" to do things, and the plumbing work for calling external commands is just verbose enough that a Python rewrite of a bash script is not necessarily better than the bash script. And furthermore it is much more likely that the Python rewrite of the bash script is much better rewritten as a Python script.

The result is that for someone who lives on the Unix command line, Perl integrates into their world better than Python does. If you have never lived on the Unix command line, the objections may sound silly. But spend months typing commands and doing the extra steps that Python requires Every Single Time will get old.

(This is historically not surprising. Perl 1 was focused on generating text reports. Perl 2 moved into being a sysadmin tool. Perl wound up as a web language because it is what all of the sysadmins recommended for text manipulation to people writing early CGI scripts.)

<comment><USER40> I use a few different languages, one of which is Python, and I use the command line a lot, and I agree that Python is too verbose for a lot of the things that I do on the command line. Therefore, Python is not something that I reach for when doing simple tasks involving pipelines and/or file operations.

I have not yet put time into learning Perl. In no small part because I was intimidated by the weirdness of some of the Perl code that I've seen. The terseness that Perl allows, and which I desire, is at once compelling and scary at the same time. For this Perl also has earned the reputation that it "Write Once, Read Never".

But let's assume that I overcome my fear of Perl. Which version of Perl would you recommend that I learn? Perl 5 or the language formerly known as Perl 6?

<comment><USER39> Learn Perl 5. Perl 6 is an interesting research project for future directions that it doesn't look like the programming world will go.

Most of the weirdness of Perl goes away when you read $ as "the", @ as "these" and a hash lookup as "of".

<comment><USER41> That's not exactly fair to Raku. A more fair critique (and keeping with the theme of this thread) is that Raku is less focused on integrating with the Unix command line than it is on tool building putting it closer to Python than Perl(5) in the spectrum of things. This was a specific design influence dating back to the first days of Perl 6, so it makes some sense.

<comment><USER39> _That 's not exactly fair to Raku._

I know that Raku supporters disagree with me, but that has been my considered opinion for several years. And this has been something I put a lot of thought into.

Let me lay out the case.

What are the key ideas invented or promoted in Perl 6 / Raku that people get excited about?

\- Object-oriented programming including generics, roles and multiple dispatch

\- Functional programming primitives, lazy and eager list evaluation, junctions, autothreading and hyperoperators (vector operators)

\- Parallelism, concurrency, and asynchrony including multi-core support

\- Definable grammars for pattern matching and generalized string processing

\- Optional and gradual typing

I got this list from [https://www.raku.org/](https://www.raku.org/). It is what Raku people think is interesting about their own language. (So I don't get to bring up things I really don't like, like twigils.)

Some of these ideas are mainstream, some not. According to Tiobe (yes, not to be taken seriously but it is accurate enough), the top languages today are Java, C, Python, C++ and C#. Let's eliminate from the list of Raku features anything that is supported by at least 2 of them to come up with things that are novel in Raku while not being broadly adopted today. The list gets much shorter.

\- Roles (OO programming)

\- Junctions, autothreading and hyperoperators (functional programming).

\- Definable grammars for pattern matching and generalized string processing

\- Optional and gradual typing

How many of these will be widely adopted by top languages in 25 years? My best estimate is 1. Could be none, could be 2, I'd be shocked if there were 3.

I say opinion, but it is a fairly well educated opinion. Here is my argument about each.

\- Roles. They have been around for some years. The only language where I have seen them used heavily is Perl 5. Nobody else seems excited.

\- Junctions are mostly a bit of syntax around any/all which is pretty convenient already. Autothreading and hyperoperators are a cool sounding way to parallelize stuff, but getting good parallel performance is complex and counterintuitive. I don't think that this is a good approach.

\- Definable grammars are an interesting rethinking of regexes, but parsing is a difficult and specialized problem. I don't see an interesting approach in an unpopular language changing how the world tackles it.

\- Optional and gradual typing sounded great when it made it into the Common Lisp standard. But over 30 years later, only Python supports it of the top 5. And it isn't widely used there. I see nothing about the next 25 years that makes it more compelling than in the last 25. (Though Raku's implementation is far, far better than Perl 5's broken prototype system. But that is damning with faint praise.)

So use Raku if you find it fun. You'll get a view into an alternate universe of might have beens. But I still believe that the ideas that are new to you won't be particularly relevant to the future of programming.

\-----

It is hard at this date to make what a similar list would have been for Perl 5 at a similar stage. People were excited about CPAN. Perl people kind of took TAP unit testing for granted and didn't appreciate exactly how important it was. Perl people liked the improvements in regular expressions but probably couldn't have guessed how influential "perl compatible regular expressions" would become across languages. Ideas we were excited about like "taint mode" went approximately nowhere. And some ideas that Perl helped popularize, like closures, were ones that few Perl programmers realized were actually supported by the language.

However it would be a true shocker if Raku was anywhere near as influential on the programming landscape 25 years from now.

<comment><USER8> > Junctions are mostly a bit of syntax around any/all

A quick look at Raku junctions makes me think they're basically a slightly tarted-up version of Icon's generators and goal-directed execution (which is no bad thing, of course but hardly novel.)

<comment><USER42> Then you definitely did not grok it. What I gather from [https://en.wikipedia.org/wiki/Icon_(programming_language)#Go...](https://en.wikipedia.org/wiki/Icon_\(programming_language\)#Goal- directed_execution) is that Icon's goal directed execution is more like `react whenever` in Raku ([https://docs.raku.org/language/concurrency#whenever](https://docs.raku.org/language/concurrency#whenever))

Junctions autothread. What does that mean? Using a Junction as an ordinary value, will cause execution for each of the eigenstates, and result in another Junction with the result of each of the executions. An example:

        # a simple sub showing its arg, returning twice the value
        sub foo($bar) { say $bar; 2 * $bar }

        # a simple "or" junction
        my $a = 1 | 2 | 3;
        say foo($a);  # executes for each of eigenstates
        # 1
        # 2
        # 3
        # any(2, 4, 6)

Documentation: [https://docs.raku.org/type/Junction](https://docs.raku.org/type/Junction)

<comment><USER43> Multi-threading and Junctions auto-threading are NOT the same thing.

Calling it auto-threading has lead many people to the wrong conclusion.

(It is possible that auto-threading may be done multi-threaded in the future, but it doesn't do it currently.)</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Anti-facial recognition makeup - <AUTHOR> http://www.theatlantic.com/features/archive/2014/07/makeup/374929/

<comment><USER2> A bit offtopic, but i think it's worth mentioning.

This would make a great captcha. Show users images of people wearing that makeup and hair style mixed with other similar images and tell them to select people from ramdom images.

I always thought a good way to build a captcha would be to use the "Uncanny valley" and the great ability humans have for detecting artifacts to create captchas. For instance, show photographs and very realistic paintings of people and make the user to tell them apart. Another example would be to use a photograph and insert somthing drawn by hand in it.

Is this already done? These things are trivial for humans but I doubt computers can do that.

</comment><comment><USER3> If algorithms continue to be confused by patterns painted over one's face, isn't it inevitable that eventually a lot of people will be openly wearing this in the street? I was punk rock, and we walked down the street looking even worse.

edit: also, it's a bit New Romantic/Gothy, although that effect might lessen with better, easier-to-form distraction patterns.</comment>


Ask HN: The three kinds of engineers throughout a company's life - <AUTHOR> I'm trying to find an article (and related comments) that I could have sworn I originally saw on HN. I've tried HN search to no success.<p>From what I remember, it described the three(?) types of engineers that are needed at a company throughout the company's lifetime of startup, growth, mature. Does this sound familiar to anyone? <comment><USER2> It's originally from the book _Accidental Empires_ , but you probably read the summary on CodingHorror:

[https://blog.codinghorror.com/commandos-infantry-and- police/](https://blog.codinghorror.com/commandos-infantry-and-police/)

<comment><AUTHOR> This is not the exact post I was thinking of, but the one I'm thinking of is definitely an extension of this one. Having this post by jeff will definitely help me on my search. Thanks!

</comment></comment><comment><USER3> This one? [https://steveblank.com/2018/11/13/its-not-change-you-fear- it...](https://steveblank.com/2018/11/13/its-not-change-you-fear-its-loss/)

[https://news.ycombinator.com/item?id=18441804](https://news.ycombinator.com/item?id=18441804)</comment>


Screw uniques and pageviews; New metric: Revenue per month - <AUTHOR> http://marcgayle.com/screw-uniques-and-pageviews-new-metric-revenu <comment><USER3> For some types of businesses, pageviews is what counts. Some type of applications need to reach a certain tipping point, after which trying to generate revenue makes a lot more sense. So they need to grow fast and then focus on money later.

Think of it like a blog trying to get big. You don't slap ad-sense on the blog right away. First you grow your audience, get your fans first, then focus on revenue. Focusing on revenue right from the start will likely make you unable to ever grow.

<comment><AUTHOR> Well...I wouldn't classify starting a blog as a startup/viable long term business.

If you are, indeed, starting a digital publishing house then I think you need to come up with a viable/more sustainable business model more quickly than just relying on advertising at some point in the future.

Hacker Monthly is a perfect example of this. They released a few editions, then quickly put up a pay wall.

A blog, in the most generic sense, I think is better served being a 'free' marketing tool. The main focus should be, as 37Signals so eloquently put it, to teach your users and subsequently build an audience.

Edit: I would also like to point out that I don't disagree that there are some businesses that make sense for pageviews/uniques - my main point is that we (the industry) are still suffering from the bad branding caused by the 90s boom & bust. i.e. start a company and figure out revenue later. I am suggesting such a drastic shift, in an attempt to drastically change that mindset. Once that mindset is changed, then we can loosen a bit - <USER2> 'encourage' some models that require pageviews first (but perhaps by that time we won't need to, because more platforms like <http://www.flattr.com> will be mainstream and bloggers can get paid for each article, than relying on advertising).

</comment></comment><comment><USER4> In advertising, there is the concept of "above the line" and "below the line."

Below the line is anything that directly contributes to sales, and above the line is the less measurable stuff, like brand awareness (usually television ads, billboards, etc.).

I can think of a lot of cases where a web site should be measured by its reach rather than its immediate profit.

For example, a web site for a movie would measure its success in terms of trailers watched, time spent playing with interactive features, etc.

Web sites for cars measure success by the number of people who use the configurator, download brochures, or similar activities. It's very hard to tie monthly sales to monthly web visits, because the customer's consideration time is often much longer than a month.

Even a pure e-commerce site can't go on sales alone. Huge numbers of people use Amazon for product research, wish-list building, reading reviews, getting recommendations, etc. Just because they don't buy something every time doesn't mean it's not measurable. It's that kind of activity that cements Amazon's central role in many people's online shopping behavior. The lifetime value of a customer doesn't show up on a monthly sales report.

Determining the value of each touchpoint in a customer journey is an inexact and difficult science. There's an entire industry evolving to try and do that, called Attribution Management. One quite I've heard repeatedly from people in that field is "if you only count the places where money changes hands, you'd dismantle everything and just keep the cash registers."

The idea of a "revenue per month" metric sounds good on the surface, but it misses the entire value chain.

<comment><AUTHOR> Remember I said this is in reference to a website/web app that is a startup. Not as promotional material...or functioning as a loss leader.

Websites for movie trailers = promo material...not a startup.

Web sites for cars = promo material...not a startup.

An e-commerce site has to go primarily off of revenues and profits. Once they focus on revenues and profits, they can then invest in advertising (both above-the-line and below-the-line).

Amazon ran in the red for many years, because it invested heavily in the infrastructure. It raised equity financing, reinvested all revenues and profits and when it went public it raised huge debt.

Over the subsequent years, it paid down that debt slowly but surely with the profits thrown off of operations. Until the point that it has been profitable for the last few years. However, this point illustrates what I was talking about...they focussed on revenues first and then went to profitability shortly after.

<comment><USER4> Sorry, the article referenced doesn't mention startups at all. It just says the web development community should ignore pageviews and only look at monthly revenue numbers.

</comment></comment></comment><comment><USER5> Great realization, I'm happy for you. But now to the next step: are you trying to grow revenue or profit? The strategies are vastly different. When building profit, you need to keep costs as low as possible. When building revenue, you increase costs as much as you can afford in a way that grows revenue as fast as possible, perhaps throwing even the idea of profit out the door.

Focusing on revenue is awesome. Throwing in the extra "and profit" feels like the safe thing to do, but in reality your execution will be completely different with profit in the equation.

<comment><AUTHOR> Martin, I fully agree. I threw it in there to show that the argument isn't just myopic on revenues alone.

The reality is that web apps and sites that generate monthly recurring revenue come in many forms. Either the microprenuer variety (i.e. one man shop) or the larger VC-backed startup (ala dropbox, etc.).

Depending on your situation it really is up to you to decide to swing the pendulum whichever way you want to go. For example, Dropbox would likely focus on growing revenues as fast as possible, sacrificing profits in the short term - and that's a perfectly respectable decision because they have tons of VC cash in the bank and makes total sense.

So that was just shorthand for me saying that once you make that mental shift from advertising/eyeballs/uniques to monthly recurring revenues, you can then decide how you want to tackle the profit question (more now or later).

<comment><USER5> I think the mechanics around how your thinking about growth, revenue, profit and costs affects your execution are among the most "unknown unknowns" among entrepreneurs. But only because this is where my greatest realizations have appeared during the last couple of years. I know now that in reality I knew nothing when I started my company. And unfortunately you mostly learn stuff like this in hindsight.

In conclusion: great post and great topic, keep writing about stuff like this.

<comment><AUTHOR> Thanks for the encouragement. Will definitely keep trying to :)

</comment></comment></comment></comment><comment><USER6> There is no magic single variable for this, you have to use a mixture of different variables and projections to get it right.

The ones you need to track wind up being a mix of the following:

1\. Page views: Track overall growth

2\. Page views per user: Track engagement

3\. Visits: Track overall growth.

4\. Unique visitors: largely useless metric since every tool out there disagrees with each other on the number.

5\. Revenue per user: Track user segmentation.

6\. Revenue per page view: Track infrastructure costs.

7\. Active users: Finer measure of engagement and churn.

8\. User acquisition/retention cost: Track cost of marketing/outreach.

(1) is important if you are selling inventory on CPM.

(2) is needed to get the mix of organic growth with existing users to increase, while keeping a different focus on acquiring new users.

(3) mostly needed for the ad sales pitch. No point getting a million visitors if you can't retain them, but sales needs inventory.

(4) has always been ~10% off between different tools. Can be an indicative measure of engagement.

(5) helps you keep ceilings in place for acceptable cost per user

(6) can help you make decisions on infrastructure where you need more than what you can buy.

(7) possibly the only user-related metric that should matter internally.

(8) can help you understand why bidding on that $5 CPC keyword does not make sense when what you make per user is $3.

</comment><comment><USER7> agreed

if you charge $50 a month, you only need 200 subscribers to hit $10,000/mo, $120,000/yr. And since it's recurring, once you hit that number, you'll continue growing with time.

So sure, it might take you a year to hit 200 paying customers, but with recurring, every month after that you add more people...so at year 5, you'll get to 1000 subscribers, paying $50,000 a month or $600,000/yr...and with most web apps you can support that just by yourself.

And even if you can't, $50K a month, will allow you to recruit about 5 people, so you can scale your workforce as you grow too

</comment><comment><USER8> The salient point being: if your app relies on a subscription model, then pageviews/eyeballs are not very relevant.

<comment><AUTHOR> Not quite. What I was really alluding to is that most web entrepreneurs should look at developing any web site/app that has recurring revenues - not only one off sales.

Web apps happen to be the best option to generate recurring revenue with the highest margins. Another option is selling infrastructure services (hosting, etc.) but those net margins are much lower because the cost of revenues are higher.

<comment><USER9> Do you have a web app?

<comment><AUTHOR> I am building mine now at the moment...so actively thinking about this stuff.

Hoping to launch in the next few weeks. We'll see though.

</comment></comment></comment></comment><comment><USER10> If the website derives most of its revenue from the ad's posted on it, then Uniques and pageviews really matter.

<comment><AUTHOR> I know...which is why I added the editor's note. I, believe, that the industry has to evolve more towards other sustainable business models than just advertising focussed.

The problem I am addressing is that many people, when they think about creating a web app/site, the first biz model they think of is advertising.

What I would love to see is a complete evolution of this mindset.

It is the equivalent of wanting to start a restaurant or any business in the 'real world' with the model of getting a great location and charging advertising from the foot traffic. That's kinda ridiculous.

If we start treating web businesses like real businesses, I think we will see some more very interesting companies develop.

<comment><USER11> Yes & No - even in a subscriber / product model, the Uniques and Pageviews is a gauge of general public interest. Which speaks to product potential.

From there you can split test your landing page, try to up your conversion rate etc...

Revenue per month is great, wonderful, "le raison d'etre" - but - those other metrics are useful for a number of business functions.</comment></comment></comment>


Google Maps (Japan) + "Real-time" Radioactivity Readings - <AUTHOR> http://radioactivity.suminb.com/

<comment><USER2> I wish this chart did a little more to explain the meaning of the units. A big red dot corresponding to 559 nano-greys / hour (the highest reading when I viewed the chart) probably doesn't mean anything to most people.

That rate is about 2.33 x the world average background radiation level (ignoring quality factors and assuming 1 GY = 1 SV) and listing that alongside the raw units would be helpful.

\- <http://en.wikipedia.org/wiki/Background_radiation></comment>


Apple IIa – ROM for the Apple IIe That Compiles Basic (2018) - <AUTHOR> https://www.teamten.com/lawrence/projects/apple2a/ <comment><USER2> The writer wonders if a compiler would fit in 14K of ROM.

I remember, for the Commodore 64, in some old Compute! book, a compiler called "Sprint" if my memory is correct.

This book featured programs you could type in using their program "MLX" (which was a BASIC program you had to type in as well). "Sprint" was one of these. The program was about 6K (it definitely was the biggest program in the book).

I never typed it in. I remember it having limitations, like considering all variables integers and not supporting the DIM statement fully.

I'm not too familiar with BBC BASIC but my understanding was you could mix 6502 opcodes in with BASIC statements and basically do inline assembly. That would seem to be the optimum strategy for old 8-bit computers that would have to store the source and object code in as little as 64K or less of RAM.

\</comment><comment>----

> A NEXT, however, must check each time the stack of FOR statements to find > the variable to increment and the line to jump to.

This is why on most BASICS we always left out the variable, most BASICs would "automatically" loop to the outermost FOR.

<comment> ta-gazette22 Here is a link to Sprint published in Compute's Gazette from January 1986 -- [https://archive.org/details/1986-01-computegazette/page/n73](https://archive.org/details/1986-01-computegazette/page/n73)

Integer and string only, no parentheses (strict left to right evaluation), array sizes were limited to 126 elements and one dimension, but longer variable names. Converting a normal C= 64 program to work with it was daunting.

This issue also has a machine language assembler, which I did use extensively.

</comment></comment><comment><USER3> Regarding Basic compilers, there were plenty to choose from, and relatively fast versus any other high level language, C only started to matter on 8 and 16 bit platforms, after UNIX started to gain market adoption.

Naturally an Amiga was much powerful as an Apple IIa, but stuff like AMOS did wonders on the demoscene.

<comment><USER4> I don't think hardly any demos were made with AMOS.

<comment><USER5> Not sure why you're being downvoted, but I owned a copy of AMOS and AMOS 3D after it came out, and I certainly don't recall anybody using it for much that I recall, other than a few homebrew games and such. Certainly not any demos I ran across.

That said, last week I went on a spree looking up AMOS - and interestingly, I found that the author released it as open source a while back, and recently released a new cross-platform language that supposedly is AMOS compatible (IIRC), and open source too. I think he had a patreon or something to sponsor it's development:

[https://www.amos2.org/](https://www.amos2.org/)

[https://en.wikipedia.org/wiki/AMOS_(programming_language)](https://en.wikipedia.org/wiki/AMOS_\(programming_language\))

Though the wikipedia article mentions a few commercial games that were made with it...

Anyhow - via finding that, and some other searching, I found that there is still an active AMOS community out there. I'm not sure I need another BASIC- ish retro-language in my repertoire, but I find it fascinating nonetheless!

</comment></comment></comment><comment><USER6> Recently I found out John Romero does a podcast where he speaks about Apple II at great length, you can find it at appletimewarp.libsyn.com

Very interesting and entertaining podcast.

<comment><USER7> Very interesting.. Though very infrequently updated.

Snakebyte used low res graphics to do collision detection! I thought about that sometime in the 80s but didn't learn assembler...

Thanks!

[http://appletimewarp.libsyn.com](http://appletimewarp.libsyn.com)

</comment></comment><comment><USER8> Cool. I got started programming in BASIC on the Apple ][ as a kid like a lot of coders my age.

Sometimes I think about buying an old machine just for the nostalgia factor.

</comment><comment><USER9> There were a few compilers for AppleSoft BASIC. The most well-known was Microsoft's BASIC compiler. It could not handle arrays, though.

The original Integer BASIC that Woz & Co wrote was faster than AppleSoft, but lacked much complexity. I never heard of a compiler for it.

</comment><comment><USER10> My computer as a teen. I would literally stay up late reading the monitor ROM listing and thinking Woz was some kind of super hero. Anything Apple did after the ][ seemed like a betrayal to me (although I do own an iPod I love).</comment>


The Facebook Fallacy – Why Facebook Would Fail (2012) - <AUTHOR> https://www.technologyreview.com/s/427972/the-facebook-fallacy/ <comment><USER2> there's still time</comment>


Unseen 9/11 photos bought at house clearance sale - <AUTHOR> https://www.bbc.com/news/technology-48689783 <comment><USER2> Different article about same subject discussed a few days ago:

[https://news.ycombinator.com/item?id=20199667](https://news.ycombinator.com/item?id=20199667) (60 points/59 comments)

<comment><AUTHOR> Good to know, thanks.

</comment></comment><comment><USER3> In this community of logical thinkers, we will each think what we think, and not say much for now...</comment>


Charlie Munger's Last Meeting - <AUTHOR> http://joekusnan.tumblr.com/post/7113195673/charlie-mungers-last-meeting <comment><USER2> On all the strong recommendations I went out and bought Poor Charlie's Almanack, and I must say it was underwhelming: a few general principles are sprinkled sparsely amongst business war stories of varying applicability to todays investor and a bunch of frankly embarrassing tributes to Munger.

Perhaps my expectations were too high, but by the fourth sycophantic letter proclaiming Mungers genius (which I do not dispute) I was throughly off-put.

</comment><comment><USER3> Also interesting: "Academic Economics: Strengths and Faults After Considering Interdisciplinary Needs" [http://www.tilsonfunds.com/MungerUCSBspeech.pdf](http://www.tilsonfunds.com/MungerUCSBspeech.pdf)

</comment><comment><USER4> 2011.</comment>


China passes US in number of startup unicorns:  China 181 vs. US 138 - <AUTHOR> https://www.scmp.com/business/money/wealth/article/2171485/china-overtakes-us-number-new-companies-worth-least-us1 <comment><AUTHOR> But there are headwinds:

Winter is coming for China's private equity as market rout slashes start-up firms' valuations

[https://www.scmp.com/business/companies/article/2171182/wint...](https://www.scmp.com/business/companies/article/2171182/winter- coming-chinas-private-equity-market-rout-slashes-start)

</comment><comment><AUTHOR> Original post title: China passes US in number of startup unicorns: China 181 vs US 138</comment>